Hmmm, well you can think that way, or you can think... "not everyone is perfect, yes he's been through it, so he won't make the same mistakes with you and you and he can work out things differently"
He denies it because he loves you. Yes he had to have loved her at the beginning or they would never had been married... but its over and he has moved on and he's with you.
You have got to trust him, believe in him some more. I'm sure you have been through relationships before, just never taken that step.
They don't have children? you didn't say? so there is no connection for them.
You are a unique person and you and he can have your own new beginning. You will be never second best, never think that.. be more positive and all be well.