Having a Relationship With a Father of a Child

Well im new to this and I was wondering about trusting a man who already has a kid with the mother in which they split up recently. I love this man very much but it is hard for me to trust him with that girl. The kid I understand but there are still doubts in my mind that he could possibly cheat on me. When we are in our realtionship I would liked to be viewed first but I know that it is impossible because he brings up his kid. The mother of the child too, she calls and they talk. I don't say anything but it too gets on my nerves. Sometimes it just hurts. He tells me that I should not go through this and that I deserve better but I love him to much to let him go. Sometimes I get moody when I am told to go home because the mother and kid are gonna stay over at his house for the weekend. It gets on my nerves.. And when I am at home I think about it. It just would hurt and I am deeply confused about it. Now he is not really talking to me and he just recently left without telling me where he is going. I am the type to show my afffection toward him but he told me to stop because it got on his nerves. So im trying.. Is he saying goodbye to me? He says he loves me but that was a long time ago. Today he rarely says that. Im hurt and don't know what to do.