no im sure you will be fine, i had the same fear.. every minute of the day i would think about this and it would do my head in..
you need a good healthy distraction, after my 1st child i had to pay constant attention to him, then other silly worries went away after about 2months..
The more you think about it the stronger the connections in your brain will become this worrying and the harder it is to get rid of so get ontop of it now.
To start off maybe go to a psychologist so they can reassure you that your fine, which will be a big ease of mind, then if you still worry after that you need to make a habbit of doing something distactive when you worry it will break that default worrying thought...
remember this and you shall be fine..
goodluck