One drink can give the baby fetal alcohol syndrome. But that isn't an every occurance thing!
Some people drink throughout their entire pregnancy and the baby is fine, where some drink for the first month because they don't know and have a baby with f.A.S.
If you are concerned just tell your doc. They can just watch the development closely. You are most likely fine though. I don't mean to worry you, just giving you the facts.
One of my friends didn't know she was pregnant until almost her fourth month. We drank every weekend (if not more) and she smoked regularly. Her son was just fine. It just depends on so many different things, but I wouldn't worry.