Your doctor meant you were 3 weeks gestational, but 5 weeks pregnant. Pregnancy starts the first day of your last period. Although your baby is only 3 weeks old, you're considered 5 weeks pregnant. I hope that makes sense.
An empty sac itself is not really a cause for worry. At 5 weeks your baby is tiny and can be missed on an ultrasound. The bleeding and drop in hCG is troublesome though. Definitely go back for your follow-up blood test.
Good luck.