It is always said that men fall asleep right after sex because the man almost always gets off.
If you orgasm from sex, or just have very long tiring sex then yeah, you are most likely going to be exhausted afterwards. But if you are with a man (or woman) who doesn't satisfy you or only lasts five minutes chances are you aren't going to be as tired as the one who got off is. Or as tired as you would be if you had great workout sex.
So, that is why you mostly hear about men falling asleep right after sex, but it does work both ways in some cases.