It does seem odd. I would see your doctor. I mean, you obviously aren't the type of guy to just accept this and move on. You seem to actually care about your partner's pleasure. So, ask your doctor. Maybe stress is playing a role in this, but you just don't realize how stressed you really are. A lot of times we don't realize how stressed we are until it starts showing in our health.
Also, after it happened that one time, you may just be so worried about it happening again that it does. The whole mind over matter thing.
You could try masturbating an hour or so before you have intercourse, but how often do we really know how to time those things? You could also just go the second time to please her, but as a woman let me just say: I personally won't go that second time if we both weren't satisfied the first time. My reasoning is... First of all I am just too turned off from the first time of not being satisfied, and secondly I don't feel that my partner is enjoying it. I feel like he is doing it just for me (and although respectful and flattering) I cannot get into it because I don't feel he is. Know what I am saying?
I am not trying to be ignorant and offend anyone, but so many people just say to go a second time if you don't last the first time and that just doesn't work for everyone.
Anyway, go see your doctor and good luck.