I guess it depends on the brand of birth control you are on. But, as a general rule, if you miss three or more days of the pill you are supposed to toss the pack and start all over again. So I don't think you were properly protected when you had sex.
Have you considered switching birth control methods to something that is easier to remember- the Depo Provera shot (a shot once every three months)
NuvaRing(remove after three weeks/ do not have it for a week to get your period, then insert a new one after seven days)
Mirena (left in for up to three years of protection)
Norplant (inserted under the skin- I'm not very familiar with this so I'm unsure how long you can keep it.)

I am on Tricyclen low. And I usually dont miss it. I keep it by the computer, but since I wasnt going on the computer much, I just missed it.