Ok. Well what time did you take the pill that day when the pills ended?
If its within a six or so hour span, you'll be fine. Some hours don't affect the pill. Say you took it at 8pm rather than 8am, then you'd have to use back up!
Also, you say you didn't use a condom, but he didn't cum in you. Do keep in mind, he does pre cum in you, which can get you pregnant too.
I think the brown discharge is just from your period since you just ended and had sex right after, its just old blood.
Pregnancy spotting wouldn't show up the day or so when you have sex. No pregnancy symptoms would show up that quick.
So relax.
If you only missed your pill by a few hours, and you usally take it regularly and still are, you have nothing to worry about.
There is a slim chance you are pregnant, but its always there since the pill is not 100% fool prove.
Hope that helps.