it is a strong possibility that he gave you herpes. my question to you is have you had past partners perform oral sex on you. if so it is possible that you ex couldve given you this disease.
there is a thing called shedding, which is when a person have the infection active but there is not signs. so if the infection is active on his lips and he touch your vagina, it is possible he gave it to you.
you can have herpes for years and not know. it sits in your nervous system and surfaces when you are stressed out, etc.
some people keep it secret about having this disease because it is hard to tell someone they have a permanent disease.
if you confront him i hope he is going to be mature and not tell your business to anyone.
it's a possibility they he didnt know, or he did know and didnt say anything.

Trust me, I know just what your going through. Not even a year ago I found out that I have herpes. I had only sex with three people, ever. I told my boyfriend, he got tested and his test came back negative. I got really scared and didn't want to tell the other two guys. Finally after waiting about 3 months, my boyfriend convinced me it was the right thing to do. I sent one of the guys a text, he immediately called me back and we had the awkward conversation. But he told me he would go get tested. His results also came back negative. So here I am today, I know that it was probably the first guy I ever had sex with. I'm too scared to tell him too. But I know in my heart that it is the right thing to do so I'll do it. Don't be like me and wait. When I finally got around to telling that guy, he was mad that I had waited so long because the longer you wait, the more chances there are to spread this disease even further.

Just because you had unprotected sex doesn't mean that that was the cause of an STD you have. In fact it's very unlikely you'd show symptoms for an infection that recent. You should treat this as nothing more than sharing information about a positive test with one of your sexual partners. You should really contact all of your sexual partners since your last screening.