Itchy burning clit area

I am slightly freaking out! I was sexually inactive for almost 2 years after having baby. Then I wa wih someone about four months ago. We were drunk and did not use protection. It was a one time thing. So four months later I go out with a guy and we used a condom but towards the end it rolled off so there was contact. Well I get cold sores once in a blue moon, I got them some how as an infant. So, I did go down on the guy before having sex. Then mid week after I saw him I got a cold sore in two different locations. It wasn't going away as quick as usual so I called my doc and for the first time ever started taking an oral med to help clear it up. So I have been taking those for a good week or two now. Anyways I was having an itchy vagina just a bit around the clit yesterday. Now I got home this evening and the clit is more itchy and the inner lips are a bit red towards the clit. No odor or discharge just a bit of itch and warm sensation. I am freaking out that I transferred my virus from my lips to him then to me! Is this possible??? And if so though wouldn't nothing happen since I an already in meds for the virus? I had no sores or signs I sores in my lips during the sex act. I know that I would get a cold sore almost every time I would share a dink with this one friend who also got cold sores only worse then me. So I was thinking that te guy I saw maybe got cold sores and irritated my virus causing the ones on my lip to happen. I am hoping his is just a yeast infection or maybe vaginitis. Freaking out though! Getting kind if close with this guy and not sure what to do!