It kind of sounds like genital warts (I'm not a doctor). With genital warts, men who have this seem to not show symptoms for years, if they show symptoms at all.
Go get it checked, he could have had an STD without knowing before you were together, or god-forbid, while you were together from someone else.
I think maybe you should both get checked together. Personally, I would go together and get checked for everything, including HIV. It's always good to know.