It's rather unlikely that you'd get it from your brother unless you two had sexual contact or sexual contact with a mutual partner that had it. its spread by skin to skin contact, and pretty hard to get from a towel that you shared or something like that.
Get it checked, it could be something like Moluscum or something else. Moluscum is very contagious but usually goes away (unlike HPV). Have a doctor check it out.
If it is HPV its possible that you have it all in that area...and it can spread to the other areas you listed. generally it stays in those 3 areas, but I would have them looked at and removed, as they can spread quickly if you leave them. Consider yourself lucky thus far that you don't have them on your rod. When its there, very easy to spread inside a partner. Because the virus is in the skin, you can even spread it without any signs of an outbreak. Its also much more sensitive of an area then where you have them now. Get rid of them asap and start using something like aldara if it'll work for you.