Puss filled Bumps under armpit

I get mini puss filled bumps located under my armpits alot. they kind of look like zits, when popped, they have a whiteish, yellowish tint to them. Occasionally I'll get one that forms to the size of a nickle below the surface and I have to bring it to a head before It can be popped. It hurts. Ive have about 3-4 on my leg but they mainly stay under my arms. I now use antibacterial soap because I have noooo Idea what this is, maybe a staph infection? I went to a dermatologist and was perscribed antibiotics but eventually it came back. Any ideas on what it could possibly be or what I should do to treat it?

-Sometimes theres so many it looks like a rash. there may only be a few. I use a seperate razor for under my arms than I do for the rest of my body. I bathe regularly. I dont share deoderant. Why is this happening?!?