I don't know what you have so I can't comment on that. Do you go to the doctors during a breakout? If not, perhaps you should go when you're actually experiencing the symptoms so that the doctor can see it and tests might be more clear.
Perhaps it's a dermatological issue and a specialist would be of more help.
As for your symptoms not coming when you have frequent sex with your wife, perhaps it is because she has a healthy vagina. Internally it is acidic which can kill bacteria, yeasts, molds, etc. It also may contain probiotics, which are beneficial bacteria, which also act to kill bad bacteria. So perhaps by having sex, you are gaining some of these probiotics or getting an acid treatment of sorts. I don't really know, just odd speculation.
If that is so, it may actually lead you to a solution. The skin is naturally supposed to be slightly acidic. Modern soaps often do not take this into account and create alkaline soaps. So a possible easy step would be to by a pH balanced soap, one that creates slight acidity. You can find these online or in a health store. You can also find probiotic soaps online that would hopefully leave an amount of probiotics on your skin to fight pathogens. Distilled water, which is extremely pure water gotten from steam distillation, is acidic and may be sprayed on after a shower.(which is why it perhaps shouldn't be drinken for long periods of time). Organic Plain (must be plain) Yogurt has probiotics, so external application of that could be helpful. Sesame Seed Oil (organic, cold pressed) could also be beneficial. It is naturally antibacterial and may attract toxins out of the skin, plus it moisturizing and leaves skin beautiful. You'll probably have a hard time finding a good oil in the stores, so google. If you try these external applications, do them for several weeks, not just as needed, perhaps you could prevent it from ever coming back. Also, change your underwear everyday, shower everyday with the new soap and wash your underwear separately in hot water. Perhaps with an anti-pathogen agent such as Manuka Oil or whatever you find through research to be good for killing in laundry.
Then we move on to internal. If it is indeed an infection of sorts, there are a few things that can be done. First, killing the infection with Colloidal Silver, Oregano Oil, Garlic, Zinc, Vitamin C, Astragalus, etc. could help. A naturopath or knowledgeable health store owner could help you pick one out. Zinc, Vitamin C, and Astragalus are great for the immune system so you should have one of those for upkeep, but as a direct infection fighter choose another. After you do that, you'll need to replenish your probiotics. This is best done with a Probiotic supplement that is enteric coated to allow it to get passed the stomach acid. Find help at a health store and follow the directions. It's great for digestion, immunity, and overall health so you can continue to take if forever if you like, but do it for 2-3 months now to make sure it holds. Also, do it again anytime you take antibiotics, conventional or natural such as Colloidal Silver.
If your problem is not dermatological, it may make good sense to treat your urinary system. Kidneys, urinary tract, urethra, bladder. Some of the infection fighting products above will most assuredly work here as well, but there are specific products that do a better job in this area. Of course you've probably heard of Cranberries being used for urinary and bladder infections. Don't drink store bought juice though, it contains sugars, other juices, and is otherwise uneffective. If you want the juice, juice fresh cranberries at home. Otherwise, just eat cranberries or buy a cranberry supplement. You might be able to find unpasteurized, organic syrup. If you can, that might work, but I'm not familiar with it so look it up. D-Mannose is another great product shown to be effective for infections around there. Buy it separately or in a combined product of cranberries. Finally, there are herbs used for cleansing the urinary system and kidneys. Herbs such as Marshmallow Root, Uva Ursi, etc. You can usually find them and others in a join product at any health store.
In the meantime avoid sugars, soft drinks, excessive alcohol as much as you want, and drink plenty of clean water.