According to the symptoms you reported, you might be experiencing an inner ear infection. It seems that the infection is subacute (not so severe). A bacterial inner infection would likely spread in the surrounding tissue. It might cause a fever and the signs would have been more severe. An acoustic neuroma with very small dimensions could also cause these symptoms, but the symptoms would not withdraw occasionally. If an inner ear viral infection is the exact diagnosis, then there really is not much to be done. You might want to use anti-inflammatory products to ease the inflammation and the symptoms.