Ms is such a variable illness that there is no standardized reply to your question. I was not diagnosed with ms until I was 41 myself. Thinking back, I remember one incident of symptoms which "could" have been a symptom of ms but could also have been a symptom of half a dozen other things. Usually if the symptoms come before a diagnosis, they are attributed to other common problems. The only symptom in my past that I can think of that "might" have been due to ms was a severe episode of vertigo; I had to lay flat on the bed as every time I even barely moved, it felt as though my mattress was tipping and I would fall off.
I got someone to take me to the er (such was my insurance at the time) where they merely diagnosed me with an inner-ear virus which of course could cause the same vertigo problems. I didn't give them a second thought once the initial episode cleared up. The symptoms that my doctor did give a second thought to was an episode of total numbness in the entire right half of my face, including scalp, eyeball, skin, inside mouth and down the throat, ear... Everything on the right side. This was probably at least 10 years since the vertigo incident, if not more.
I think the general line of thinking now is that a diagnosis will most likely come between the ages of 20 and 40, but that is by no means hard and fast. I've read where teenagers have had symptoms, and then people over 40 (like myself) are diagnosed for the first time, too.
I've read where optic neuritis is very often the first symptom presented in a case of ms. I have never had it (and i'm so thankful) although during one of my few exacerbations, I had to endure double vision. I believe the disease was much ignored even as recently as 15 years ago because the medical technology just did not exist to confirm or rule out ms. It seems to me that the mri is the most used test to make a diagnosis from, and yet mris were not invented yet 15 years ago.
I do know that the earlier you are diagnosed and go into active treatment, the less permanent disability you are likely to have. I am surprised that your doctor is not taking your situation more seriously, given your presenting symptom of optic neuritis plus your family history.