joeyp,
Surgery does not cut out pain. It is done to correct an anatomical problem. unfortunately, you have found out the problem with herniations. It is taught that the results of surgery vs nonoperative treatment are the same at two years. Presently, in the US, most patients don't want to put up with the symptoms for that long, so they will have surgery.
Another thing is that, a discectomy, to remove the portion of the disc material that has herniated and is pressing on a nerve root, is very good for relieving leg pain, but does not have as good a track record for alleviating back pain. Because the disc itself will still be damaged, the tear in the anulus will still be there.
Thus, unfortunately, you are in that gray area. You might want to speak with the surgeon again, discuss your concerns. Ask what the surgery is supposed to do, exactly. Spine surgery is not without significant risk. But, there are also benefits, for the right surgery for the right problem in the right patient.
Good luck.