I too, have done alot of drugs, and I quit them after realizing they were going to ruin my life very soon. I did not stop smoking weed though, and continued to smoke weed for about 6 months after quitting all hard-drugs, but when I smoked, I didn't feel as good as I did when I used to smoke weed. In fact, I felt horribly anxious when I'd blaze up.
I stopped smoking weed in September, after having a full-blow panic attack, because I had accidentally smoked some weed with PCP in it (stupid dealer..). Ever since then I have had *CONSTANT* anxiety symptoms, and my doctors have done nothing to help me.
In fact, on my last appointment, I did not go, because I have trouble waking up in the morning now, and I also have trouble remembering things (i forgot about the appointment). She said she didn't want to see me anymore, because I had done this before (same reason).
What happened to the hypocratic oath!?
Anyways, as for your anxiety, just don't pay attention to it. When you get a symptom, just put yourself in a smug mindset and think "stupid anxiety". That's what I do, and it usually goes away. Mind over matter I guess.
EDIT:
OH, and also try to figure out what issues are causing the anxiety, and have a nice emotional confrontation with those problems. Then try to find some light in them, or some light in solving them. That's what I'm struggling to do with my anxiety.