Did you quit abruptly? Your dose is very low. It's not really enough to produce a rebound effect, but that can be psychological as well. In other words, if you feel that you're anxious, your anxiety will increase. It's a vicious circle.
Benadryl won't do anything to help. The only thing that will help is to go back to the highest dose you were taking. Do this for a week and see if all the symptoms disappear. If they don't, you'll know it's not the Lorazepam. If they do disappear and you feel absolutely normal, then start tapering. Prepare yourself mentally for the tapering. Anticipating quitting entirely is going to make you pre-anxious, as it were, and will not help when you finally stop.
Continue with your max dose for one more week. Then, depending on the structure of your pills (if you can cut them or not) lower your daily dose by a third (ie. 100 MG becomes 66 MG). Continue this for two weeks (you can do this faster, but I sense you need to go very slowly). After that, reduce by another third (ie. if it was 100MG full dose, now 33 MG). After two weeks of this, start with alternate days: one with the third of the max dose, the next with nothing. Continue this for two weeks.
Congratulations. You are now taking too little to possibly have any withdrawal symptoms except for what's in your mind. If anxiety persists, see your doctor. There are other non diazepam-based drugs for anxiety (SSRIs is one) that you can switch to that are non-clinically addictive.
Hope that helps.
Nick (not a doctor, but should be)