The reason is definitely psychological. The fact that you have no problem achieving an erection stimulating yourself manually proves that there is no physical cause. While it's nice to know everything is fine physically...Psychological problems can often be very difficult to understand and overcome. You can be comfortable with a girl, be extremely attracted to her, want to have sex with her, and things just don't work properly. Once this happens it makes it even more difficult the next time because you have this worry that it'll happen again in the back of your mind. It's an endless cycle that has to be broken.
I suggest masturbating less (no more than 2-3 times/week) and finding a partner you're comfortable with to help you get through the problem. The more you think about it the more it'll become a problem. You need to relax and enjoy the moment. Once you get past it you should be fine.
I strongly advise against taking hormones. If you need a pill to get past the problem initially go see a gp and get a prescription for viagra. If it helps you can slowly wean your way off of it by lowering your dose each time you have intercourse until (hopefully) things work without it. This is a method I used and have had some success with.
