I'd say he'll probably be relieved if you know and you're okay with it. Us guys have a lot of pressure on us when it comes to sex, I hate to say, and a huge side-effect of anxiety is some form of erectile dysfunction. That's a really sweeping statement I know, but trust me, being anxious about getting an erection, or having an orgasm usually means you'll have trouble attaining either.
In a typical loving relationship, communication about this sort of thing can be hard, but if you really persevere and try to understand where he's coming from (Medically, I'm assuming there's nothing wrong with him, because he's still able to make sweet love to you, so chances are he's created a pharmaceutical dependence in his mind). If you slowly work at finding out what kind of sex he needs to maintain his erection without the drugs, you could be doing him and your relationship a huge favor.
I'm not speaking out against Viagra, but chances are if he doesn't need the drug other than to put his mind at ease, (It's 20 bones a pill by the way) it's a good thing not to be on it.