erectile dyfunction or mental problem?

My BF of several years never had a high libido and would get decently hard only when inside me but by now he almost lost his libido totally and never gets a hard erection unless stimulated strongly and long by hand or orally, and when I stop for a moment he loses it immediately. He is 39 and is in generally good health, (no probs ex. having lost one of his kidneys as a young child). He says he is masturbating daily and all is fine and that I should be patient. So I am, never nag him, try to be sexy, not push him(but we haven't been able to have intercourse for years!) I love him but afraid to marry him for this reason: is it ED? or can he be gay? Or is it that he is imagining/having a completely different woman besides me and rejects me subconsciously?(he says when he masturbates he thinks of always the same person, an 'imaginary woman' he calls her but whose description fits a common friend and which is totally my opposite. (boyish in all aspects, me very womanlike). Please advise!