Well, is there a pressure for him to come inside you? Do you get highly upset during sex?
If he feels there is a lot of stress involved with sex, he's probably going to avoid it (or, have extreme difficulties when he is doing it).
How is your relationship in general (sans sex)? Throughout this post, I'm not really getting a sense that you and he are a good couple with one another. He's good for the kids, he's good for the friends, he's good for thehouse...but what about you? You may want to have a serious discussion with him about the status of your relationship (and not imparticular the sex life). The sex life may be a symptom of larger problems.