LexueBunnie, the best alternative is the one that does not work with the pill - natural family planning or fertility awareness methods. As you are not ovulating, you do not get all the fertility signs like cervical mucus so these methods are no good.
So what is left is pull out, barrier methods and IUDs. The modern IUD contains birth control hormones and are pretty effective. On the barrier side there are condoms which he does not like, the diaphragm which takes a lot of planning to use and the sponge which contains nasty spermicides (nonoxynol-9).
My best suggestion would be for him to check on you taking your pills to get comfortable that you are doing your bit, and to pull out. Pull-out is good when used with birth control pills. If he gets an accident, it is not a train crash. Very far from it. He can also finish inside a week around your bleeding time when you are totally infertile, pill or no pill.
Best of luck!