Sure he says that he shoudlnt' have to shoulder the responsiblity since he didn't want it - but frankly, he wasn't forced to have sex with you, so he has to take responsibility for the part that he played in it. After all, it's a possible side effect of sex right? No matter how careful you are, there is only one sure-fire way to avoid pregnancy.
I would meet with him - perhaps with a third party present to keep things on the cool side (maybe your best friend, sister, family member, etc... Probably a woman would be better, in my experience a man might not like what he has to say and you wouldnt' want it to get violent). At least if nothing else, you have a witness to his stupidity.
I would suggest just telling him that it's his choice to be part of this child's life. You can't force him, and I think you know that. By law he has to pay for that child since he helped create it. So take the money, use it to raise your child and when the baby asks about their father, be honest. Don't be hurtful, but tell the child that their father has been helping with money to buy clothes, food, toys, etc, but wasn't ready to be a father. Maybe one day he'll come around - if not, think of it this way - even if he is an ass, he did do one great thing, he gave you a child that you will love forever.
Sorry you are having such trouble with this.