I understand what you're going through, trust my word on this. My boyfriend would not allow me to leave PERIOD when he wanted to discuss things, and he absolutely disgusted abusive men. But he would grab me and hold me back, yank me away, pull me. Just yesterday he got seriously physical and never in my life would I have imagined he would get this bad.
There is a possibility that your boyfriend could some day hit you. I know you love him, but you need to seriously think about what you could be getting into and sit down with your boyfriend and discuss things. If he ignores you and brushes it off, he's not serious about the relationship, and I think you should cut it off before it's too late.
This is coming from someone who JUST went through all this, and a fellow girl. I'm 19.