Boyfriend not supportive of mental health issues

I'm in a pretty new relationship- only 6 months, but I feel like we're really close and honest with each other and I was convinced that he was the one. I have post-traumatic stress disorder from being gang raped as a teenager and I have night terrors and anxiety attacks. With my boyfriend's encouragement I have sought psychiatric help and am working through my problems but I still have a long way to go. The problem now is I feel like my boyfriend is sick of supporting me and would rather be doing other things than helping me when I'm upset. When we were at one of his friend's parties I saw someone who looked like one of my attackers and panicked. He followed me outside and we sat in my car while I had a panic attack and after about 10 minutes he said "well, I have to go back to the party. See you later" and left. He didn't come and check on me and I sat there for hours crying and anxious. That sort of thing has happened 3 times now, with the third time being last night. Now I don't know if I can risk it happening a fourth time as I'm pretty fragile as it is. I love him and everything else about our relationship is really great but it hurts so much that he doesn't put me first when I ask for help or just clearly need it. I don't know what to do.