He doesn't sound like he deserves you.

I'd dump him without having to think about it. You don't deserve someone to be a jerk to you like that!

If he's incapable of understanding, or atleast be considerate about your feelings, then he isn't capable of being in a relationship with you.