Would he be willing to go to any kind of counselling with you? Sounds like he's the one with the problem, not you. We all know you're a good person, what's his problem?