Tough one!!
I don't know if I could give him up but if I was in your situation and I KNEW that he was getting a REAL GOOD home I would let him go. That way he would be happy and you could help some more unfortunate kitty souls.