Do NOT feel guilty. I've had to rehome a few cats myself and yes, the guilt does go away. Especially when you find a wonderful new home for him.

You are doing what is in your cat's best interest. That's all that matters.