This was just a thought...and I know it may be bad suggestion, but since he has lived for some time as an outdoor cat and has some survival skills already; what do think the chances are that he could live outside your home? Obviously the area you live in would be a huge factor as to whether this would be a feasible idea. I wonder if he had a little shelter of his own outside, if he would stick around as an outdoor kitty for you.
Like I said, I was just a thought.