Thats a tough question as he obviously wants to be free.
But as he gets older, he will be more vulnerable to Dogs, Mean Kids and other dangers.
Nugget and Scratchy were big street wise Cats, and they are both gone now.
Is there anyway you could take him and have him in an outdoor porch?