Well, I have to ask too..
If you have 4.. what is one more going to matter? I mean, you've established he's a lover boy, and sweet.. and he LIKES YOU.. so I guess I'm at a loss.. but if you really can't take him in yourself.. then I agree with Jen and the others..
Get him to where he can be inside IMMEDIATELY! Even if it can't be with you, it would be better to know that at least he will be inside and not getting any chance of becoming another outdoor cat statistic! That way, even if you can't take him in, you'll know you saved his life from that, and that you've set in motion for him to have a good home.

I wouldn't worry about trying to see if he belongs to anybody. If he does, let them go to the shelter for him. He shouldn't have been outside... period.