He probably belongs to a neighbor. Next time you see him, you could put a note on his collar (use use those blank address labels). Write something like: Call me at xxx-xxxx and fold it over his collar so that stays on there for a while. If they call you, you can just say that their baby comes for a visit and you wondered if it was okay to feed him or if his shots are up-to-date because he wants to play with your baby. If no one calls you but the note is gone the next time you see him, they either don't want to call or he licked the note off his collar
Anyway, that's what we did with Seymour when he came for a visit one day. Actually, we did it twice because my sister wanted him but wanted to make sure he didn't belong to someone. Now Seymour is a 20 pound hunk o'love!