I don't know, I really don't think I'd worry about it.

If he was really stalking you, or up to no good, he wouldn't have told you he was doing it. The fact that he mentioned seeing your house seems to me to mean that it's all innocent.

I personally like to look at houses. I'm interested in houses, yards, neighborhoods, etc. Sometimes my boyfriend and I just go for a ride to look at houses and yards. If I happened to see a co-worker's or a friend's car at a house, I probably would say, "Hey, do you live on Main Street? I happened to drive by, and saw your car there. Nice house!" or something like that.