That is creepy! What about someone coming to read your meter? Is that anywhere near the door? If so, you could call the energy/water company.

I know sometimes service people just walk into my yard, even though I have a large gate that is very securely shut (to keep the dogs from getting loose).