That is a tough one. But some cats are working cats - at the barn where I keep my horse there are semi-feral barn cats. They live in an unheated barn year round (there are warm hay/straw bale bedrooms, but to me it still feels cold) but they are there to keep the mice down. Every barn I've been to has had outdoor barn cats - and those unheated barns can get cold! Most working cats (mousers) have to be outside for their jobs. I would maybe stop, and knock and ask the people (maybe make something up like it looks like a cat you know and wanted to get a closer look?) and see what the situation is.