I think you should knock on the door and politely ask. If the cat belongs to them, they may be very sad if he/she suddenly disappears. If it's a stray cat, he/she may be there because they're feeding every day at a certain time - but, as you say, the cat shouldn't be out for that long!