if anyonegets really worried, they can always put child gates around the door, and by the time the cats realise whats happening, you can get out the door. just open the gate go in, shut the gate.open door, go out, close door. pretty complex but thats if anyone gets really desperate. omg i get annoyed too when someone stands there taslking or holds onto the door and is standing inside. you cant exactly grab the door from them lol. my two get outdoortime and i spend 20 minutes a day getting them out of trees etc, the family tells me to leave them. i wish they would understand that after reading hundreds of cat books,I know cats ok. it drives me mad.
perhaps you could keep the door to the room with front door in it closed??