Well-intentioned theft is still theft. It's their cat. Unless there is proof that the cat is abused, give it back. If you're concerned about it's welfare then politely mention to them that the cat is not safe in the hallway and seems shy. Then offer to take it off their hands to a more relaxing place (don't say "better", that may be insulting), so they won't have to worry about it anymore.

I've had my boys for almost 2 years and Havoc still gets nervous and jumpy about some things. Does that mean I abuse him??? No. He's just spooked easily by cartain things (plastic bags, aluminum foil, the big,scary monster that obviously lives in the attic, etc). If he got out and I found out somebody **poofed** him knowing darn well that he was my cat, I'd be pretty ticked off.