Whenever I've kittysat, I've always made it a point to go and meet the cat/kitties so they won't think some stranger. I also get a tour of the place so I know where everything is. I'm SURE your cat will be just fine. I think you might get the cold kitty shoulder when you return, but they get over it.

Have a great time!