I know exactly what you are saying. This past Friday I had to put Zoee away (she stays in the bathroom while we are not here) and she didn't want to go in there. Usually she is very good about it, but that night she hung her head down and walked slowly in. I felt horrible!!! But I knew she'd be fine and we were going to visit her "cousins" for the weekend, so that made up for the fact she had to be locked up for a few hours Friday night.
In my opinion, cats are more independent (usually) than dogs. They tend to be able to take care of themselves better. I don't feel as bad leaving Taz home for the weekend mainly because he has the ENTIRE house to himself. He probably loves it!!