Why do they HAVE to sleep only on the clothes I want to wear today?!?!

Because they have to make sure that they leave their scent on them so that you can smell your cats all day and not miss them so much when you are away from them.
Plus, that's just what cats do.
Ya gotta love them.