My cats both sleep with me (and sometimes on top of me) but I enjoy it. It provides an extra layer of warmth

The only time I have tried to keep them out the both scrathed and meowed at the door until I couldn't stand it any longer and just let them in.