Sounds freaky ..!
I don't have an experience like that, as my cats sleep in a seperate room. But I think the cats just think:" hey, that looks like a mighty good spot!!Move over!"