My mom's cat Belle always does this to her - it drives her nuts - I think Mom probably cleans the litter too much (2x a day for our 2 boxes seems to be good).

My cats always compete to see who can "deflower" the litter boxes first after my husband puts in the new litter for them. It's kind of funny to watch.