I find our most standoffish cat Milly, likes to have alone time with us.

I.e. If I find her in our bedroom alone, she is more likely to stay around and get cuddles and she purrs more.

Could you take Charlie to another room to give him a couple of private one on one cuddles ?