Maybe from his eye? If it is uncomfy, as you said the vet said, that could produce low-level but constant stress.

He sure looks sweet!

Don't furry cats LOSE their hair, or pull it out, under stress?

Go figure!