I'm so sorry to hear this. My cat Ziggy is very good at the vet but I've noticed that on the way home she'll start panting because she's become so stressed out. I've been trying to spray the carriers with a calming spray but I sometimes forget.

Maybe you could give him some kind of a pill for anxiety on the day of his vet visit. Good luck.