I don't think it's too little. If he's at the proper weight then I don't think you need to worry.
It's so much harder to get weight off a cat than a dog. My cat Suki is overweight and nothing we do makes her lose any.