I agree with Karen, he needs some confidence building exercises. Now is the time, before things escalate and he does get hurt.

Even just 2 minutes playing with him with one of those toys on a string, morning and evening, will help. Remember to put the toy away at the end, those strings are deadly for cats and the toys are only meant to be used with supervision.