He is extremely young and really should not be away from his litter yet. Understand that he is bonding with you and you need to help him through this period. Actually you should cherish this period since it such a sweet and rewarding period. Once he realizes you are not going to leave, he will be ok.