If it's been four days and he is occasionally visiting I would say that he is feeling things out. He may just be uptight about his move and being in a new place. Hopefully when he gets used to his new home he will settle down and become friends with your babies. It's hard to tell because sometimes they just stay a bully! I hope friendship is soon coming. It sure makes things more peaceful!