What you do now will set the pattern for the future and what they will expect. Putting a stop to this now will save you a lot of heartach later. It sounds like you will have to be firm with them but better now than later.

I'd tell them they have each other.... HE is all you have and right now he needs his Mother. He has got to get use to it just being the two of you.

They may just be trying to help but if it is allowed to happen, I would be afraid my child would end up afraid I would leave him too. He needs ALL of you but he needs to be at home with his Mama.