To be honest calling a baby a female and male is something I dont think I've ever done, I refer to them as a baby girl or baby boy and for a child older its a boy or man depending on the ages, It might just be a way of speech in London but I've never see anyone get upset over this, I can see where you are coming from though