I would politely say that you're both working on it. If they keep it up, bring up a fault of theirs.