I've had this issue myself with Nicki. Once a man stopped me while walking Nicki and demanded I picked up poo that OBVIOUSLY wasn't hers (she's a hundred pound pooch, the offending poo was from something along the lines of 20 pounds - just a small difference in size!) He started to really argue with me when his wife came running out.... I swore if she could have slapped him, she would have. She pointed out the size difference and made him appologize! LOL

that said, I doubt it matters WHERE your dogs go.... unless HE is the property owner, and HE pays for the lawn's upkeep, he can keep his grumblings to himself.