I think if they are playing rough and the dog doesn't act comfortable or the owner seems upset about it the best thing to do is call them away to play in another area.

So, I guess to directly answer your question...if the above is the situation, then yes (IMO) it's rude to not call them off.

However, if the dogs they are playing rough with are playing rough back and appear to be having a good time, I don't see any reason to call them off.