I do understand limits, especially if your space is small but I can honestly say that I didn't notice much difference going from 1 to 2 cats or from 2 cats to 3, except for more fighting.

Now going from three to five like we just did is quite noticable, but nice.