I think for the cats it is much better to be confined than to live in a cage or even outside. But the question you have to answer is how is it for you?

Noone can do more for her cats than you do. However as Donna points out, the fosters can bring infections in or behavioral problems etc.

I know that I couldn't do it. I would break down when I had problems like you had them the last weeks. So if you decide to do less (and this is your decision) you should not feel bad about it. You make a huge difference for the cat world. And you are allowed to define your personal limit.