Dont' feel guilty. I think your fostering cats is wonderful. You are giving them what they need: shelter, safety, love, food.
If you didn't have the cats, where would they be? In a tiny cage, outside dealing with the elements and not so nice of people, hunting for food?
Is it the "ideal" situation for the cats to be confined to one room? maybe, maybe not, but compared to what they could face....I think they would pick your one room and all the love you have to give them to the alternative. And, yes again, it is only for a short part of their life and you are making it more peaceful for them.