I would consult an allergist to see if there are some sort of meds you can get on to prevent reactions. My father in law is allergic to cats, but is fine when he comes to my house because he is only there a few hours. When his daughter comes to visit from out of town and brings her cats to stay his house, then he takes some sort of meds or gets an allergy shot becuse they'll be there a week or more.

Some allergies do come and go, though. I had cats a lot growing up without issues, but I still used to have sinus problems when I worked at the local vet some years later. But they got better over time, to the point where I adopted a few felines since then.