There are some cats that never like to be inside, like my neighbor's cat. All of their animals are outdoor ones and we ended up taking and taking care of one cat that running around the neighborhood.
I'm sure within time the cat will warm up to you. I think it has a lot to do with patience.