Cats are notorious, both to vets and caregivers, for hiding illness until it is almost, and sometimes is, too late to help them. They are prey animals and so hide illness in order to survive. I am sure that everyone on PetTalk with cats, even those with years of experience, can tell you about a time they didn't realize one of their cats was truly ill. In other words, take your cat to a vet. We could throw a hundred possibilities at you, but they can't take the place of a professional examination. Did your vet register the cat's weight when he was neutered? Do you know how many pounds he has lost? Since you say really skinny, I assume it is pounds and not just a little. Just because one cat has worms, doesn't mean the other doesn't, and having worms has never led to my cats having a dramatic weight loss. He may have lost muscle mass, but he shouldn't have reached "really skinny".