Cats need shots once a year for chlamydia and rabies. Usually given at the same time. The feline Luekemia shots are also I believe yearly or every 2 years. Not sure. However, if your cats are NOT exposed to other cats then skipping the luekemia shots are okay. They are quite expensive. I think they were about $35 per shot.
The FIV shots and such may be important if you are planning or are pregnant. Pregnant women should NOT clean out litter boxes. Some diseases are transmittable to humans. Getting these shots are important.
I don't think your vet was a crackpot. Some shots are not given until kittens are atleast 6 weeks to 6 months old. It really depends. They may have already been given the shots at the shelter and he already knew that.
Ps. If your going to get your cat fixed I would wait until after it's first heat. You can do it earlier but my cat had problems and I wish I had waited. Her "boobs" are low and it makes her look fat. Cats who get fixed after their first heat also are maturer. Good luck!