As I believe Fiv is passed through sex and deep puncture bites. If you think that he won'tt become agressive towards the others (or the others towards him), then there shouldn;t be a major issue with letting him meet the gang.

Good luck at the vets!