I don't let my cats out - too many dangers. That is a topic that has been debated here since the beginning of time, so do a search on outside vs inside to find out more.

Definitely do not let your unneutered males out! They can father kittens at a very young age and are much more apt to get into fights (requiring big vet bills) than neutered males.