Whatever you decide, do not ever let a cat out that isn't "fixed". Boy cats who are NOT altered will run further away....
I let my cats out too, but only because we have a large garden; so they don't go anywhere unsafe