You can check "anti-aliased" to make a smoother edge, or after you make a selection you can modify it and feather it a couple pixels.