As far as not changing all the water: not true. Many people into the larger cichlids have to do 100% changes daiy in the 150+ gallon tanks. So long as the new water is not chlorinated and has the same ph and temperature it is fine. Beneficial bacteria does not reside in the water and nothing good comes of keeping the old water.
For a 3 gallon betta tank I recommend cleaning the gravel and taking out half the water. If you want to rinse the whole thing do so only with regular or dechlorinated water. No bleach. By rinsing the gravel and decoratons you will be killing some of the good bacteria. Only rinse the filter sponge when absolutely needed. Do so in old aquarium water and try not to expose any filter media to the air for more than a few seconds. You don't want to kill this bacteria. I rinse my sponges once every month or two.
Do you test your water at all? What your betta has sounds like ammonia burns or a ph problem to me, not fin rot. But I could be wrong.