I have a drughouse next to me. Been going on for 7 years. No one will do anything about it. Although the woman has been arrested for drugs and on probation, it still goes on.
And yes, as Catlady said, you have to stand up to these people or they will walk all over you.