Karen has given you the best advice. This does need to be dealt with as soon as possible so you can assert your rights once and for all.

Years ago, we had a cabin at Big Bear. We had an odd pie shaped lot - and at the top of our lot people built a lovely home right up to our plot line. Because they thought it would look lovely to have a nice fence around their property, they put it up and lopped off a huge chunk of our land. They even removed the assessor's property markers.

We had the property surveyed again and new markers installed. When we approached the owners to remove the fence they ignored us.

The fence one weekend magically "disappeared" and we never heard one word.