The mushrooms are actually doing an important task, and helping break down the roots of the tree. What kind of tree was it, do you remember? Oak roots can have yummy mushrooms that grow on them ... but you'd need an expert mushroomer to tell you.

Just put on some gloves and pick the mushrooms and throw them away. Eventually, they will run out of material to work on, and won't reappear. If you just kick them over, you're distributing the spores to help more appear!