It depends on how much you need/want the job.

If jobs are plentiful where you live, and it is easy to get another one, then there is no harm in standing your ground. If you lose your job or hours, and can/want to find another one, no big deal.

If, however, it is hard to find a job or you really like this job, in general, then you probably need to just bite your tongue and do what they say. That's the reality of the working world.