Can’t one just say no?
Sure. But you have to be prepared to lose your job, income, and benefits. Maybe be unable to file for unemployment benefits. Doubtful you would get a reference from the company firing you for your future job applications. I realize this doesn’t apply to everyone, but it would apply to most.
I did say no. I was asked to give a reason and I did.