I am no expert on IRS enforcement actions, but absent a valid filing, if they assess tax based on income reported to them by 3rd parties I would certainly want to challenge any assessment that I considered incorrect by sending my return with the correct filing status, deductions, exemptions,credits, and tax. But yes, you most certainly have the right to ignore their notices.
What if the law does not require you to file? There are provisions in the IRS code that simply have no basis in law; i.e., statutes. Once the IRS sees that you actually know the law, they leave you alone because opening that can of worms would be a huge problem.
The IRS will issue you a “jeopardy assessment “ and issue you a statutory notice of deficiency. These are also called 90 day letters as that is the length of time you have to appeal it to tax court.
If you do nothing. You lose your right to appeal.
I have however had clients pay the balance due where other CPA’s bungled the audit, then I prepared a 1040x amended return which caused the IRS audit to be redone and the client got the $$$ back when no other remedy was available. This works as long as the 3 year statute of limitations has not expired.