No, it would be a civil matter. It would not rise to the level of criminal fraud because the taxpayer does have an arguable position even if that position doesn't ultimately prevail. If the IRS disagreed with the taxpayer's position, it would just disallow the tax return(s) and assess tax based on what it believed to be the correct filing. It would then be up to the taxpayer to challenge the IRS position, first with the IRS itself and, barring a settlement with the IRS, then in tax court.
And if the tax court ruled against them, but they didnt pay...