Yes, you can invoke the 5th amendment under those circumstance.
What are you talking about? As a taxpayer, you have to sign a return stating all of your income and its sources. When you sign it, you say you are signing it under penalties of perjury.
If you refuse to file on Fifth Amendment grounds, or if you file but refuse to sign it, you are in a world of trouble.
So what are you talking about?
And we haven't even covered the Fourth Amendment violations involved in an audit.