Exactly. The key word is SELF-incrimination in the 5th amendment.
The 5th amendment does NOT say "I can refuse to answer any questions at all".
That's "contempt" in any courtroom.
Sheesh, this stuff came up on Perry Mason in the '50s all the time.
How many coutroom shows feature the judge saying: "The witness WILL answer the question!".
It would make for pretty boring TV, if the witness just said: "I don't feel like it, judge."
"Uh, ok I guess the trial is over."
Scooter Libby is the perfect example of why an innocent man should not co-operate with the cops/FBI/DoJ.