Seems the issue is that your silence IN CONTEXT can be used to infer, a la:
Q: “Did you do A?”
A: “No.”
Q: “Did you do B?”
A: “No.”
Q: “Did you do C?”
A: “No.”
Q: “Did you do D?”
A: “No.”
...
Q: “Did you do X?”
A: “No.”
Q: “Did you do Y?”
A: [silence]
Q: “Did you do Z?”
A: “No.”
Yeah, he did Y.
Perhaps, but the burden of proof should still on the court that you did Y. Your silence should not be able to be used against you.