So, in other words, my right to speak is not absolute - I am not permitted to defame you, as you agree. Wisely so, I think.
Now that we've agreed that the right to speak is not absolute, let's apply that very same logic so that we may come to understand that none of your rights can be absolute. Once we realize that, the argument becomes not whether we should draw a line restricting that right, but rather where that line should be drawn. You seem to think that it should be drawn in such a way as to prevent consensual searches, in much the same way that you think a line should be drawn across the First Amendment preventing defamation. Most people disagree with you on the searches, but agree with you on defamation. But if you persuade enough people to move the line on the Fourth Amendment, it will be so. Good luck.