“And then they can search your residence at will without a warrant?”
Well, yes - if your wife invites them to do so. Your wife has legal authority to invite someone on to your property.
And you used to have the legal right to forbid it.