They already have access to the information that they want you to provide. Think about the IRS, data bases, and so forth.
Given that they’ve got all of that, I see no reason to further disrupt my life by giving them the time of day.
There is no centralized government database of information about you, and the Census has no access to information from the IRS, etc (and likewise, other government agences have no access to information about individuals from the Census; the FBI has actually tried a couple times to get Census data, but they were shot down in court.)
I could understand it if this was an annual event, but it’s not. And by not giving them the time of day, you’re actually doing the opposite. A ringing doorbell that my family (and dog) would have to ignore daily until it stopped is a hassle and a HUGE disruption. Ditto with a ringing phone when it has to be ignored because I’m avoiding someone. But to each his own.