Ah, but consider the following:
The income tax is constitutional thanks to an amendment. (I've heard the "wasn't properly ratified" arguments, and I'm not convinced.)
Ronald Reagan signed a law to index the tax brackets to prevent bracket creep. If an income tax is constitutional, it logically follows that it is constitutional for the government to pass a law mandating the appropriate collection level.
To complete a constitutional function, it is reasonable to say that the government can take whatever reasonable steps are necessary to complete that function. The Constitution doesn't say that the government can buy army uniforms, for example, but it reasonable to say that the government can do this as part of the task of national defense.
In order to index the tax levels, we need to have a Consumer Price Index. In order to have a Consumer Price Index, we need to sample the population and figure out what they purchase so that we can apply appropriate weights to that index.
Guess who does that sampling? The Census Bureau. That's probably why the Census worker was knocking on doors.
The CPI is managed by the Labor Department. The Census Bureau / Commerce Department puts out some of the second-tier economic statistics like housing starts and retail sales, but as far as I know they only survey businesses for those.