"What legitimate use is that? "
Ever take an economics course? If you have you will know that predicting future econmic trends is best served by reviewing past data. Unless you think the government plays no role in the US economy you can probably figure out why the government has a legitimate need for data.
Yes, in fact I would be willing to wager that my economics credentials far surpass 99.9% of the population. However, as far as I know, the Fed doesn't plan monetary policy around how many toilets are in your house. Most of the census data is used for things such as determining how much of various handout programs need to be allocated to states and localites. As I am against those programs, I see no need to provide the data that they require.