[semimojo]

The Census Bureau became a permanent agency within the Department of Commerce and Labor following passage of An Act to Provide for a Permanent Census Office in 1902.

The Census Office only carries out the directives of the Congress.

In fact the 1902 Act clearly states exactly what information the Office could collect for that particular census.

Congress never gave up their authority.

[Cboldt]

See too, the text of the constitution ...

The actual Enumeration shall be made within three Years after the first Meeting of the Congress of the United States, and within every subsequent Term of ten Years, in such Manner as they shall by Law direct.

Title 13 of US Code: Census <- there's the "by law direct" stuff