If that small portion is 10-15%, or 31-46 million, that comes to 26-39 people to count for each "worker." Subtracting even 50% of the Census staff for "management" and "support" staff and that's still 52-78 people to count for each actual Census counter!
AMAZING!
Once you take even the smallest part of this process out of the hands of the computers and the mail delivery service, that part is going to take far more time and effort.