Withholding actually only dates back to the Current Tax Payment Act of 1943.
Before WWII income taxes primarily fell on millionaires. Some government funding still came from tariffs rather than income taxes. The income tax didn’t become widespread until the 1943 act was passed.
Dig, but I remember picking up in some research I did some time ago that the actual method of collecting taxes was discussed and debated by the powers that be for some time before that, with the withholding people being defeated at first until they were worn down by those who favored it by '43. Just going off the top of my head, here, but there was talk about whether it was just to collect for next year's taxes or better to bill for last year's taxes, etc., etc. I think this was the stuff you were alluding to further upstream.
But this is just the history, of course. The system we have in place now (withholdings) absolutely blows, and I think it's led to many of the issues we face currently viz-a-viz massive government spending. Just the attitude withholdings fosters is absolutely devastating. Every time I hear someone talking about the money he/she is getting BACK I want to tear out the remaining hairs on my head.
If we have to have a federal income tax, I think it should be a bill for "services rendered" that you pay out-of-pocket just like every other bill, perhaps on a quarterly basis. Two benefits, there: (1) People could stash money in an interest-bearing account until it's time to pay the piper, and (2) People will no longer be detached from the dollar amount (typically in the several thousands) they pay to the federal government. I am fully convinced once people wrap their heads around that amount, you'd see wholesale change. And that's exactly why the powers that be will never do it.