Yeah, the idea that the Speaker of the House should be neutral makes a lot of sense. In fact, I suspect that many of the Framers envisioned a neutral Speaker of the House as well. However, after Henry Clay got the ball rolling to expand the Speaker's powers in the House at the beginning of the 19th century, a precedent was set for an increasingly powerful Speaker of the House. Of course, since the powers of the Speaker are not defined in the US Constitution, the current powers of the Speaker of the House could be drastically curtailed by a simple majority vote in the House at any time. A simple change of the House Rules would be enough to do this.
In the 1890, Thomas Reed acquire such power that he was dubbed Czar Reed, Finally, under Joe Cannon, the House rebelled and the Speakers powers were reduced. Sam Rayburn, who served as speaker for about 20 years, managed the House largely by persuasion rather than by formal powers. Mr. Sam was a New Dealer but more populist in than liberal.