When the Constitution was written, they did not envision the popular vote having any direct part in the presidential election. The voters would elect their state legislatures; the state legislatures would select the members of the Electoral College; the Electoral College would vote for President and Vice-president.
The problem with the Colorado system is that, since caucus attendees don't know the intentions of even the first level of delegates they are voting for, there is not even an indirect connection between original voter intent and the outcome.
It is a fundamental rule in our system of government that every government employee and officer has the privilege of occupying their position only because they have been appointed or elected to it by someone else. Likewise with whoever appointed them. Ultimately this chain must end up with actual voting members of the public.
The Framers dealt with the problem of "the masses are asses" by making the aforementioned levels of indirection different lengths, appointments and elections by different methods, for different terms, etc. But, again, all these levels of indirection must ultimately end up at the voters.
The un-American system used in Colorado violates this principle by breaking the link between voter intent and who gets the office by inserting three levels of unknown and corruptable delegates who, like kinks in a transmission cable, have no real function other than to serve as points where noise and distortion can be injected.