It still took the slowpokes 140 years to build what is basically just a stack of rocks.
Hey, no metal tools, no draft animals, and in a climate where wood, materials, and pretty much everything rotted faster than you could usually build.
Plus permanent structures were typically smaller in the form of rock walls, and other utilitarian - like bird hunting blinds (star mounds).
Ironically, while Polynesians were navigating thousands of miles over open ocean using the stars, Europeans still thought the world ended over the horizon.