Lee's surrender at Appomattox did not end the concept of state secession, as you point out. However, the end of the South's attempt at independence greatly discouraged any future attempts of leaving the Union. The end of Reconstruction was a concession to the Southern states to allow them to run their states free from Federal military occupation in exchange for their loyalty to the Union. Only 12 years after the end of fighting, both Yankees and Rebels were exhausted, and the Northern public was tired of the low level guerrilla warfare where the Army was used to protect the freedmen and suppress the former Confederates. The GOP leadership was at this point more interested in conquering the West and industrialization than in the former slaves.
The decades after the end of Reconstruction resulted in the aggrandizement of Federal power, under both Republicans like Theodore Roosevelt and Democrats like Wilson and Franklin Roosevelt. The states lacked the power to effectively resist Federal authority. The civil rights movement of the 1950s through the 1970s was effectively a second Reconstruction. Several governors like Orval Faubus and George Wallace opposed integration, but backed down under the weight of Federal military power.