Read my post about the education that many thousands of engineers got working on Apollo. Do you think they just walked away from that incredible experience and never used any of those skills and business contacts again?
I'm convince that this is the true legacy of Apollo. It might be the hardest to document, but nevertheless the largest by far. And combine it with the push for science training after Sputnik (which has now been abandoned).
It seems that you are confusing the point here. I don't think anybody is saying that we should support the space program primarily as a source of technological improvement. What they are saying is that there are tangible benefits to an investment in space, not including the exploration itself.
There are many situations that occur that spur technological advancements, there was a great leap in powered flight in WWI and WWII. It would be silly to say that the things pioneered during that time would never have come about, but those situations did cause them to come about at a much accelerated rate.