As I see it, there are three problems with space.
First, the present management and methods of "Big Space" are wretchedly bad. There's a very serious lack of competent management and systems engineering. On the other hand, Rutan would face serious difficulties in this realm because he's not set up to handle the sort of huge project that a real manned space system entails. Airplanes -- even his SS1 -- are easy by comparison. I think this, more than anything else, is the primary problem that must be solved and I honestly don't know how one would go about doing so.
Second, Rutan, et al. cannot get us there commercially, because there is no money to be made from it right now. All of the "space mining" ideas and such only make sense if you've already got a bunch of infrastructure already in place, and private industry is not going to put it there -- the horizon for a decent return on investment for infrastructure would be measured in decades, if ever.
And so you need to bring the government into the act. The problem with government "manned space" is that it's not sexy, and will be brutally expensive no matter how efficiently you can shove hardware out the door. It would require a significant and very long-term commitment by Congress ... a rather forlorn hope at this point.