There isn't just one type of consumer downloader. The most avid downloaders are almost always avid consumers of CDs, DVDs, etc. The "casual" downloaders are not economically significant.
Plus there is a price-based solution: If a service like the Apple iTunes store were priced at $0.05 per cut, rather than $0.99 per cut, I'd rather download a quality digital recording than a rip of unknown quality.
Consider the cost of producing a record versus the cost of producing a major movie. The price of music recordings could fall pretty far and still be profitable relative to costs.
Have kiosks in malls and such that a consumer can go to, pick individual songs at $1.00 per, and have those ripped on a blank c.d. An average c.d. of 15 songs will still bring the desired $15.00 price range for the sellers.
Chicken and egg. The 99 cent price factors in swapping. If most people downloaded rather than swapped, the price could be a nickel. But unfortunately the music cartel will make sure to kill the chicken to get a couple of extra eggs.