Remember the "refi boom" 2001-2002? Instead of a return to "normalcy" in the mortgage markets when the boom ended, Fannie/Freddie launched an increasingly aggressive series of subprime mortgage products. That lead to inflation in housing prices, and the eventual crash of the housing market due to non-performing loans.
I don't know how you can say CRA applied to banks and not mortgages when CRA dictated the number and kind of mortgages banks were required to issue.
There's a very good video called "Burning Down the House" that provides a summary of these events:
http://www.youtube.com/watch?v=1RZVw3no2A4