I would be willing to bet that card users are paying more for the costs of others' defaults, rather than for the people not financing balances.
We would have to look into this comples issue - there are tax deduction and banks ability to "create" money for lending.