Of course we should.
With 18% of federal revenue (and growing) being used to pay interest on the debt, that would mean that a growing proportion of the taxes you pay would go directly to the foreign governments who hold the debt. In essence, you're paying taxes to support foreign governments rather than our own. And that doesn't even begin to get into the issue of the money we also hand over to them as "foreign aid".