I would hang on to the audit determination and agreement and the last year return to show it was settled. It sounds like you can pitch the rest.
As regards ones own returns, it never hurts to hold copies of page 1 and 2 of your signed Form 1040 beginning with your first year working in case you ever need to challenge SSA about your earnings record.
Yea, I think I have most of my own tax returns... and I have some statements from the SSA which show my history of earnings/contributions.