It is without question that the states that left the union did so to form their own union, aka Confederate States of America, not to overthrow the US government.
One can parse words in any fashion to meet one's agenda and in this case I think that you've done just that. When Forrest said, "of the government which we fought to destroy", he obviously meant the destruction of the 'union', not the overthrow of the US government.
You apparently missed my point that there were honorable, principled men on both sides of the issue.
Oh well, have fun with the "yankee" hating if it gives you some comfort.