This is direct from politifact.com:
Weve previously reported that there were indeed budget surpluses during his time as speaker of the House, from 1995 to 1999. By fiscal year 1998, the federal budget did reach a surplus of $69 billion. And in fiscal year 1999 — which Gingrich can claim some responsibility for, even though he was out as speaker for most of the fiscal year — it was in surplus as well, to the tune of $126 billion.
“The budget ended up balancing faster than either party expected simply because economic growth was so strong,” said Chris Edwards, an economist at the libertarian Cato Institute.
He does credit Gingrich for pushing President Bill Clinton toward reducing the deficit.
In his 1996 budget, “Clinton proposed to stabilize the deficit at around $190 billion. He did not propose to balance the budget. The Republicans under Gingrich pushed him into it,” Edwards said, and that materialized in the Balanced Budget Act of 1997.
Let's look at the bottom line, shall we? Our national debt increased each and every year that Newt Gingrich was in Congress, and every year since. Such a state is not indicative of a surplus. It is indicative of not having enough money to cover all the spending that has been authorized, and borrowing ensued.