The last drop in temps was indeed sudden and recent, and is called the Younger Dryas.
As you point out, the conditions have never become like the glaciation since those long-ago times, and nothing we can do would have such drastic impact.
Yes, but Younger Dryas was really only a blip at the end of the already long and very cold period I described.* It's barely visible on the 450k yr. graphs, and doesn't compare (except in suddenness) to the big drops after most interglacial periods.
*I may have been "off" slightly -- looking more closely at more graphs, it appears to me that Younger Dryas may have occurred as temps were recovering, but not fully recovered, from the minimum of the last glacial. This makes sense: It suggests even more (2 or more miles deep) long-frozen Canadian ice melting than I'd thought! But either way, it's a "blip" in the overall trend / pattern.
Change the time scale by a factor of about 400k and YD may be roughly (very roughly) analogous to a cold snap following a few warmer days in (around here) early March.
Agreed mostly on the rest of your comment, but a large scale nuclear war might do it ("nuclear winter") -- different mechanisms, though...