It's not the "53 seconds". Perry has been awful in every debate. Every single one. Saying that before the "53 seconds" he was having his best debate means little, his other debate performances were all awful. It is disingenuous to claim that people are criticizing Perry just because of his "53 second" amnesia, it is the "53 seconds" on top of all the other awful debate performances.
How about that 10 + year record?
This has become folklore as it's constantly repeated. He has not had "awful" debates but has been singled out as the poster child of bad debates.