Like by tomorrow afternoon. The longer he waits, the longer people are going to remember. This guy has no chance at winning this particular race, but that's not to say that he won't be given another shot.
Assuming Issa's lining up for Senator next year, Tom could come in as budget director, gain a lot of visibility, and be set for Lt. Gov, Controller, Sec of State, etc in 2006.
Then again, he might ride this thing all the way to the end, and take a chance at not even being re-elected in TO. The golfing set out in Westlake/N. Ranch have the same memories as Newport Beach.