John was monitoring the system during tonights debate. Still has not specifically IDd the bottleneck.
Here was his response when I asked what was happening:
Database is nearly completely idle (0.25 load average) and web servers arent very busy. I dont know why its screwed up. It doesnt make sense. It must be a network problem.
I saw myself that the load average was not high and that the cpus were mostly idle all night. I asked him if it could be a DOS attack or a configuration problem and he said could be. Hed look into it. Said the system was not overloaded and it would be a shame to buy a new database server and not solve the problem.
Im in favor of swapping out the system a piece at a time until the bottleneck is found, but thats just me. Dont know what Johns going to do.