If you feel better blaming them, then that's OK. Looking at the numbers, though, the low turnout hurt Specter more than Toomey.
For Toomey to win, the general consensus was that he had to really carry the 'T' to offset Specter's 'C' (The T being the rural counties, the C being Pittsburgh, Philly, etc.)
The turnout in Allegheny Co (Pittsburgh) and the neighboring counties (Beaver, Butler, Westmoreland) was really low. Toomey actually carried all four, albeit by a small margin.
Toomey also carried most of the T counties, again, though, the margin was very small (for instance, Lancaster, one of the most conservative counties in PA, only went to Toomey by less than 6K votes (57-43)...that was a bad omen).
The county that won it for Specter....Bucks county, where Specter pulled in 75%, getting an extra 87K votes. The high turnout in Toomey's base could not counter that blow (Lehigh Valley went 67% to Toomey, but was only worth 10K extra votes).
Bucks Co. is Bush republicans. That is what did Toomey in.