I heard on FOX that Specter doesn't like him either because he voted against Specter's point of view on a PA SC case a few years back. The Supreme Court upheld Alito and not Specter's view. I can imagine Specter would hold a grudge on someone who bested him.
From RNC Email:
SEN. ARLEN SPECTER (R-PA) ON
JUDGE SAMUEL A. ALITO, JR.
Sen. Arlen Specter (R-PA): "I've known Judge Alito through the better part of two decades. He sits on the Court Of Appeals for the Third Circuit, which includes my state of Pennsylvania. He brings to this nomination a very distinguished record: was in the Solicitor General's Office, was U.S. Attorney for New Jersey. He has been on the federal bench now for 15 years. We are in the process of assembling his opinions. It is estimated that he has been involved in about 3,500 cases, has some 300 opinions which he has written. So that we have a very good idea as to his approach to jurisprudence." (Sen. Arlen Specter, Press Conference, 10/31/05)