That is a VERY important point! And don't forget the federal district judges.
A federal judge can have influence for decades. I shudder to think who Obama will appoint should he get four more years.
Both Antonin Scalia and Anthony Kennedy will be turning 80 in the next 4 years. Regardless of whether we get Obama or Romney, if we lose either Scalia or Kennedy we will be screwed.
With Obama, we will get another Elena Kagan or Sonia Sotomayor. With Romney, we will get another David Souter, Sandra O'Connor or William Brennan. Either way, there will be a left wing majority on the Supreme Court legislating from the bench for another 30 years.
Be afraid. Be very afraid.