This has been an issue for years, but it is not as straightforward as it seems.
There are 9 Supreme Court justices and 179 appellate judgeships, for all intents and purposes these are the ONLY judges that determine constitutional matters.
On the other hand, there are 678 district court judgeships (they primarily deal with criminal and civil matters), there are 352 bankruptcy judgeships and 551 full-time and part-time magistrate judgeships.
In other words, there are less than 200 federal judges involved in constitutional matters and over 1500 who have nothing to do with these issues.
Supreme Court and Court of Appeals judges often have book and law school deals that offset their incomes. However, normal judges DO NOT and the problem has been these judges leaving to return to private practice and the courts risk losing talent this way. A talented judge in his early 40s with a mortgage and three kids in college can usually make at least twice as much money in private practice and these are the ones that Roberts (and Rehnquist before him) is worried about losing.
judges rarely see pay increases and while now may not be the right time for an increase, many will be lost to the private sector without COLA's.