It's not exactly clear. Certainly the judge is appointed for life, but the court itself has no such protection.
Likely Congress can make the court disappear, giving the judge no facilities, clerks, or staff, or anything else except pay. Such a judge would be in a world of hurt.
That would be unconstitutional considering judges are guaranteed the same remuneration as when they were appointed. Taking away office expenses has the effect of denying the judge remuneration. Furthermore, taking away their jurisdiction and not seating them on a new court (perhaps a court the combines two old jurisdictions that have become too small), is at best a poor attempt to circumvent the constitution, and perverts the rule of law.