Stevens is the senior justice though (came on during Ford's administration). Even if Bork had been confirmed, Roberts would still have been in by Stevens in all likelihood.
Now if Rehnquist had not died, and Roberts confirmed for O'Connor's seat, yes, he would have been sworn in by Rehnquist.