Garland is probably thinking “if I had been confirmed as a Supreme Court Justice in 2016, this ruling would have gone the other way.” So why should he be concerned with the safety of the conservative justices?
And conservatives in any branch of government are probably thinking the same about him.