Look at the ages of the “Conservative” judges left on the court and you will see that if Obama gets another term he may have a better chance of success.
But even if they did retire, that would mean Obama would be able to simply fill SCOTUS with judges that answer only to him. Anyone who tried anything remotely close to that failed badly, and being afraid of that approaches irrational paranoia.