Exactly.
I know a dispatcher who works from home covering a rather large area. It works because he works. The line will be answered and what ever service you need will be dispatched.
The person before.... not so much.
I am not against remote work, used to do it myself long before there were even half the stuff there was now to make it easier.
The square root of the people in any office do over half the work. And that is with the boss looking over their shoulder. With the entire office working remotely it is closer to 80% of the work. And that just does not work.
After I made the break I'm not going back even if I wanted to. Out of sight and out of mind proved to be just as real as I ever suspected it would be. The changing of the guard did not help that one iota at all either.
It is possible to be a significant individual contributor or minor manager remotely but I can't see it for other functions. Even significant individual contributor problem solvers could have a hard time without presence. I have made very high powered dispersed teams work well in the past but it takes an effort to organize them in a joint effort.
To everything there is a season. Knowing when to leave can be the hardest thing of all to do right.