Our stuff at work is pretty locked down — vnc is blocked.
I usually use ssh and run our scripts from the command line.
Linux is really the way you want to go if you have administration tasks. There are so many more free and useful Open source tools that you can use, built-in support for various programming languages, etc. The only thing you have to be careful about is where you get your tools. I trust sourceforge and a few other sites. I stay away from any portals based in Russia or China.
Yup. When I'm doing my "real" work, it's all scripts and command line.