I'm not a hard-core *nix admin. What in that sounds most useful to you?
Mostly that you have more control over what's loaded than you do on, say, a windoze box. I have servers that I use for some of our prototyping work that doesn't even have a keyboard, mouse, or monitor attached to. Most Unix variants have a concept known as a 'runlevel' that tells the system to start up certain processes and/or services at each one. Let's say I want to do something like maybe expand the size of the /home partition. I'd boot the system in single-user mode where the vast majority of services aren't available. (On a headless system you'd need something like an ILO card to give you console access when network services aren't there.) In single user mode, I can unmount the /home partition, (it may not even be initially mounted in runlevel 1), resize the volume, then bring it up to runlevel 3 or 5.
Most computers I use regularly boot up in runlevel 3 which doesn't fire up the X server unless I want it to. Even though it would be swapped out if unused, it's still eating up virtual memory, and enough cpu/interrupts to keep it active. A system that is primarily a web server, application server, email, or DNS don't actually need a graphical interface, so I figure it makes more sense not to have it running. You'd normally be accessing them by SSH for command line access and SCP for moving files on and off the server. Telnet and ftp are evil daemons whence come more security issues than can be named.
The gui itself on an X-based system opens unsecure ports that I prefer to have closed unless I absolutely need them.
I admit that I don't know a heck of a lot more than bare generalities as far as the GUI goes with OSX, but I figure it probably presents more of a profile to an attacker than ssh does. I look forward to having a chance to explore it more though, as it seems to have proven to be fairly robust against attacks so far.
The bottom line, is that for many things, I prefer to use a command line rather than a GUI. For other things, a GUI is better (like workstations). To me as a good admin, I figure you present the smallest profile to attack, and maximize system resources where youj can.