Posted on 01/28/2019 5:06:01 AM PST by ShadowAce
Learn a few techniques for avoiding the pipe and making your command-line commands more efficient.
Anyone who uses the command line would acknowledge how powerful the pipe is. Because of the pipe, you can take the output from one command and feed it to another command as input. What's more, you can chain one command after another until you have exactly the output you want.
Pipes are powerful, but people also tend to overuse them. Although it's not necessarily wrong to do so, and it may not even be less efficient, it does make your commands more complicated. More important though, it also wastes keystrokes! Here I highlight a few examples where pipes are commonly used but aren't necessary.
One of the most common overuses of the pipe is in conjunction with cat. The cat command concatenates multiple files from input into a single output, but it has become the overworked workhorse for piped commands. You often will find people using cat just to output the contents of a single file so they can feed it into a pipe. Here's the most common example:
cat file | grep "foo"
Far too often, if people want to find out whether a file contains a particular pattern, they'll cat the file piped into a grep command. This works, but grep can take a filename as an argument directly, so you can replace the above command with:
grep "foo" file
The next most common overuse of cat is when you want to sort the output from one or more files:
cat file1 file2 | sort | uniq
Like with grep, sort supports multiple files as arguments, so you can replace the above with:
sort file1 file2 | uniq
In general, every time you find yourself catting a file into a pipe, re-examine the piped command and see whether it can accept files directly as input first either as direct arguments or as STDIN redirection. For instance, both sort and grep can accept files as arguments as you saw earlier, but if they couldn't, you could achieve the same thing with redirection:
sort < file1 file2 | uniq
grep "foo" < file
The xargs command is very powerful on the command line—in particular, when piped to from the find command. Often you'll use the find command to pick out files that have a certain criteria. Once you have identified those files, you naturally want to pipe that output to some command to operate on them. What you'll eventually discover is that commands often have upper limits on the number of arguments they can accept.
So for instance, if you wanted to perform the somewhat dangerous operation of finding and removing all of the files under a directory that match a certain pattern (say, all mp3s), you might be tempted to do something like this:
find ./ -name "*.mp3" -type f -print0 | rm -f
Of course, you should never directly pipe a find command to remove. First, you should always pipe to echo to ensure that the files you are about to delete are the ones you want to delete:
find ./ -name "*.mp3" -type f -print0 | echo
If you have a lot of files that match the pattern, you'll probably get an error about the number of arguments on the command line, and this is where xargs normally comes in:
find ./ -name "*.mp3" -type f -print0 | xargs echo
find ./ -name "*.mp3" -type f -print0 | xargs rm -f
This is better, but if you want to delete files, you don't need to use a pipe at all. Instead, first just use the find command without a piped command to see what files would be deleted:
find ./ -name '*.mp3" -type f
Then take advantage of find's -delete argument to delete them without piping to another command:
find ./ -name '*.mp3" -type f -delete
So next time you find your pinky finger stretching for the pipe key, pause for a second and think about whether you can combine two commands into one. Your efficiency and poor overworked pinky finger (whoever thought it made sense for the pinky to have the heaviest workload on a keyboard?) will thank you.
It's not required (much) anymore.
I just knew legalization was a bad idea. I think we should all put down the pipe.
Thank you for putting in the disclaimer, Because this even scared me. Way over my pay grade and capabilities. I won’t be piping my cats anywhere. lol
LOL! Ya think??
You and me and about a dozen others still alive - everyone else is GUI-bound.
Dumb article that misses the whole point of the “Unix way”, which IS to stitch various simple commands together to accomplish a task
...
# vi hellfrworld.sh
vi> echo "Hello Freeper World."
vi> exit(0)
vi> :wq
# make hellofrworld
make> cp hellofrworld.sh hellofrworld
make> chmod +x hellofrworld
#
#./hellofrworld
Hello Freeper World.
#
If used the bourne, korn, csh, tcsh, wksh, and of courcs bash.
The ultimate shell however is the emacs editor.
The command line rules!
The command line language itself is a powerful programming tool and is frequently use used to automate administrative tasks and failrly complex programs and is frequently the glue invoked behind the scenes of a Graphical User Interface (GUI) that i merely passed the parameters that were selected on the GUI.
When I was a grad student most MS thesis were on the design of Graphical User Interfaces from someone getting their MS CSA with a Human Computer Interaction concentration.
We affectionately called GUI's the idiot interface back in the day!
antediluvian - of course!
But I degress!
While the "Unix way" is to stitch various command together to get a job done, as you say, there can be more efficient ways of writing those commands.
No point in chaining commands together when one command will do the whole job.
That's a pity.
If I could spell and type at the same time I’d rule the world!
I also hate cats! Sorry viking kitty. I just do. Almost as much as I hate Democrats. (LOL)
I see what you did there.
Yup. I frequently write script-writing scripts, with enough intelligence in them to parse out information from the input file so that it can do many different things on a per-server basis.
With scripts, I can verify SAN connections, check and reset SNMP, perform multiple-password logins, configure network bonding/teaming, make configuration changes across 1500+ servers in less time than most people can do 10 servers, check ssh connections to any number of servers, and, of course, perform our monthly scheduled updates on all of our servers, determing whether or not to reboot them afterwards, and to notify the application owner of the update/reboot beforehand.
With scripts, I have singlehandedly lowered the effort of our team from a month-long task to something a single person can do in less than haf an hour.
I love the command line.
Or Democats, for that matter...
One thing cool about it though, at least with Linux you can do anything you want to do with it if you like.
I pity those that are unaware of the command line powers, particularly for bypassing recalcitrant GUI functions that don’t do what they are supposed to do.
I don’t have my photos on a server so that I can’t post one here but my response would be a military K-9 dog with really wide open mouth with exposed death, growling and salivating and barking with a caption that says, “Go ahead and run! He likes fast food!”
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.