Bash Pipelines


  • [time [-p]] [!] command1 [ | or |& command2 ] …


A pipeline is a sequence of simple commands separated by one of the control operators | or |& (source).

| connects the output of command1 to the input of command2.

|& connects standard output and standard error of command1 to the standard input of command2.

Show all processes paginated

ps -e | less

ps -e shows all the processes, its output is connected to the input of more via |, less paginates the results.

Using |&

|& connects standard output and standard error of the first command to the second one while | only connects standard output of the first command to the second command.

In this example, the page is downloaded via curl. with -v option curl writes some info on stderr including , the downloaded page is written on stdout. Title of page can be found between <title> and </title>.

curl -vs '' |& awk '/Host:/{print} /<title>/{match($0,/<title>(.*)<\/title>/,a);print a[1]}'

Output is:

> Host:

But with | a lot more information will be printed, i.e. those that are sent to stderr because only stdout is piped to the next command. In this example all lines except the last line (Google) were sent to stderr by curl:

* Hostname was NOT found in DNS cache
*   Trying
* Connected to ( port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host:
> Accept: */*
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Date: Sun, 24 Jul 2016 19:04:59 GMT
< Expires: -1
< Cache-Control: private, max-age=0
< Content-Type: text/html; charset=ISO-8859-1
< P3P: CP="This is not a P3P policy! See for more info."
< Server: gws
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< Set-Cookie: NID=82=jX0yZLPPUE7u13kKNevUCDg8yG9Ze_C03o0IM-EopOSKL0mMITEagIE816G55L2wrTlQwgXkhq4ApFvvYEoaWF-oEoq2T0sBTuQVdsIFULj9b2O8X35O0sAgUnc3a3JnTRBqelMcuS9QkQA; expires=Mon, 23-Jan-2017 19:04:59 GMT; path=/;; HttpOnly
< Accept-Ranges: none
< Vary: Accept-Encoding
< X-Cache: MISS from jetsib_appliance
< X-Loop-Control: 81E4F9836653D5812995BA53992F8065
< Connection: close
{ [data not shown]
* Closing connection 0

Modify continuous output of a command

~$ ping -c 1 # unmodified output
PING ( 56(84) bytes of data.
64 bytes from ( icmp_seq=1 ttl=53 time=47.4 ms
~$ ping | grep -o '^[0-9]\+[^()]\+' # modified output
64 bytes from 
64 bytes from 
64 bytes from 
64 bytes from 
64 bytes from 
64 bytes from 
64 bytes from 
64 bytes from 
64 bytes from 
64 bytes from 

The pipe (|) connects the stdout of ping to the stdin of grep, which processes it immediately. Some other commands like sed default to buffering their stdin, which means that it has to receive enough data, before it will print anything, potentially causing delays in further processing.