Standard Input/Output and Pipes

  • You should be able to :
    • Redirect I/O channels to files
    • Connect commands using pipes
    • Use the for loops to iterate over sets of values
  1. Standard Input and Output
  2. Redirecting Output to a file
  3. Useful Pipe Targets
  4. Combining Output and Errors

Standard Input and Output

  • Linux provides three I/O channels to Programs
    • Standard input (STDIN) – Keyboard by Default
    • Standard output (STDOUT) – terminal windows by default
    • Standard error (STDERR) – terminal windows by default

Input/Output Streams

One of the most important features of Linux (and UNIX) is the ability to redirect a command’s input, output and error data. In general, this allows the input from a program to come from any source, and the output to go to any source. In addition, the output from one command can be fed directly to the input of another command.

Standard input defaults to the keyboard. Most commands use files as their source of input, but will accept standard input from other sources, via redirection.

Standard output, by default, is the screen or terminal window. Most commands send their output to standard output without explicitly being told to do so. When standard output is sent to a destination other than the screen, such as a file, one is redirecting standard output.

But there is a second output data stream. It is called the standard error. This is a secondary output stream that carries warnings, usage messages, error messages and other “out-of-band” information, in an output stream distinct from standard output. This error stream is normally sent to the screen but may be redirected elsewhere. Consider the following command and its output, which assumes that there is a file called myfile in your home directory, but no file called fakefile.

amar@amar-pc:~$ ls -l myfile fakefile
ls: cannot access 'fakefile': No such file or directory
-rw-rw-r-- 1 amar amar 0 Nov 20 00:41 myfile
Standard Input/Output and Pipes

Though both lines are being printed to the screen, the first line is directed to the standard error stream and the second line to the standard output. Later we will see how the standard output and standard error can be redirected to other locations.

These streams are abbreviated as STDIN (called file descriptor number 0), STDOUT (file descriptor number 1), and STDERR (file descriptor number 2). The fact that there are two output channels allows separation of error messages from normal output. For example, error messages could be saved in a file with the normal output going to the monitor.

Redirecting Output to a File

  • STDOUT and STDERR can be redirected to files:
    • command operator filename
  • Supported operators include:
    • > Redirect STDOUT to file
    • 2> Redirect STDERR to file
    • &> Redirect all output to file
  • File contents are overwritten by default. >> appends.

Altering I/O Sources and Destinations

The standard output of commands, which ordinarily displays on the terminal, can be redirected into a file or piped into another command.

Standard error, which also ordinarily displays on the terminal, can be redirected into a file. Although it is also possible to pipe standard error into a command using some fairly complex syntax, this is generally not done.

Standard input, ordinarily coming from the keyboard, can be redirected from a file. More commonly, the standard output of one command can be piped into the standard input of another command.

Common Redirection Operators

command > file – Direct standard output of command to file:

command >> file – Append standard output of command to file

command < file – Send file as input to command

command 2> file – Redirect error messages from command to file

command 2>> file – Append error messages from command to file


  • This command generates output and errors when run as non-root:
    $ find /etc -name passwd
  • Operators can be used to store output and errors:
    $ find /etc -name passwd > find.out
    $ find /etc -name passwd 2> /dev/null
    $ find /etc -name passwd > find.out 2> find.err

Redirecting STDOUT and STDERR

amar@amar-pc:~$ find /etc -name passwd
find: ‘/etc/polkit-1/localauthority’: Permission denied
find: ‘/etc/cups/ssl’: Permission denied
find: ‘/etc/ssl/private’: Permission denied
Standard Input/Output and Pipes

The second line of output is printed to STDERR. It is the result of find attempting to access a directory that the non-privileged user running it does not have permission to read. The third line of output is printed to STDOUT and shows an accessible file the command has found. Note what happens when the same command is run, but STDOUT is redirected to a file:

amar@amar-pc:~$ find /etc -name passwd > find.out
find: ‘/etc/polkit-1/localauthority’: Permission denied
find: ‘/etc/cups/ssl’: Permission denied
find: ‘/etc/ssl/private’: Permission denied
amar@amar-pc:~$ cat find.out
Standard Input/Output and Pipes

Because STDERR and STDOUT are distinct, redirecting one does not affect the other. The following command would redirect STDERR (also known as “file descriptor 2”) instead of STDOUT:

amar@amar-pc:~$ find /etc -name passwd 2> find.err
Standard Input/Output and Pipes

The trick here is to remember that within the program STDERR is identified by the number 2, hence 2>, meaning “redirect file descriptor 2”. STDOUT is identified by the number 1, but since redirecting it is the default behavior when > is used, it rarely needs to be stated explicitly.

Redirecting the STDERR of a program means that you will only see its output, which is helpful in situations like the above where you know the command will generate some errors but do not care about them enough to want to see them, But if you really do not care about the errors, why waste a file storing them? There is a special file on your system that is very useful in this sort of situation, /dev/null. /dev/null is a black hole for data. Anything that is sent to is simply ignored. Hence, to store the output of this find command, but ignore its errors, you could run:

amar@amar-pc:~$ find /etc -name passwd > find.out 2> /dev/null
Standard Input/Output and Pipes

Overwriting vs Appending

Try running one of the above commands repeatedly. You will notice that every time output is redirected to a file, the contents of that file are destroyed or “overwritten”. Suppose you wanted to run a command multiple times over the course of a day or a week and store its output each time? This can be done by using the >> operator (2>> for STDERR) instead of >.

Standard Input/Output and Pipes

Redirecting STDOUT to a Program (Piping)

  • Pipes (the | character) can connect commands: –

    command1 | command2
    • Sends STDOUT of command1 to STDIN of command2 instead of the screen.
    • STDERR is not forwarded across pipes
  • Used to combine the functionality of multiple tools
    • command1 | command2 | command3… etc.

Understanding Pipes

Two of the basic tenets of UNIX philosophy are: make small programs that do one thing well, and expect the output of every program to become the input to another, as yet unknown, program. The use of pipes let you leverage the effects of these two design principles to create extremely powerful chains of commands to achieve your desired result. Most command-line programs that operate on files can also accept input on standard in, and output to standard out. Some commands like tr are designed to only operate within a pipe (can not operate on files directly).

Any command that writes to standard output can be used on the left-hand side of a pipe.

Any command that reads from standard input can be used on the right-hand side of a pipe.

Multiple commands can be chained together with pipes.

Example output

amar@amar-pc:man$ pwd
amar@amar-pc:man$ ls -C | tr 'a-z' 'A-Z'
Redirecting STDOUT to a Program (Piping)

In the example above, lower case letters of the alphabet from the file listing are converted to upper case.

Useful Pipe Targets

  • less: View input one page at a time:

    $ ls -l /etc | less
    • input can be searched with /
  • mail: Send input via email:

    $echo "test email" | mail -s "test"
  • lpr : Send input to a printer:

    $ echo "test print" | lpr
    $ echo "test print" | lpr -P printer_name

Piping to Email

The traditional mail command is rather old fashioned for a mailer. But, it is extremely useful for mailing the standard output of a command. The -s option allows you to specify a subject for the email.

Piping to a Printer

You can also pipe output to the lpr command. this will send the output to your system’s default printer. If you want to use a non-default printer you can specify one with the -P option.

Combining Output and Errors

Leave a Comment

%d bloggers like this: