Linux Data Streams and Redirection

This post explains how input, output, and errors work in Linux, It also shows how to use redirection and pipes to connect commands and manage results in a simple way.

What You’ll Learn

• What are data streams (stdin, stdout, stderr)
• How to redirect outputs and errors
• How to use pipes to connect commands
• Real examples from daily use

1. What Are Streams

In Linux, every program has three main data streams:

1. stdin (0) – standard input, usually from keyboard
2. stdout (1) – standard output, normal messages
3. stderr (2) – standard error, error messages

Think of them like water pipes. Data flows in and out through these pipes.

This helps us send data to files, separate errors, or connect commands together.

2. Redirecting Output

Send output to a file ( > )

echo "Hello World" > output.txt

This creates or overwrites the file.

Add output instead of overwrite ( >> )

echo "Line 1" > file.txt
echo "Line 2" >> file.txt

Now the file has both lines.

Example use

df -h > disk_usage.txt
date >> my_log.txt
ps aux > running_processes.txt

3. Redirecting Errors

Normal redirection saves only normal output, not errors. To save errors, we use 2>.

ls /fake_folder 2> errors.txt

Redirect both output and errors

command > output.txt 2> errors.txt     # separate files
command > all.txt 2>&1                 # same file
command &> all.txt                     # simpler way

To ignore errors completely:

command 2> /dev/null

4. /dev/null – The Black Hole

/dev/null is a special file that throws away everything sent to it.

Examples

command &> /dev/null           # ignore all output
ping -c 1 google.com &> /dev/null
apt update 2> /dev/null

Use this when you want your command to run quietly.

5. Pipes ( | )

A pipe sends the output of one command into another command.

command1 | command2

Examples

ls | wc -l                       # count files
ps aux | grep nginx              # find a process
du -sh * | sort -hr | head       # largest files
cat file.txt | grep "error"      # search in a file

Combine many commands

cat /var/log/syslog | grep "error" | sort | uniq | wc -l

This reads a log, finds errors, sorts them, removes duplicates, and counts them.

6. Input Redirection (<)

Instead of typing input manually, we can take it from a file.

wc -l < file.txt
sort < unsorted.txt

Here Document (<<)

cat << EOF > file.txt
Line 1
Line 2
EOF

This creates a file quickly with multiple lines.

7. Combine Redirection and Pipes

Save output and show it

command | tee output.txt

Add instead of overwrite

command | tee -a output.txt

Example

sudo apt update | tee update.log

8. Quick Reference

Output Redirection

cmd > file         → save output (overwrite)
cmd >> file        → add output (append)
cmd 2> file        → save errors
cmd &> file        → save output and errors
cmd > /dev/null    → discard output
cmd 2> /dev/null   → discard errors
cmd &> /dev/null   → discard everything

Pipes and Input

cmd1 | cmd2        → send output of first command to the second
cmd < file         → read input from a file
cmd | tee file     → show and save output at the same time

9. Common Mistakes

Wrong

ls > file.txt | grep txt

Output goes to file, so pipe gets nothing.

Right

ls | grep txt > file.txt

Wrong

command > output.txt   # misses errors

Right

command &> output.txt

Wrong

echo "Entry" > log.txt   # overwrites log

Right

echo "Entry" >> log.txt

10. Real Examples

# Log script output
./my_script.sh &> script_log.txt

# Backup with progress
tar -czf backup.tar.gz /home/reza | tee backup.log

# Find today’s errors
grep "$(date +%Y-%m-%d)" /var/log/*.log 2>/dev/null | grep -i error

# Clean old logs
tail -100 /var/log/app.log > temp.log && mv temp.log /var/log/app.log

# Monitor memory
free -h | grep Mem | tee -a memory_log.txt

Key Points I Learned

1. Three data streams: stdin (0), stdout (1), stderr (2)
2. > overwrites, >> appends
3. 2> redirects errors
4. &> redirects both output and errors
5. /dev/null discards output
6. Pipes | connect commands
7. tee shows and saves output
8. Redirection order matters

My Quick Cheat Sheet

command > file.txt         # save output
command >> file.txt        # add output
command 2> errors.txt      # save errors
command &> all.txt         # save all
command 2> /dev/null       # ignore errors
command1 | command2        # pipe
command | tee file.txt     # show and save