Shell Fundamentals

Why This Matters

Most engineers learn shell skills through osmosis — picking up a command here, a trick there, over years of stumbling through Stack Overflow answers. This is wildly inefficient. The core concepts fit in your head in a day, and they pay dividends for your entire career.

The shell is the most universal developer interface. Languages come and go, frameworks churn, but pipes, redirects, and environment variables work the same way they did 40 years ago. Every server you SSH into, every Docker container you debug, every CI pipeline you troubleshoot — they all speak shell.

If you can't navigate a terminal fluently, you're working with one hand tied behind your back.

Pipes

Pipes are the shell's killer feature. They let you chain simple commands into powerful compositions.

The pipe operator | takes the standard output (stdout) of the command on the left and feeds it as standard input (stdin) to the command on the right.

# Count how many Go files are in the project
find . -name "*.go" | wc -l

# Find the 10 largest files in the current directory tree
du -ah . | sort -rh | head -10

# Show unique HTTP status codes from an access log, sorted by frequency
awk '{print $9}' access.log | sort | uniq -c | sort -rn

# Find all TODO comments and show just the filenames
grep -r "TODO" src/ | cut -d: -f1 | sort -u

The philosophy behind pipes is the Unix philosophy: each tool does one thing well, and pipes let you compose them. grep searches, sort sorts, uniq deduplicates, wc counts. Learn these small tools and you can build almost any text-processing pipeline on the fly.

Common pipe commands worth knowing

grep    - Search for patterns
sort    - Sort lines
uniq    - Remove adjacent duplicates (use with sort)
wc      - Count lines, words, characters
head    - Show first N lines
tail    - Show last N lines (tail -f for live following)
cut     - Extract columns from text
awk     - Pattern scanning and processing
sed     - Stream editing (find and replace)
tr      - Translate or delete characters
xargs   - Build command lines from stdin
tee     - Write to both stdout and a file

Redirects

Redirects control where output goes and where input comes from.

Every process has three standard streams:

stdin  (0) - Standard input
stdout (1) - Standard output
stderr (2) - Standard error

Output redirection

# Write stdout to a file (overwrites)
echo "hello" > output.txt

# Append stdout to a file
echo "world" >> output.txt

# Redirect stderr to a file
make build 2> errors.log

# Redirect both stdout and stderr to the same file
make build > build.log 2>&1

# Modern bash shorthand for the above
make build &> build.log

# Discard all output
make build > /dev/null 2>&1

Input redirection

# Feed a file as stdin
sort < unsorted.txt

# Here document (multi-line input)
cat << EOF
This is line one.
This is line two.
EOF

# Here string (single-line input)
grep "error" <<< "this is an error message"

Useful patterns

# Save output to a file AND see it on screen
make build 2>&1 | tee build.log

# Run a command only if the previous one succeeded
make build && make test

# Run a command only if the previous one failed
make build || echo "Build failed!"

Environment Variables

Environment variables are key-value pairs that configure how programs behave. They're inherited by child processes, which is what makes them useful for configuration.

# Set a variable for the current session
export DATABASE_URL="postgres://localhost:5432/mydb"

# Set a variable for a single command only
DATABASE_URL="postgres://test:5432/testdb" python manage.py test

# View a variable
echo $DATABASE_URL

# View all environment variables
env

# Unset a variable
unset DATABASE_URL

Variables vs environment variables

There's a subtle but important distinction:

# Shell variable (only visible in current shell)
MY_VAR="hello"

# Environment variable (visible to child processes)
export MY_VAR="hello"

If you set a variable without export, scripts and programs you launch won't see it. This is a common source of confusion when configuring applications.

Important environment variables

PATH        - Where the shell looks for executables
HOME        - Your home directory
USER        - Your username
SHELL       - Your default shell
EDITOR      - Your preferred text editor
TERM        - Your terminal type
LANG        - Your locale settings

PATH

PATH is the most important environment variable you'll deal with. When you type a command like python or node, the shell searches the directories listed in PATH, in order, until it finds a matching executable.

# View your PATH (colon-separated list of directories)
echo $PATH

# Typical output:
# /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

# Add a directory to PATH (prepend — searched first)
export PATH="$HOME/.local/bin:$PATH"

# Add a directory to PATH (append — searched last)
export PATH="$PATH:$HOME/go/bin"

# Find which executable the shell will use
which python
# /usr/local/bin/python

# See ALL matching executables in PATH
which -a python

Debugging PATH issues

When a command "isn't found" or the wrong version runs, it's almost always a PATH issue.

# "command not found" — the binary isn't in any PATH directory
# Fix: find where it's installed and add that directory to PATH

# Wrong version running — another version appears earlier in PATH
# Fix: reorder PATH so the correct directory comes first

# Changes don't persist — you edited PATH in the shell, not in your rc file
# Fix: add the export line to ~/.bashrc or ~/.zshrc

RC Files

RC files (run commands) are scripts that execute when your shell starts. They're where you configure your shell environment.

Which file to edit

This depends on your shell and how you start it:

Bash:
  ~/.bashrc        - Interactive non-login shells
  ~/.bash_profile  - Login shells (often sources .bashrc)

Zsh:
  ~/.zshrc         - Interactive shells (most common to edit)
  ~/.zshenv        - All shells (including non-interactive)
  ~/.zprofile      - Login shells

Fish:
  ~/.config/fish/config.fish - All interactive shells

For most purposes, edit ~/.zshrc (macOS default) or ~/.bashrc (Linux default). If you're not sure which shell you're using, run echo $SHELL.

What goes in your RC file

# PATH additions
export PATH="$HOME/.local/bin:$PATH"
export PATH="$HOME/go/bin:$PATH"

# Environment variables
export EDITOR="vim"
export LANG="en_US.UTF-8"

# Aliases (covered in detail in the next section)
alias ll="ls -la"
alias gs="git status"

# Tool initialization
eval "$(starship init zsh)"
source "$HOME/.cargo/env"

# Custom functions
mkcd() {
    mkdir -p "$1" && cd "$1"
}

Reloading changes

After editing your RC file, either open a new terminal or source the file:

source ~/.zshrc
# or
source ~/.bashrc

Practical Patterns

Command substitution

Use the output of a command as part of another command:

# Create a directory with today's date
mkdir "backup-$(date +%Y-%m-%d)"

# Kill a process by name
kill $(pgrep -f "my-server")

# Set a variable from command output
CURRENT_BRANCH=$(git branch --show-current)

Brace expansion

Generate multiple strings without repetition:

# Create multiple directories
mkdir -p project/{src,test,docs,config}

# Create numbered files
touch file_{01..10}.txt

# Rename with backup
cp config.yaml{,.backup}
# Expands to: cp config.yaml config.yaml.backup

Process management & history

Run a command in the background with &, list jobs with jobs, bring one forward with fg %1. Suspend a running process with Ctrl+Z, then bg to continue it in the background.

Search your command history with Ctrl+R — start typing and it finds the most recent match. Run !git to re-execute the last command starting with "git". Use ^old^new to re-run the last command with a substitution.

Common Pitfalls

Editing the wrong RC file. On macOS with Zsh, edit ~/.zshrc. On Linux with Bash, edit ~/.bashrc. Editing ~/.bash_profile when you should be editing ~/.bashrc (or vice versa) is a classic source of "it works in one terminal but not another."

Forgetting to export environment variables. If a program can't see a variable you set, you probably forgot export. Without it, the variable exists only in the current shell, not in child processes.

Overwriting PATH instead of appending to it. Never do export PATH="/my/new/path" — this replaces your entire PATH and breaks everything. Always do export PATH="/my/new/path:$PATH".

Not quoting variables. $MY_VAR without quotes can break on spaces or special characters. Use "$MY_VAR" in almost all cases.

Ignoring exit codes. Every command returns an exit code (0 for success, non-zero for failure). Use $? to check it, and use && to chain commands that depend on each other.

Key Takeaways

Pipes let you compose simple tools into powerful pipelines. Learn the core utilities (grep, sort, uniq, awk, sed, xargs) and you can process almost any text data without writing a script.

Redirects give you control over where data flows. Understanding stdout, stderr, and the difference between > and >> prevents data loss and simplifies debugging.

PATH determines which programs your shell can find. When something isn't working, PATH is the first thing to check.

Your RC file is your shell's configuration. Invest time in setting it up properly — every terminal you open for the rest of your career will benefit.

These are not advanced topics. They're fundamentals that most engineers learn piecemeal over years. Spend a focused afternoon with them and you'll be ahead of engineers with twice your experience.