Shell: Explanation & Insights

A program that reads the commands you type and asks the operating system to run them; the primary way to control Linux from text.

What It Is

The shell is the program that reads what you type and runs it. That's the whole job, and it's worth saying plainly because the word gets thrown around for three different things that beginners tend to blur together. When you open a terminal and see a $ waiting for you, the thing reading your keystrokes, figuring out what you meant, and launching the right program is a shell — usually bash or zsh. It is, at heart, a read-eval-print loop: it reads a line, evaluates it (works out which program to run and with what arguments), prints whatever that produces, and loops back to the prompt to do it all again. You've been talking to a loop this whole time.

Here is the distinction that clears up most of the confusion, and we'll set it down once and for all so you never have to wonder again. The command line is the line of text you type — the literal characters on that one row. The shell is the program that reads that line and executes it. The CLI — the command-line interface — is the whole text-driven style of working: you type words, the computer does things, no mouse involved. So "the command line" and "the CLI" aren't separate tools you install; they're names for the way of working inside a shell. The command line is what you type, the shell is what reads it, the CLI is the entire approach — the opposite of pointing and clicking at a GUI. One is the sentence, one is the listener, one is the language. Hold those apart and the jargon stops fighting you.

And the name itself is a small gift, because it tells you exactly where this program sits. The shell is named for the hard outer layer wrapped around a core — the shell around the nut, the shell around the seed. The core here is the kernel, the privileged program that actually commands the hardware. You don't get to touch the kernel directly; that would be chaos. Instead you touch the shell, the outer layer the operating system wraps around its kernel, and the shell relays your wishes inward. It is, quite literally, the part of the system you put your hands on — the rind you hold while the kernel does the dangerous work underneath. Once you've seen that, the word stops being arbitrary and starts being a tiny diagram of how the whole machine is built.

The rest of this page goes through everything the shell actually does with a line of text — how it splits your words, expands a * into filenames, swaps $HOME for your home directory, hunts down the program you named, and wires programs together with pipes and redirection. By the end you'll be able to look at a line like grep error *.log > out.txt and narrate, step by step, exactly what the shell is about to do with it.

Why It Matters

Everything you do on a server, you do through the shell. There is no other door. When you ssh into a box, the thing that greets you on the far side is a shell waiting for a command; when you write a deploy script, you're writing instructions for a shell to run unattended; when a cron job fires at 3 a.m., it's a shell that carries it out. A server has no buttons and no windows — the shell is the entire interface, which means your fluency in it is your fluency on the machine.

That matters more on a server than it does on a laptop, and the reason is worth dwelling on. A GUI is wonderful for things you do once: clicking through a settings panel, dragging a file. But a server's work is the opposite of once — it's the same actions, repeated, reliably, often while nobody's watching. The CLI wins precisely because text can be saved, replayed, and automated. A thing you can type is a thing you can put in a script, schedule, version-control, paste into a runbook, and hand to the next person. You cannot save a sequence of mouse clicks and trust it to run itself at midnight. The shell turns "what I did" into "what the machine does from now on," and that conversion — from one-off action to durable, repeatable instruction — is the whole reason serious infrastructure lives at the command line.

There's a payoff in your own head, too. Because the shell is just a program reading text, everything it does is inspectable and composable. You learn a handful of small tools — grep, echo, cd — and the shell lets you bolt them together into combinations nobody shipped, on the spot, for a problem only you have right now. Master the shell and you stop being a person who runs commands and become a person who builds them.

What Happens When You Press Enter

This is the heart of the page, so we'll take it slowly. You type a line, you press Enter, and in the blink before output appears the shell runs through a fixed sequence of steps on your text. Every one of them is worth knowing by name, because when something behaves strangely it's almost always one of these steps doing exactly what it was told rather than what you meant.

Take this line as our specimen:

grep error *.log > out.txt

1. Splitting into words. First the shell chops the line into tokens on whitespace. Here it sees grep, error, *.log, >, out.txt. The first word is special — it's the command, the thing to run. The rest are arguments and operators. This splitting is why spaces matter so much in the shell, and why a filename with a space in it causes such grief unless you protect it (we'll get to quoting).

2. Expansions. Now the shell rewrites certain tokens before the command ever sees them. This is the step that surprises people most, so look closely:

Glob expansion. A *.log isn't passed to grep as the literal text *.log. The shell expands it first, replacing it with the names of the matching files in the current directory — app.log access.log error.log — and grep only ever sees those final names. The * is a wildcard: * matches any run of characters, ? matches exactly one, and [abc] matches any one of the listed characters. This is globbing, and it belongs to the shell, not to the command. Run it for yourself:

echo /etc/*.conf
# /etc/adduser.conf /etc/ca-certificates.conf /etc/debconf.conf /etc/deluser.conf ...

Notice we used echo to see what a glob expands to — a genuinely useful habit, because it shows you exactly what the shell will hand the real command before you run anything destructive with it.

Variable expansion. Any $NAME is replaced with the variable's value. $HOME becomes /home/arndt, $USER becomes your username. The shell does this swap silently and the command never knows a variable was involved — it just receives the final text.

echo "$HOME"
# /home/arndt

There's also tilde expansion (~ becomes your home directory), command substitution ($(date) runs date and pastes its output in), and arithmetic ($((2 + 2)) becomes 4) — all variations on the same theme: the shell rewriting your line before execution.

3. Finding the program. Now the shell has a clean command — grep — and needs to find the actual executable file on disk. It does not search your whole system. It walks a specific list of directories called the $PATH, in order, and runs the first matching file it finds. Your $PATH is just a colon-separated list of folders:

echo "$PATH" | tr ':' '\n'
# /home/arndt/.local/bin
# /usr/local/bin
# /usr/bin
# /bin

That ordering is load-bearing: if a program named grep existed in both /usr/local/bin and /usr/bin, the one earlier in the $PATH wins. Want to know which one will win? Ask which:

which bash
# /usr/bin/bash

Not everything is a file on the $PATH, though. Some commands — cd chief among them — are builtins, baked into the shell itself rather than living as separate programs. That has to be true for cd: changing directory means changing the shell's own state, and an external program can't reach back into its parent to do that. Run type cd and the shell tells you straight: cd is a shell builtin.

4. Fork and exec. For an external command, the shell now does the central trick of Unix: it forks — makes a copy of itself as a new process — and then that copy execs, replacing itself entirely with the grep program. The shell (the parent) waits for grep (the child) to finish, then reaps its exit status and returns you to the prompt. This fork-then-exec dance is how every command you run comes to life. The shell is, in a real sense, a factory for processes.

5. Redirection and pipes. Before the child runs, the shell wires up its inputs and outputs — and our > out.txt is exactly that. The shell opens out.txt and points the command's output channel at it, so what would have hit your screen lands in the file instead. The program doesn't know or care; it just writes to its output as always. We'll give redirection and pipes their own section, because they're where the real power lives.

That's the whole loop. Read, split, expand, find, fork, exec, wire up, wait, repeat. Once you can run that film in your head, the shell stops being magic and becomes a machine you can predict.

Pipes and Redirection

Three small channels exist on every process, and once you know them the shell's plumbing makes complete sense. Every program is born with stdin (where it reads input), stdout (where it writes normal output), and stderr (where it writes errors, kept separate on purpose so error messages don't pollute the data). By default stdin is your keyboard and stdout/stderr are your screen. Redirection and pipes are nothing more than the shell re-pointing those three channels somewhere else before the program starts.

Redirection sends a channel to or from a file:

> sends stdout into a file, replacing its contents. echo hi > greeting.txt writes hi, clobbering whatever was there.
>> sends stdout into a file, appending to the end. echo line2 >> greeting.txt adds to it. The doubled symbol is the whole difference between "overwrite" and "add to" — mix them up and you'll erase a log you meant to grow.
< feeds a file into a program's stdin, so sort < names.txt sorts the file's contents as if you'd typed them.

You can redirect stderr separately by numbering it — stderr is channel 2. Watch the error message land in a file while the screen stays clean:

ls /nonexistent 2> err.txt
cat err.txt
# ls: cannot access '/nonexistent': No such file or directory

That 2> is you telling the shell "send channel 2 to this file." The fact that errors travel on their own channel is one of the quietly excellent decisions in Unix — it means you can capture a program's real output while letting its complaints scroll past, or the reverse.

Pipes are the showstopper. A pipe — the | symbol — connects the stdout of one program directly to the stdin of the next, with no file in between. The first program's output flows straight into the second's input, the two running side by side, the kernel carrying the bytes between them. It's the shell letting you bolt small tools into a bigger one on the spot.

A pipe I actually reach for: I want the busiest processes by memory, but only the top few. No single tool does precisely that — so I build it:

ps aux --sort=-%mem | head -5

ps lists every process and knows nothing about trimming; head keeps the first few lines and knows nothing about processes. Neither was written with the other in mind, yet the pipe joins them into a tool that prints exactly the memory hogs I care about. Chain a third with another | and you've refined it further. Each program stays small and single-minded; the shell does the joining. That composability — little pieces, snapped together fresh for the problem in front of you — is the deepest idea the command line has, and the pipe is the symbol that unlocks it.

Variables, Environment, and Quoting

A shell variable is just a name holding a value. You set one with no spaces around the = (the shell is fussy here — x=hello, never x = hello, because spaces would split it into words), and you read it back with a $:

name=world
echo "Hello, $name"
# Hello, world

There are two flavours, and the difference matters the moment you run a program. A plain shell variable lives only inside the current shell. An environment variable is one you've marked for export, so it gets copied into every child process the shell spawns. That's how settings reach the programs you launch — $PATH, $HOME, $LANG, $TERM are all environment variables, inherited by everything you run. You promote a plain variable to an environment one with export:

export EDITOR=vim

Now any program you start can see $EDITOR. To list everything currently in your environment, printenv dumps the lot. The mental model: plain variables are private notes to yourself; exported ones are announcements your children inherit.

Quoting is the other half of the story, and it's where careful people separate from frustrated ones — because quoting controls which of those expansions from earlier actually happen. Three levels:

No quotes — full expansion and word-splitting. The shell expands globs and variables, then splits the result on whitespace.
Double quotes "..." — expansion happens ($HOME still becomes your home directory) but word-splitting and globbing do not. The string stays one piece.
Single quotes '...' — nothing expands. Every character is literal. '$HOME' is the five characters dollar-H-O-M-E, not your home directory.

The difference is sharp and worth seeing run live. Put two words in a variable and watch what quoting does:

x="a b"
for w in $x;   do echo "[$w]"; done
# [a]
# [b]
for w in "$x"; do echo "[$w]"; done
# [a b]

Unquoted, $x expanded to a b and the shell split it into two words. Quoted, it stayed a single value. This is the exact bug behind a thousand broken scripts: an unquoted variable holding a filename with a space in it gets silently split into two arguments, and your command does something baffling. The rule that saves you: quote your variables — wrap "$var" in double quotes by default, and only drop them when you specifically want splitting or globbing.

Pro Tip

When a command misbehaves, put echo in front of it. Because the shell does all its expanding before the command runs, echo rm *.tmp shows you the exact list of files the shell is about to hand rm — without deleting anything. You're inspecting the shell's work before trusting it with the real command. This one habit has saved more data than any backup.

The Shell Families

There are several shells, and they're cousins more than rivals. Knowing the family tree saves you from confusion when a script that runs in one chokes in another.

sh — the original Bourne shell, or these days a minimal POSIX-standard shell. It's the lowest common denominator, the dialect every Unix-like system understands. A #!/bin/sh script aims to run anywhere, which is exactly why you keep it plain. On many systems sh is just a symlink to a more capable shell running in a stripped-down compatibility mode.
bash — the Bourne-Again Shell, the default on nearly every Linux distribution and the one you'll meet most. It's sh with decades of conveniences bolted on: arrays, better arithmetic, smart tab-completion, command history. When someone says "shell scripting" without qualification, they almost always mean bash.
zsh — the Z shell, bash-compatible for the most part but richer for interactive use: cleverer completion, theming, glob patterns that go further. It became the default login shell on macOS, which is why it's everywhere among developers.
fish — the friendly interactive shell, which breaks from tradition for a nicer day-to-day experience: autosuggestions as you type, syntax highlighting on the command line, sane defaults out of the box. The trade-off is honest — fish deliberately is not POSIX-compatible, so sh/bash scripts won't run in it unchanged.

My standing advice: learn one shell well rather than dabbling in all of them. The concepts — expansion, pipes, redirection, quoting — are identical across the family, so depth in one transfers almost completely. And for scripting, default to bash (or strict sh when you need maximum portability). It's installed everywhere, every answer online is written for it, and it has no surprises waiting for the next person who reads your script. Use fish or zsh interactively if they make you happy — but write your scripts in the lingua franca.

Interactive vs Scripting, Login vs Non-Login

The same shell program wears two hats, and they explain a config puzzle that trips up nearly everyone.

Interactive vs scripting is about who's typing. An interactive shell has a human at the keyboard: it shows a prompt, keeps a history, completes filenames when you hit Tab. A non-interactive shell is running a script — no prompt, no human, just a list of commands executed top to bottom and then gone. Same bash binary; it simply behaves more chattily when it knows you're watching.

Login vs non-login is about whether this shell is the start of a session. A login shell is the first shell you get when you authenticate — when you ssh into a box, that initial shell is a login shell, and it reads your profile files (/etc/profile, then ~/.bash_profile or ~/.profile) to set up the session: your $PATH, your environment, the works. A non-login shell is one started within an already-established session — open a new tab in your terminal and that shell skips the profile and reads ~/.bashrc instead, on the assumption the session was already set up by the login shell above it.

This is the source of the classic head-scratcher: you add a line to ~/.bashrc and it works in your local terminal but mysteriously not over ssh, or vice versa, because the two paths read different files. The fix most people land on is to have ~/.bash_profile simply source ~/.bashrc so both roads lead to the same config. Knowing which file a given shell reads, and when, turns that frustration into a thirty-second fix.

Cheat Sheet

The moves worth committing to muscle memory, grouped by what they do.

# --- Expansion: see what the shell will actually do ---
echo *.log              # show what a glob expands to (before running anything risky)
echo "$HOME"            # show a variable's value
echo $(date)            # command substitution: run a command, paste its output
echo $((3 * 4))         # arithmetic expansion -> 12

# --- Finding programs ---
which bash              # which executable on $PATH a name resolves to
type cd                 # is it a builtin, an alias, or a file?
echo "$PATH"            # the search path, colon-separated

# --- Variables and environment ---
name=value              # set a shell variable (NO spaces around =)
export NAME=value       # set an environment variable (children inherit it)
printenv                # dump the whole environment
unset NAME              # remove a variable

# --- Redirection ---
cmd > file              # stdout to file (overwrite)
cmd >> file             # stdout to file (append)
cmd < file              # file into stdin
cmd 2> file             # stderr to file
cmd > file 2>&1         # stdout AND stderr to the same file

# --- Pipes ---
cmd1 | cmd2             # stdout of cmd1 becomes stdin of cmd2
ps aux | grep nginx     # filter a process list

# --- Quoting ---
echo "$var"             # expands $var, but no word-splitting (the safe default)
echo '$var'             # fully literal: prints the text $var

Note

An alias is a shortcut you define for a longer command — alias ll='ls -lh' makes ll expand to the longer form every time you type it. Set one with alias, and put the ones you want permanently in your ~/.bashrc so they survive a new session. Aliases are pure interactive convenience; they're expanded by the shell before anything runs, and they don't carry into scripts.

Reading It by Example

Pattern recognition is the fastest way to fluency. Glance at each line and narrate what the shell does with it:

cat access.log | grep 404 | wc -l — three programs joined by pipes: pour out the file, keep only lines containing 404, count them. The shell builds a "count of 404s" tool out of parts none of which knew about the others.
tail -f app.log — no shell magic at all, just one program the shell forks and execs; it runs until you stop it.
sort names.txt > sorted.txt — fork sort, but first point its stdout at sorted.txt. The sorted output never touches the screen; it lands in the file.
echo "Backing up to $HOME" — the shell expands $HOME inside the double quotes, then runs echo with the finished sentence. Single quotes here would have printed the literal $HOME.
cp file{,.bak} — brace expansion, a bash flourish: the shell turns file{,.bak} into file file.bak, so this copies file to file.bak in one stroke. Pure shell rewriting; cp just sees two filenames.
rm *.tmp — the shell expands *.tmp to every matching file first, then hands the whole list to rm. If no file matches, bash by default passes the literal *.tmp through — and rm complains it can't find a file called *.tmp. That surprise is globbing showing its seams.

History and Philosophy

The shell is as old as Unix itself, and its story is the story of a deliberate, almost stubborn idea: that the way you control the computer should be an ordinary program, not a privileged part of the system welded in. On many other operating systems of the era, the command interpreter was baked into the core. Unix made a different call — the shell would be a regular program like any other, swappable, replaceable, no more special than ls. That single decision is why you can choose between bash, zsh, and fish today; the shell was never the kernel's business, so anyone could write a better one.

The first widely used shell was the Thompson shell in 1971, but the one that set the template was Stephen Bourne's Bourne shell (sh) in 1979 — the ancestor whose syntax still echoes in every #!/bin/sh script running on the planet. Then came a fork in the road. The Berkeley crowd wrote the C shell (csh) with a more C-like syntax and nicer interactive features; the C shell's interactivity was pleasant and its scripting notoriously fragile, which is a large part of why the world eventually settled back on the Bourne lineage for serious work. In 1989 the GNU project shipped bash — the Bourne-Again Shell, a pun that tells you precisely its mission: be the Bourne shell, reborn with everything good the others had learned. It became the default almost everywhere and has held the crown for three decades.

And here's the thread that ties the shell to the whole of Unix philosophy. Doug McIlroy's idea — small programs that each do one thing well, strung together with plain text — needed something to do the stringing. The pipe was his invention; the shell is what makes the pipe usable, the conductor standing in front of an orchestra of single-minded little tools, cueing each to pass its output to the next. The shell doesn't try to be powerful by being big. It's powerful because it's the joint between everything else — the thin layer that lets a hundred sharp little tools become any tool you need. Pull that thread and it unravels into the entire design of Unix: the reason your server is a box of small programs you can recombine at will, rather than one giant application you can only obey.