S.M.A.R.T.: Explanation & Insights

Your disks keep a diary about their own death. The question is whether you read it in time.

What It Is

S.M.A.R.T. stands for Self-Monitoring, Analysis and Reporting Technology, and it is exactly what it sounds like: firmware inside every modern HDD, SSD, and NVMe drive that tracks dozens of internal health counters — temperature, hours powered on, sectors remapped, read errors corrected, write cycles burned — and makes them available to the operating system on request. The drive doesn't just store your data; it watches itself storing your data and takes notes. Those notes are S.M.A.R.T. attributes, and reading them is the single most direct way to know whether a disk is healthy, aging, or actively dying.

The technology has been shipping in every consumer and enterprise drive since the mid-1990s, which means every server you will ever touch has it. It runs silently in the background with no performance cost, no configuration, and no setup — you just have to ask. The tool that asks is smartctl, from the smartmontools package, and a single command — smartctl -a /dev/sda — dumps the entire diary in one shot. If you administer a server and have never run that command, go run it now. It takes two seconds, and whatever it shows you is already true whether you look or not.

Here is the catch, and the reason this page exists: S.M.A.R.T. is necessary but not sufficient. It catches the slow deaths — surface degradation, spare pool exhaustion, controller errors accumulating over weeks and months — and those are the majority. But a power surge, a head crash, a controller that dies mid-write — sudden catastrophic failures produce no warning, no graceful countdown, just silence. Backblaze, who run over 250,000 drives and publish the data, found that S.M.A.R.T. attributes predicted roughly 50-60% of failures in advance. That is genuinely valuable — half your future drive replacements announced before they hurt — but it means the other half arrive unannounced. So S.M.A.R.T. is your early warning system, not your only defence. You still need RAID and backup. S.M.A.R.T. tells you the storm is coming; RAID keeps the lights on during it; backup lets you rebuild after it.

Why It Matters

The argument is simple: drives fail, and most of them give warning first.

A spinning HDD with an annualised failure rate of 1-2% doesn't flip from healthy to dead in one tick. It develops bad sectors — small patches of magnetic surface that can no longer hold a bit reliably. The drive's own firmware detects these during normal reads and writes, remaps them to a reserved spare area, and increments a counter. That counter — Reallocated_Sector_Ct, attribute #5 — is the drive whispering "I'm losing ground." If you're watching it, you have days, weeks, sometimes months to plan a replacement, order the hardware, schedule the swap during business hours, and do the whole thing calmly. If you're not watching it, you find out when the spare pool is exhausted, the filesystem hits an uncorrectable read, and the kernel drops the mount to read-only at 3 a.m. on a Saturday.

The same logic applies to SSDs, just on a different axis. Flash cells wear out after a finite number of write cycles (the program/erase endurance), and the drive tracks how much of its write budget it has consumed. When the endurance counter reaches 100%, the drive is past its rated life — not broken, not losing data, just past warranty. An NVMe drive expresses this as Percentage Used, and it's the most misunderstood number in all of S.M.A.R.T.: a drive at 100% used is like a car past its factory service interval. It probably has years left. It just needs closer attention.

The point is: the information exists, it costs nothing to read, and ignoring it is the difference between a planned swap and an emergency. Every minute you spend understanding S.M.A.R.T. pays back tenfold in nights you sleep through.

The Attributes That Actually Matter

A full S.M.A.R.T. report can show forty or fifty attributes, most of which are vendor trivia you will never need. The art is knowing which handful actually predict failures — and ignoring the rest. Here they are, in the order a working admin should care about them.

The Pre-Fail Trio

These three attributes are the core of disk health monitoring. Any one of them non-zero and climbing deserves your immediate attention. All three non-zero means you are watching a drive die in real time.

ID	Name	What It Counts	What Non-Zero Means
5	Reallocated_Sector_Ct	Sectors that failed and were remapped to the spare pool	The surface is degrading. A few over years = normal aging. Rising between readings = active failure.
197	Current_Pending_Sector	Sectors that read badly and are waiting to be remapped or recovered	The freshest evidence: these failed this read cycle. They either recover or graduate to reallocated/uncorrectable.
198	Offline_Uncorrectable	Sectors the drive tried to recover during offline testing and could not	Data that lived there is gone. Non-zero is serious — these are the sectors that didn't make it.

Think of them as a pipeline. A sector starts healthy, then reads badly (197 — pending), then either gets remapped to a spare (5 — reallocated) or can't be recovered at all (198 — uncorrectable). Watching the flow through this pipeline tells you whether a drive is stable-but-scarred (a few in #5, zero in #197 and #198) or actively hemorrhaging (all three climbing).

Best practice: any non-zero Reallocated_Sector_Ct on an HDD means start planning a replacement. Not panicking — planning. Order the spare, schedule the window, and watch the counter weekly. If it climbs, accelerate the timeline. If it's been stable for months, it might hold for years — but you've already lost your margin for surprise.

The Fleet Predictor

187 — Reported_Uncorrect. Errors the drive handed back to the operating system unfixed — the count behind those I/O error lines in dmesg. Backblaze's annual drive statistics reports consistently show attribute 187 as one of the strongest single predictors of imminent failure across hundreds of thousands of drives. When #187 starts climbing, the drive is no longer handling its own problems — it's pushing them upstream to your kernel, your filesystem, and your users.

The Cable Impostor

199 — UDMA_CRC_Error_Count. This is the one that saves you from an expensive mistake, because it is not a disk problem. CRC errors happen on the SATA cable between the drive and the controller — electrical noise, a loose connector, a kinked ribbon cable. A non-zero value that appeared once and stopped is usually a one-time glitch during a hot-plug or a vibration. A climbing value means the cable needs reseating or replacing — not the drive. Swapping a perfectly healthy disk because of CRC errors, then watching the counter climb on the replacement through the same bad cable, is one of the classic admin mistakes. When #199 is climbing but #5, #197, #198, and #187 are all zero: it's the cable. See disk cable errors for the full walk-through.

The Endurance Counter (Not an Emergency)

202 / 233 / Wear_Leveling_Count / Media_Wearout_Indicator — different vendors, different names, same idea: how much of an SSD's write endurance has been consumed. This counts age, not damage. An SSD at 100% endurance is past its rated write budget the way a car is past its warranty mileage — still running, still storing data, just past the manufacturer's comfort zone. It is worth monitoring (especially on write-heavy database servers) but it is never a 3 a.m. emergency. See SSD worn out.

Raw vs Normalized: The Trap

Every S.M.A.R.T. attribute has two representations, and confusing them is the number-one beginner mistake. Here is what a single row looks like in smartctl output:

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   100   100   005    Pre-fail  -           0

The VALUE / WORST / THRESH columns are normalized — vendor-defined scores that count down from some starting point (usually 100 or 200) toward THRESH. When VALUE reaches THRESH, the drive considers the attribute "failed." A VALUE of 100 does not mean "100 bad sectors" — it means pristine, full marks, the best possible score. The normalization formula is proprietary, vendor-specific, and undocumented. Different manufacturers use different scales for the same attribute ID.

The column you actually want is RAW_VALUE — the honest count. RAW_VALUE = 0 for Reallocated_Sector_Ct means zero sectors have been remapped. RAW_VALUE = 47 means forty-seven. No normalization, no vendor math, just the number. When in doubt, read the raw.

Pro Tip

Some vendors pack multiple values into a single RAW_VALUE field. Temperature is a common offender: RAW_VALUE = 35 (Min/Max 18/52) crams current, minimum, and maximum into one cell. Power-on hours sometimes shows RAW_VALUE = 8760h+00m+00.000s. The raw value is not always a bare integer — glance at the format before you compare numbers.

NVMe vs SATA: Two Different Worlds

If SATA S.M.A.R.T. is a messy attic — forty years of vendor-specific attribute IDs, proprietary normalization, and names that change between firmware revisions — then NVMe S.M.A.R.T. is the clean apartment somebody moved into after the divorce. The NVMe specification defines a mandatory, standardized health log that every compliant drive must implement identically. No attribute IDs, no vendor normalization, no guessing:

Critical Warning:                   0x00
Temperature:                        35 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    7%
Data Units Read:                    143,917,338
Data Units Written:                 94,231,007
Media and Data Integrity Errors:    0
Error Information Log Entries:      0

Every field means exactly what it says, on every drive, from every vendor. Critical Warning is a bitmask — 0x00 means nothing wrong; any bit set flags a specific condition (spare capacity low, temperature exceeded, gone read-only, NVM subsystem degraded). Media and Data Integrity Errors is the NVMe equivalent of the pre-fail trio: data that failed the drive's own integrity check. Non-zero is a real defect. Available Spare draining toward its threshold means the internal spare reserve is running low — see NVMe spare exhausted. And Percentage Used is the endurance counter: 7% means this drive has consumed 7% of its rated write life.

The practical upshot: NVMe drives are dramatically easier to monitor than SATA drives because the health data is standardized. There is no "attribute 5 means something different on Samsung vs Western Digital" problem. The cost is that you get fewer attributes — no pending sector count, no offline uncorrectable — because the NVMe spec chose clarity over granularity. For most server monitoring, that trade-off is a win.

How I Inspect It

Two commands cover everything. Run the first on any disk whose health you want to check; run the second periodically to exercise the drive's own internal diagnostics.

The Full Report

smartctl -a /dev/sda

This dumps the complete S.M.A.R.T. report: identity, overall health verdict, attribute table, error log, and self-test history. For NVMe drives, the device path is /dev/nvme0 (or /dev/nvme0n1 — check with lsblk). Behind a hardware RAID controller, you need to reach through the controller to the physical disk — smartctl supports this with a -d flag:

smartctl -a -d megaraid,0 /dev/sda    # LSI/Broadcom MegaRAID, disk slot 0
smartctl -a -d cciss,0 /dev/sda       # HP SmartArray

If neither works, your controller may not support S.M.A.R.T. passthrough — see SMART unavailable for the full troubleshooting path.

The Quick Health Check

smartctl -H /dev/sda

This returns just the one-line overall health verdict: PASSED or FAILED. Useful in scripts, but understand its limits: the verdict only trips when a "Pre-fail" attribute's normalized VALUE drops below THRESH. Manufacturer thresholds are deliberately conservative (they're calibrated to warranty periods, not your uptime requirements), so a drive can have dozens of reallocated sectors, a climbing pending count, and a growing error log — and still report PASSED. The verdict catches late-stage failure. The attribute table catches early-stage degradation. Read both.

Self-Tests

The drive can test itself — and it should, regularly. S.M.A.R.T. defines two flavours:

smartctl -t short /dev/sda    # ~2 minutes, quick surface scan
smartctl -t long /dev/sda     # hours, full surface scan

A short test takes about two minutes and checks a sample of the surface plus the drive's internal electronics. A long test reads every sector on the disk and can take hours on a large HDD — but it's the only way to find bad sectors hiding in rarely-read corners. The results go into the self-test log, which smartctl -a includes automatically:

Num  Test_Description  Status               Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline     Completed without error  00%     85590            -
# 2  Extended offline  Completed without error  00%     85422            -

A clean log shows Completed without error for every test. A test that found something shows the LBA (logical block address) of the first bad sector — which you can then map to a file with debugfs or hdparm --fibmap if you need to know what was lost.

Best practice: run smartctl -t short weekly via cron on every disk, and smartctl -t long monthly. The tests run in the background and don't affect performance noticeably. They're the difference between finding a bad sector in a quiet scan and finding it when a user's SELECT hits it.

Manufacturer Thresholds: Why They Lie

The THRESH column in the attribute table is set by the drive manufacturer, and it is set for their benefit, not yours. The threshold represents the point at which the manufacturer considers the attribute to have "failed" for warranty purposes. For Reallocated_Sector_Ct, a typical threshold is 5 (on a normalized scale where 100 is new). That means the drive has to lose an enormous number of sectors — far past the point where you'd want it anywhere near production data — before the manufacturer's own firmware considers it failed.

Waiting for the official threshold to trip is like waiting for the oil light to come on before you think about maintenance. By the time the light is on, the engine is already damaged. The useful signal is any non-zero raw count in the pre-fail attributes, and especially a rising count between readings. You don't need the manufacturer's permission to replace a drive you don't trust. The threshold is their warranty math; your threshold should be your operational risk tolerance — and for most servers, that's "non-zero and climbing means we schedule a replacement."

The Backblaze Evidence

Backblaze, the cloud storage company, has been publishing quarterly and annual drive failure statistics since 2013, drawn from a fleet that has grown to over 250,000 drives across multiple manufacturers. Their data is the closest thing the industry has to a large-scale, controlled, public dataset on which S.M.A.R.T. attributes actually predict failure — and the findings are remarkably consistent:

Attributes 5, 187, 197, and 198 are the strongest failure predictors, year after year. Drives with non-zero values in these attributes fail at dramatically higher rates than drives with clean counts.
No single attribute catches everything. Even combining all four, roughly 40-50% of failed drives showed no prior S.M.A.R.T. warning at all — they just stopped working. S.M.A.R.T. is a net that catches the slow deaths and misses the sudden ones.
Failure rates increase with age, but not linearly. There's a "bathtub curve": elevated infant mortality in the first year (manufacturing defects), a long stable period, then rising failure rates after year three or four. The Backblaze data confirmed what the industry had assumed but never proven at scale.
Manufacturer and model matter enormously. AFR ranges from under 0.5% for the best models to over 5% for the worst, within the same year of deployment. Picking the right drive model matters more than almost any other storage decision.

The takeaway is practical: monitor the four key attributes (#5, #187, #197, #198), act on non-zero values, but never rely on S.M.A.R.T. alone — because the drives that die without warning are the ones that hurt the most.

Gotchas

Things that catch people, from the most painful to the merely annoying:

The PASSED verdict is dangerously reassuring. A drive can have reallocated sectors, pending sectors, and errors in the log while still reporting PASSED, because the normalized values haven't crossed the manufacturer's threshold yet. Read the raw attribute counts, not the headline. The full walk-through is in failing disk — including two real drives where FAILED meant fine and PASSED meant dying.
SATA attributes are vendor-specific chaos. Attribute ID 5 means Reallocated_Sector_Ct on most drives, but some vendors reuse the same ID for different things. Samsung SSDs have their own attribute set. Seagate's Raw_Read_Error_Rate (ID 1) looks terrifyingly high on every drive — it's just how Seagate counts, and it's normal. Know your vendor, or stick to the handful of attributes (5, 187, 197, 198, 199) that are consistent across almost everyone.
You can't reset S.M.A.R.T. counters. The drive firmware owns them. If you replace a cable and CRC errors stop climbing, "fixed" means the count stops growing — it doesn't go back to zero. Deltas between readings are what you monitor, not absolute values.
Virtual disks have no S.M.A.R.T. Cloud VMs, VPS instances, and anything running on a hypervisor — the virtual disk your OS sees has no physical health data to report. This is expected, not a bug. See SMART unavailable.
Hardware RAID controllers hide physical disks. If smartctl -a /dev/sda fails or shows the RAID controller's virtual disk instead of the physical drive, you need the -d passthrough flag. Not all controllers support it — some actively refuse to expose S.M.A.R.T. data, which is one of the arguments for software RAID (the kernel's md driver) over hardware controllers: with software RAID, each physical disk is visible to the OS and to smartctl without any tricks.
Self-tests don't run automatically by default. The drive can test itself, but it won't unless you tell it to. Set up a weekly short test via cron or enable the smartd daemon. The sectors hiding in rarely-read corners of a multi-terabyte HDD won't announce themselves until something tries to read them — and you'd rather that something be a scheduled test than a production query.

History and Philosophy

S.M.A.R.T. began in the early 1990s as separate, incompatible monitoring schemes from individual drive makers — IBM's Predictive Failure Analysis (PFA), Compaq's IntelliSafe, and proprietary systems from Seagate and others. Each vendor tracked different internal metrics and made them available through different interfaces, which made standardization a pressing problem the moment anyone tried to monitor more than one brand. In 1995, Compaq, Seagate, Quantum, and Conner Peripherals proposed a unified specification under the name S.M.A.R.T. — and the industry adopted it, partly because the idea was good and partly because the alternative was writing a different monitoring integration for every drive manufacturer on the market.

The original promise was bold: the drive would predict its own failure before it happened. The reality turned out to be more nuanced. A landmark 2007 study by Google (Pinheiro, Weber, and Barroso, "Failure Trends in a Large Disk Drive Population") analysed over 100,000 drives and found that while S.M.A.R.T. attributes — especially scan errors and reallocated sectors — were statistically correlated with failure, 36% of failed drives had zero S.M.A.R.T. warnings beforehand. A parallel study by Carnegie Mellon reached similar conclusions. The technology was useful, but it was not the crystal ball the industry had hoped for.

The deeper reason is architectural. S.M.A.R.T. monitors the things a drive can observe about itself — surface quality, head positioning accuracy, controller temperature, error correction success rates. But catastrophic failures often originate in components the drive doesn't instrument: a power regulator that degrades, a motor bearing that seizes, a head that parks wrong and sticks. The drive's self-monitoring is limited to its own instrumented subsystems, and the failure modes that produce no warning are precisely the ones outside that instrumented perimeter.

The NVMe specification, arriving two decades later, learned from SATA's mistakes. Instead of leaving attribute definitions to individual vendors, NVMe standardized a mandatory health log with defined fields and defined semantics. Every NVMe drive reports the same set of health counters, in the same format, with the same meaning. It's a smaller set than SATA's vendor-specific sprawl — but every field is universally comparable, which is worth more than fifty fields you can't trust across brands. This is the direction storage monitoring is heading: fewer numbers, but honest ones.

The philosophy of S.M.A.R.T. — that hardware should monitor itself and report degradation before it becomes failure — has spread far beyond disks. ECC memory detects and corrects bit flips. CPU thermal throttling prevents damage before it happens. Network interface cards track CRC errors on their links. The principle is the same everywhere: instrument the component, watch the counters, act on the trend. S.M.A.R.T. was one of the first systems to take that principle seriously at the commodity hardware level, and for all its flaws — the vendor inconsistency, the misleading thresholds, the failures it can't see — it remains the foundation of every disk health monitoring system in existence. Including the one watching your servers right now.