mdadm Plugin

Monitor Linux Software RAID arrays managed by mdadm, tracking array state, degraded or failed devices, and rebuild/resync progress.

Overview

The mdadm plugin reads the kernel's RAID status directly from /proc/mdstat and reports the health of every active array on the host:

  • Array State - active, degraded, recovering, resyncing, reshaping, inactive, or failed
  • Device Counts - total, active, working, failed, and spare devices per array
  • Degraded Detection - flags any array missing one or more devices
  • Sync Progress - recovery / resync / reshape / check / repair action and completion percentage
  • Array Size - usable array size in GB
  • Summary Counters - total arrays, degraded arrays, and failed arrays across the host

The plugin is read-only. It never issues write commands, assembles, or modifies arrays.

Requirements

RAID Support

  • Linux only - the plugin relies on the kernel md driver and /proc/mdstat
  • At least one assembled software RAID array (md0, md1, …)

Dependencies

  • No Python packages required. The plugin uses only the standard library.
  • mdadm should be installed on the host (the standard tool for creating and managing the arrays). The md driver exposes status through /proc/mdstat regardless, which is the plugin's primary data source.

Verify RAID is Present

# Primary data source β€” must exist and list at least one array
cat /proc/mdstat

# Confirm mdadm is installed
mdadm --version

# Detailed view of a specific array
mdadm --detail /dev/md0

Example /proc/mdstat with a healthy mirror and a degraded RAID 5:

Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid1 sdb1[1] sda1[0]
      104320 blocks super 1.2 [2/2] [UU]

md1 : active raid5 sdd1[3] sdc1[1] sdb1[2] sda1[0]
      3145728 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/3] [UU_U]
      [>....................]  recovery =  1.2% (12345/104320) finish=0.5min speed=12345K/sec

unused devices: <none>

If /proc/mdstat does not exist, or lists no active/inactive md devices, the plugin reports itself as unavailable and collects nothing.

Configuration

The agent reads its configuration from the shell environment file at /opt/statusradar/config.env, which is loaded by systemd (EnvironmentFile=). It is not a YAML file.

Enabling the Plugin

mdadm requires no plugin-specific environment variables β€” just add mdadm to the PLUGINS list.

Edit /opt/statusradar/config.env:

PLUGINS=mdadm

To run alongside other plugins, use a comma-separated list:

PLUGINS=mdadm,nginx,docker

Restart the agent to apply:

sudo systemctl restart statusradar-agent

There are no host, port, credential, or path options to set. The plugin always reads /proc/mdstat.

Collected Metrics

The plugin returns host-level summary counters plus a per-array breakdown.

Host Summary

Metric Description Unit Type
total_arrays Number of arrays found in /proc/mdstat Count Gauge
degraded_arrays Arrays missing one or more devices Count Gauge
failed_arrays Arrays in failed or inactive state Count Gauge

Per-Array Metrics

Each array under arrays[] is keyed by its device (e.g. /dev/md0) and includes:

Field Description Unit Type
device Array device path (/dev/mdN) - Label
level RAID level (raid0, raid1, raid5, raid6, raid10, linear, multipath) - Label
state Array state (see below) - Label
total_devices Member devices listed for the array Count Gauge
active_devices Devices the array expects (from [N/M]) Count Gauge
working_devices Devices currently up (from [N/M]) Count Gauge
failed_devices Devices marked failed (F) Count Gauge
spare_devices Devices marked spare (S) Count Gauge
size_gb Usable array size Gigabytes Gauge
sync_action Current background action: idle, recovery, resync, reshape, check, repair - Label
sync_percent Completion of the current sync action (100.0 when idle) Percent Gauge
degraded Whether the array is degraded Boolean Gauge
state_numeric Numeric health for alerting (see below) - Gauge

State Values

The state field is derived from the kernel status word and the device-status string ([UU], [U_], etc.):

  • active / clean - all devices present and healthy
  • degraded - one or more devices missing (_ in the device map, or working < expected)
  • recovering - a recovery is in progress (e.g. rebuilding onto a replacement disk)
  • resyncing - a resync is in progress
  • reshaping - a reshape is in progress (e.g. growing the array)
  • inactive / failed - the array is not operational

Numeric State for Alerting

state_numeric makes it easy to alert on RAID health in VictoriaMetrics:

Value Meaning States
1.0 Healthy active, clean
0.5 Warning degraded, recovering, resyncing
0.0 Critical failed, inactive (and any other state)

Dashboard Metrics

The StatusRadar dashboard surfaces:

Overview Card

  • Total Arrays - total_arrays
  • Degraded Arrays - degraded_arrays
  • Failed Arrays - failed_arrays

Per-Array Detail

  • State - color-coded from state_numeric (green / amber / red)
  • RAID Level - level
  • Devices - working_devices / active_devices with failed_devices and spare_devices
  • Size - size_gb
  • Sync Progress - sync_action and sync_percent while a rebuild or check is running

How Detection Works

  1. The plugin checks that /proc/mdstat exists and contains at least one md device in an active or inactive state. If not, it is unavailable.
  2. It parses /proc/mdstat line by line:
    • Array header lines (md0 : active raid1 sdb1[1] sda1[0]) yield the device, state, level, and member device list. Devices flagged (F) are counted as failed, (S) as spare.
    • The status line (104320 blocks super 1.2 [2/2] [UU]) yields the array size, the working/active device counts from [N/M], and the up/down map from [UU] / [U_]. A _ in the map, or working < expected, marks the array degraded.
    • Progress lines (recovery = 1.2% ...) set sync_action and sync_percent and update the state to recovering, resyncing, or reshaping.
  3. mdadm --detail /dev/mdX is available as a fallback for richer per-array detail, but the fast /proc/mdstat path is the primary source.

Testing

Manual Plugin Test

Run the plugin directly to see what it would report:

cd /opt/statusradar
export PLUGINS=mdadm
python3 plugins/mdadm_plugin.py

Expected output (host with two arrays):

Plugin name: mdadm
Enabled: True
Available: True

Metrics collected:
  Total arrays: 2
  Degraded arrays: 1
  Failed arrays: 0

Array details:
  /dev/md0:
    Level: raid1
    State: active
    Devices: 2/2 (failed: 0)
    Size: 0.1 GB
    Sync: idle (100.0%)
    Degraded: False
  /dev/md1:
    Level: raid5
    State: recovering
    Devices: 3/4 (failed: 0)
    Size: 3.0 GB
    Sync: recovery (1.2%)
    Degraded: True

Host with no RAID arrays:

Plugin name: mdadm
Enabled: True
Available: False

mdadm not available or no RAID arrays found
To enable: export PLUGINS=mdadm
To test: cat /proc/mdstat

Verify in Agent Logs

sudo grep mdadm /var/log/statusradar/agent.log | tail -n 20

Troubleshooting

Plugin Reports "Available: False"

Cause: No RAID arrays are assembled, or /proc/mdstat lists no active/inactive md devices.

Check:

# Does the file exist and list arrays?
cat /proc/mdstat

# Is the md driver loaded?
lsmod | grep -E 'md_mod|raid'

# Are arrays assembled?
mdadm --detail --scan

If /proc/mdstat shows only Personalities : ... and unused devices: <none>, there are no arrays to monitor β€” this is expected on a host without software RAID.

No Metrics Collected

Check 1: Is the plugin enabled?

grep '^PLUGINS' /opt/statusradar/config.env

PLUGINS must include mdadm.

Check 2: Did the agent restart after the config change?

sudo systemctl restart statusradar-agent
sudo systemctl status statusradar-agent

Check 3: Run the plugin manually

cd /opt/statusradar && export PLUGINS=mdadm && python3 plugins/mdadm_plugin.py

Array Always Shows Degraded

Symptom: degraded: True persists.

This is the plugin correctly reporting a real problem β€” one or more member devices are missing or failed.

# Identify the failed/missing device
cat /proc/mdstat
mdadm --detail /dev/md0

Look for a _ in the device map (e.g. [U_]) or a (F) flag next to a member. Replace the disk and add it back with mdadm --manage /dev/md0 --add /dev/sdX1; the array will move to recovering and sync_percent will climb toward 100%.

Sync Progress Stuck

Symptom: sync_percent is not advancing.

# Current rebuild speed limits
cat /proc/sys/dev/raid/speed_limit_min
cat /proc/sys/dev/raid/speed_limit_max

# Watch live progress
watch -n5 cat /proc/mdstat

A slow rebuild under heavy I/O is normal; raise speed_limit_min only if the array can tolerate the extra load.

Size Reported as 0.0 GB

Very small arrays (a few MB, such as boot partitions) round down to 0.0 GB because size is reported to two decimal places. Confirm the raw block count in /proc/mdstat if exact size matters.

Performance Impact

The plugin reads a single small file (/proc/mdstat) per collection interval and parses it with regular expressions.

  • CPU: negligible (sub-millisecond parse)
  • Memory: negligible
  • I/O: one read of an in-memory kernel file; no disk access to the arrays
  • No impact on RAID performance or rebuild speed

Use Cases

1. Detect Disk Failures Early

Alert the moment a member device drops out, before the array loses redundancy entirely.

Alert on:

  • degraded_arrays > 0
  • failed_devices > 0 on any array
  • state_numeric <= 0.5

2. Track Rebuild Progress

Watch a replacement disk resync after a failure and confirm the array returns to healthy.

Monitor:

  • sync_action and sync_percent
  • state_numeric returning to 1.0

3. Catch Failed Arrays

Page immediately if an array becomes inoperable.

Alert on:

  • failed_arrays > 0
  • state_numeric == 0.0

Next Steps