My AI agents were eating my SSD

How I found out

I was shopping for a cheap mini PC to run tests around the clock, so I checked the health of the drive I already had. The answer was worse than I expected.

I'm building Renor, a memory layer for coding agents. On a normal day I run about 10 agents on one laptop, and up to 22 at the peak. They write code, review each other's PRs and run the test suite before every merge.

The SSD is a 512 GB NVMe drive. Its own health report said:

Then I measured the write rate: about 500 GB a day. At that pace the remaining ~24 TB lasts about seven weeks.

Nothing had warned me. The drive doesn't complain, the operating system doesn't complain, and none of my agents knew the disk was finite. My first reaction was: why did nobody tell me?

Where the writes came from

It took a day of measuring, per process and per cgroup, to find the sources. None of them was a bug. Each one was a reasonable default that multiplies badly when ten agents do it at once.

Most of the testing ran on the laptop for two reasons: the merge gate runs locally, and my private-repo GitHub Actions minutes had run out for the month.

The last piece surprised me most. About half of all writes belonged to no process at all. That was the filesystem's own overhead, plus thousands of short-lived commands (git, node, shells) that agents start and finish faster than any monitor can count them.

Why agents make this worse

A human developer runs the test suite a few times a day. An agent fleet runs it every time any agent wants to merge, and nobody is watching the meter.

Four things stood out.

1. Most of the data is throwaway. Scratch files, temporary databases and dependency copies live for minutes and then get deleted. They never needed to touch the SSD.
2. Nobody can see it. Linux counts writes per process, but not per agent or per task. It took my agents and me a full day to work out who was writing what.
3. There is no budget. Nothing knew the drive had about 24 TB left, so nothing slowed down. To each agent, 72 full test runs in a day looked normal.
4. The filesystem multiplies small writes. btrfs never overwrites data in place. Every small change inside a big file (a database, a browser profile, a log) also rewrites a piece of the filesystem's index. Agent tools are full of exactly those files.

What I changed

I didn't buy anything. Every fix was a setting, a rule for the agents, or moving work somewhere free.

Results

The first night after the fixes, the laptop wrote 61 GB a day. Over the next day, which included a launch, it averaged 90 GB, with a quiet-hours floor of 51 to 55 GB. Test runs are no longer the main writer. The browser, the agent tools and filesystem overhead are.

Rated life is not a cliff, and many drives keep working well past 100%. The integrity-error count hasn't moved since I started watching. But a drive with that history can still fail without warning, so the real protection is a backup, not a number.

Part two: the agents left, the writes didn't

Moving most of the agents to a server cut the laptop's load from about 20 to about 2. Writes only fell from 100–140 GB a day to about 90. So the agents weren't the whole story.

The per-process counters I had been using (/proc/<pid>/io) were hiding most of it. They can't read processes owned by root, they miss commands that finish within a second, and they never see the filesystem's own writes. The kernel's per-cgroup counters (io.stat under /sys/fs/cgroup) see all three.

Three things turned up:

1. The drive had never been trimmed. dm-crypt, the Linux disk encryption layer, drops TRIM requests unless you allow them, and the weekly trim timer was off. The SSD never learned which blocks were free, so it kept moving dead data around internally. One command fixed it (cryptsetup --allow-discards --persistent refresh), and the first fstrim freed 274 GiB. TRIM doesn't lower the GB written; it lowers how much flash wear each GB costs.
2. A voice-typing app had left a recording running. It saved microphone audio as a new small file every 0.3 seconds. I suspect that pattern fed a lot of the filesystem's bookkeeping, but I haven't proven it.
3. A broken service was crash-looping. It restarted every few seconds because a model file was missing, and its errors added about 6 GB a day to the system log. Stopping it took the log down to under 1 GB a day.

I also switched the mounts from relatime to noatime, so reading a file no longer updates its access time on disk.

Where it stands: the last full day was 90 GB. That's still more than twice my 40 GB target, and leaves about 9 months of rated life at this pace. It's too early for TRIM to show in the wear percentage. Next I'll count which programs force the disk to sync (fsync) and move the worst ones to RAM.

If you run agents on your own machine

Most of this takes about ten minutes. Start with these five commands:

# wear and total written
$ sudo smartctl -a /dev/nvme0 | grep -E 'Percentage Used|Data Units Written'
# GB written since boot
$ awk '{print $7*512/1e9, "GB"}' /sys/block/nvme0n1/stat
# 0 means TRIM never reaches the encrypted drive
$ cat /sys/block/dm-0/queue/discard_max_bytes
$ systemctl is-enabled fstrim.timer
# how much the system log holds
$ journalctl --disk-usage

Then work down the list:

1. Read your drive's health. "Percentage Used" and "Data Units Written" from smartctl. Note them today and compare next week.
2. Measure a day of writes. The 7th field of /sys/block/nvme0n1/stat is sectors written. Read it twice, 24 hours apart, and multiply the difference by 512 bytes.
3. Count by cgroup, not by process. io.stat under /sys/fs/cgroup includes root processes, short-lived commands and the filesystem's own writes. Per-process counters miss all three.
4. Check that TRIM reaches the drive. On an encrypted disk, discard_max_bytes must not be 0, and fstrim.timer should be enabled.
5. Put scratch in RAM. Point TMPDIR for test runs at a tmpfs such as /dev/shm.
6. Share dependencies across worktrees. Use reflinks, hardlinks or a content-addressed store like pnpm's.
7. Stop duplicate runs. If two agents want the same test run on the same commit, run it once and share the result.
8. Use free CI for anything public. Public repos get GitHub-hosted runners at no cost.
9. Look at your browser. A long-lived profile with web chat apps open can write tens of GB a day.
10. Check your logs and background apps. A service stuck restarting, or an app recording in the background, can write gigabytes a day.

What's next: a write budget for agent fleets

Every fix above was manual. As agent fleets grow, I think this needs to be automatic, the way a cloud bill has a budget.

The tool I want to build next has three parts:

• A daily budget from the drive's real health. Remaining endurance divided by how long you want the drive to last. For my drive, about 24 TB over two years is roughly 33 GB a day.
• Writes counted per agent and per task. Each agent or test run gets its own cgroup, so the kernel's own counters say exactly who wrote what.
• Placement by how long data lives. Scratch that dies in minutes goes to RAM. Only data that has to survive goes to the SSD. When the day's budget runs low, heavy jobs wait or move to CI instead of grinding the disk.

Storage hardware already places data by expected lifetime internally. Doing it at the level of an agent fleet, with a budget tied to the drive's health, is the part I haven't seen yet. If you have, or if your agents have done this to your disk too, I'd like to hear about it.