---
title: "Stop counting what you shipped. Start counting what survived."
url: "https://ctosync.com/insight/stop-counting-what-you-shipped-start-counting-what-survived/"
author: "Nick Sawinyh"
published: "2026-09-18"
updated: "2026-09-18"
---

# Stop counting what you shipped. Start counting what survived.

I spent most of last year building with AI in the loop, and somewhere around the fourth project I noticed my dashboards had stopped telling me anything.

Merged requests were up. Velocity was up. Cycle time looked healthy. And I had this nagging sense that the codebase was getting worse rather than better, with no number that agreed with me.

The problem turned out to be structural. Almost every engineering metric we use counts things that were created: commits, merges, story points, lines. All of them rest on a quiet assumption, which is that producing the artifact was the expensive part, so counting artifacts is a decent proxy for effort and value. That assumption held for about twenty years. It doesn't hold now. When generating a plausible change costs close to nothing, counting changes measures close to nothing.

The standard metrics also double-count failure. A change that ships on Monday and gets quietly repaired on Friday shows up twice as productivity, once when it merged and once when somebody cleaned it up. The team looks fastest exactly when it's doing the most rework.

So I stopped asking how much we produced and started asking a different question. Of everything we merged last month, how much is still standing?

## What the tool actually checks

I built something around that question. For each merged change inside a window, it checks three things. Was it reverted? Was it implicated by a later fix touching the same files? Did it churn? Every failure it counts prints an evidence chain you can click through and argue with, because a metric nobody can challenge is a metric nobody trusts.

Two things surprised me once it was running against real history.

The first was how much noise hides inside "a later fix touched the same file." Some paths get touched by everything. A README, a lockfile, a central config. If a path shows up in a quarter of all the change sets you are comparing against, it's not evidence of anything, it is just gravity. I added a guard that refuses to let those ubiquitous paths confirm a link. That one rule dropped my counted failures from 19 to 13. Nine of them had been README collisions.

The second was batch closes. One fix in my sample claimed to implicate sixteen separate earlier changes. No real fix does that. That is somebody clearing a backlog in a single commit, and the tool was reading a housekeeping message as sixteen independent failures. Capping the fan-out per fix removed all sixteen.

Both corrections made the number smaller. That matters more than it sounds. A verification tool that only ever finds more problems is a tool fitting itself to its own thesis. If your checker cannot talk itself out of a finding, it's not checking anything.

## How I would suggest using it

Don't use this to compare teams. The number is meaningless across different codebases, review cultures, and repository shapes, and the first person to turn it into a leaderboard will ruin it for everyone.

Use it inside one team, with the configuration frozen, as a trend. And use it as a tripwire. Change the model, change the review process, raise the autonomy level, then watch whether survival moves. That's the moment the number earns its keep.

The reason I trust it as a measure is that you can't game it by working harder in the usual direction. Shipping more only helps if the extra work survives, which is the behavior you wanted in the first place.

None of this is an argument against AI-assisted development. I use it every day, and I'm faster for it. But the speed of generation moved the bottleneck, and most teams haven't moved their attention to match. It used to be hard to write code and easy to trust it. Now it's easy to write and expensive to trust. If you have adopted the first half of that trade and not the second, the gap does not announce itself. It shows up later as work you thought you had already finished.

---

**Nick Sawinyh** is a product and engineering leader who has spent over a decade taking technically complex products to market across DeFi, AI tooling, and government technology. He co-founded DexGuru, a DEX analytics platform that reached $1B+ in cumulative volume, and now works on data infrastructure and open-source developer tools. He writes at [sawinyh.com](http://sawinyh.com) and builds in the open at [github.com/sneg55](https://github.com/sneg55).