Tensor LabsTENSORLABS

The outage you'll pay for is in code you didn't write

Reachability plus maintainer-health scoring turns four hundred alerts into the ten packages that can hurt you

August 3, 20264 min read4 sectionsBy Ahmed Abdullah
The outage you'll pay for is in code you didn't write

Introduction

The war room had been running for five hours when someone finally read the changelog out loud. We were on a reliability engagement with a platform team, invited after a quarter of ugly incidents, and this one was the classic of the genre: checkout intermittently failing, no deploy that day, no infra change, dashboards green except the one that mattered. The culprit, eventually, was a transitive dependency, a library nobody on the team had chosen, pulled in by a library they had chosen, which had shipped a patch release that changed connection-pool behaviour under load. The version bump had ridden in through an automated update three days earlier, reviewed by no one, because patch releases are "safe."

The team owned roughly 900 direct and transitive dependencies. Asked which ten of those could take checkout down, nobody could answer. That is not a criticism of the team. It is a description of every team.

Your reliability is a portfolio of other people's decisions, and almost no engineering organization manages that portfolio. It just receives the returns.

Why the scanner list is not the answer

The standard tool here is a vulnerability scanner, and its output is the most confidently ignored artifact in software: four hundred CVEs, sorted by a severity score that knows nothing about your system. The score cannot see that the "critical" in a PDF library is in code your service never calls, while the "low" in a connection pool sits directly on the path that earns the company money. So the list gets triaged by vibes, then muted, and the actual risk, which was never really about CVEs, ships itself in a patch release on a Tuesday.

Dependency risk is only fractionally about known vulnerabilities. The larger share is operational: how central is this package to your critical paths, how actively is it maintained, how concentrated is its maintainership, how often do its releases change behaviour, how deep is it buried in the transitive tree where no human reviews its arrival. Each of those is measurable. Almost nowhere are they measured together.

(The library that took checkout down had one maintainer, who had announced a hiatus in a repository discussion eight months earlier. The information was public. It was just not anyone's job to know it.)

Scoring the portfolio like a portfolio

The build we did afterward treats the dependency graph as a risk model, not a list. Three ingredient families, joined per package. Reachability: static call-graph analysis maps which dependencies actually sit on your critical execution paths, which instantly reclassifies most of the scanner backlog, the majority of "critical" findings turn out to live in code that cannot run, while a handful of quiet packages reveal themselves as load-bearing. Health: maintainer count and concentration, release cadence and its variance, issue-response decay, the actuarial signals of a project going dark, scored from public data continuously rather than discovered in a war room. Blast radius: for each load-bearing package, what breaks if it misbehaves, derived from the same call graph plus your service topology.

The joined score produces a short list with a property the CVE list never had: it is worth reading. Ten packages, ranked by expected pain, each with a reason in plain language and a recommended posture, pin and monitor, wrap behind an interface, fund the upgrade, or replace while it is calm.

Four hundred alerts ranked by global severity is noise. Ten packages ranked by whether your checkout path depends on a hobby project is a work plan.

We built the reachability-plus-health scoring for that team and re-ran their scanner backlog through it: 391 of 407 findings demoted to unreachable, 16 confirmed real, and 3 packages flagged for operational risk that no scanner had ever mentioned, including the pool library that had caused the five-hour war room. The automated-update policy changed the same week: patch releases still flow, but load-bearing packages get a one-day soak with behavioural diffing instead of a straight ride to production.

The question to ask your own tree

You likely know your cloud spend to the dollar and your dependency risk not at all. The first artifact is cheap: the reachability pass over your main service, which typically collapses the scanner backlog by an order of magnitude and names the packages that actually deserve engineering attention. At TensorLabs we run that pass and the health scoring as a bounded exercise on your repositories, read-only, no agents installed.

Reply with your primary language and roughly how many services you run, and you will get back what reachability passes typically uncover at that scale, plus the three public-signal checks you can run on your own tree this week without us. If your backlog survives the pass intact, your scanner was right all along, and that is worth knowing too.