Metastability and the End of the Biological Rate Limit
Source control architectures built for human typing speeds are collapsing under unbounded AI inputs, requiring hard limits to prevent metastable failure.
By Owen Tate
Sparked by Ask HN: GitHub employees what's going on? Why? · discussion
There is a persistent narrative circulating about GitHub's recent infrastructure stumbles. If you read through the Hacker News discussion regarding their downtime, the consensus confidently points to the Azure migration and a presumed post-acquisition cultural decay. Commenters are highly eager to frame the intermittent outages as a symptom of enterprise bloat. I was reading through the thread yesterday, and it serves as a fascinating study in how we prefer social explanations for architectural limits. Organizational friction is undeniably real, and I certainly do not discount the drag of corporate integrations. But organizational friction doesn't rewrite the laws of physics. What we are actually watching is a brutal, unforgiving lesson in queueing theory.
For decades, the architecture of source control implicitly relied on a hidden, extremely robust safety valve. We modeled pull requests, continuous integration triggers, and commits as a Poisson arrival process strictly bounded by human physiology. A developer typing at 40 words per minute—who occasionally needs to sleep, argue in pull request comments, or walk away to drink coffee—acts as a natural physical governor on a system's arrival rate ($\lambda$). The entire surrounding ecosystem of webhooks and status checks was designed around this absolute ceiling. We built large-scale source control architectures assuming they would only ever need to process human-speed inputs. The database tables were tuned for humans. The load balancers were provisioned for humans.
I was running the math on this phase shift last week, and you can see the collapse coming if you model it as a comparative M/M/1 queue. In our biological era, the arrival rate remained comfortably below the database's service rate ($\mu$). The queue depth stayed relatively flat, easily absorbed by standard connection pooling. Now, swap out the coffee-drinking human for an unbounded AI coding agent. We are dealing with an ecosystem that processed 5.2 billion contributions in 2024, and that acceleration curve is steepening exponentially.
Let's simulate escalating the load. If we have a system where the AI arrival rate ($\lambda$) reaches 50% of the service rate ($\mu$), our average queue length is exactly 1. The system ticks along nicely. Double that arrival rate so we hit 90% capacity, and the math dictates our average queue length multiplies by nine. Push it just a fraction higher, to 99% capacity. The queue length rockets to 99. As $\lambda$ approaches $\mu$, the denominator in our queueing equation ($1 - \rho$) goes to zero. The queue length mathematically approaches infinity. Oof!
To make matters worse, a git commit rarely functions as an isolated event. Every single push triggers a sprawling fan-out of required state checks, CI/CD pipeline runs, branch protection rule evaluations, and Slack notifications. When an AI agent pushes a minor syntax correction, the pipeline spins up a fresh container, downloads the dependencies, runs the full test suite, and hits the API for status updates. A single commit generated in three seconds by an LLM spawns gigabytes of network traffic and minutes of compute time. The ratio of input effort to system load has fundamentally inverted.
A theoretical infinite queue length on a whiteboard tells only half the story. The operational reality involves what that queue does to a hypervisor in a messy production environment filled with network jitter, partial packet loss, and thermal noise. A version control backend operates as a highly coordinated, stateful ecosystem. Our architectures heavily rely on optimistic concurrency control (I’m assuming other clients are going to leave this repository alone while I write this commit). That optimistic assumption proves perfectly valid when $\lambda$ is governed by human typing speed.
When a fleet of AI agents spams concurrent, overlapping commits to the same monolithic repository, optimism violently breaks down. The database engine must abandon its fast path and fall back on pessimistic database locks. Pessimistic locks require cross-node coordination. In distributed systems, coordination is the absolute enemy of scale. The overhead generated by simply deciding who gets to write next begins to consume more CPU than the writes themselves.
This is exactly where the pristine math hits the muck of the implementation. When the queue depth blows up, the system categorically refuses to gracefully degrade into a slightly slower version of itself. As outlined in the classic USENIX paper Metastable Failures in the Wild, a distributed system under this kind of duress enters a self-sustaining death spiral. High AI arrival rates cause database lock contention to spike. That contention causes transactions to time out. The timeouts prompt the agent clients to automatically retry their requests. The retry storms dump even more load into the already infinite queue.
A mathematical asymptote on a chart suddenly manifests as a blaring alarm dragging an operator out of bed at 3 AM. The operator has absolutely no visibility into the cascading hypervisor locks. They stare at a wall of red graphs, trying to decipher whether they are dealing with a localized network partition or a global database meltdown. The on-call engineer—despite their best efforts—has been set up for failure by an architecture relying entirely on legacy human usage patterns.
Corporate reorganizations will never resolve a mathematical phase shift. Tweaking our prompt alignment to politely ask the AI to commit less frequently completely misses the structural vulnerability. To survive the removal of the biological rate limit, we must explicitly rate-limit the agent at the architectural edge. We have to enforce hard, physical constraints back onto the queue before the traffic ever reaches the data plane.[^1] Treating the LLM as a hostile, unbounded subsystem prevents these cascading lock failures entirely.
This leaves us with an uncomfortable but necessary heuristic. A system designed for biological constraints collapses under the weight of an unbounded generator. True resilience requires putting unconstrained coordination inside a mathematically deterministic box.
[^1]: There is always a temptation to try and out-scale the queue by dynamically provisioning more shards. Throwing horizontal scaling at a fundamental coordination problem provides a fantastic way to run out of money long before you run out of lock contention.