Real-Time is Not Fast Time

A recent Async Rust vs RTOS showdown benchmark made the rounds on Hacker News, sparking a massive wave of victory laps. The researchers demonstrated that a cooperative asynchronous loop running on embedded hardware could process tasks faster than a traditional Real-Time Operating System. The subsequent Hacker News comments were filled with web developers pointing out that cooperative async loops produce objectively lower average latency than RTOS schedulers. The dominant web dev dogma was vindicated: If the async loop processes tasks in 1ms on average, it's objectively better than the RTOS taking 5ms. Mathematically, they are completely right about the numbers. But if you put this code in a pacemaker, the patient dies.

Before we go further, I want to establish some strict boundaries on my own knowledge here. I’ve never built a pacemaker, and almost certainly don't understand the exact FDA compliance rules. But the combinatorics are universal. To understand why optimizing for blazing-fast statistical averages actually creates systemic risk in hardware, let's look at a toy model of a medical device. Imagine a critical system that must process a sensor pulse exactly every 10 milliseconds. Missing a single deadline by a fraction of a millisecond is a catastrophic failure.

We can simulate this dynamic using an environment most web developers already know. Let's build a tight Python asyncio event loop to process 10,000 simulated heart pulses.[^1] Because we want massive statistical throughput, we are going to rely on cooperative multitasking. We will run our vital pulse reader concurrently with a background network synchronization task, yielding control back to the event loop as fast as possible to maximize our CPU utilization.

import asyncio
import time
import random

async def background_sync():
    while True:
        await asyncio.sleep(0.005)
        # A rare, unexpected tail-spike
        if random.random() < 0.0001:
            time.sleep(0.015)

async def read_pulses():
    latencies = []
    for pulse_id in range(1, 10001):
        start = time.perf_counter()
        await asyncio.sleep(0.001) 
        
        latency = (time.perf_counter() - start) * 1000
        latencies.append(latency)
        
        if latency > 10.0:
            avg = sum(latencies) / len(latencies)
            print(f"Average execution: {avg:.1f}ms. Patient died at pulse {pulse_id}.")
            raise SystemExit

async def main():
    asyncio.create_task(background_sync())
    await read_pulses()

asyncio.run(main())

If we execute this script, our event loop processes the data with incredible efficiency! Right up until we hit a minor anomaly:

Average execution: 1.2ms. Patient died at pulse 4211.

Combinatorics are undefeated.

The average execution time of our system right up until pulse 4,210 was barely over 1 millisecond. The statistical throughput was astonishingly good. However, cooperative multitasking systems suffer from a glaring structural vulnerability: if a single concurrent task runs slightly too long, it will block the execution of all other asynchronous tasks. Because an async loop relies entirely on the honor system—where tasks must voluntarily yield control to let other work happen—a stray 15-millisecond tail spike in a low-priority background sync completely halts our critical 10-millisecond deadline.

This particular confusion between speed and reliability is a very old, recursively enumerable internet argument. While digging through academic archives recently, I found John Stankovic warning about this exact dynamic in 1988. In Misconceptions about real-time computing, he cautioned early software engineers that real-time computing is not equivalent to fast computing.

In embedded programming, average latency is a vanity metric. If a processor is generally fast but occasionally stalls to collect garbage or manage a network buffer, it is functionally useless for safety-critical hardware. The only number that actually matters is the <dfn>Worst-Case Execution Time (WCET)</dfn>, which explicitly defines the maximum length of time the task could take. Figuring out the WCET for a modern microchip with complex branching logic and memory caches is often obnoxiously hard, requiring a level of formal verification that looks like moon logic to a web developer, but it is the absolute foundation of real-time engineering.

To guarantee our medical device actually works, we have to abandon the cooperative async loop entirely. Instead, embedded developers use an RTOS with a hyper-rigid preemptive scheduler.

Preemptive systems do not politely ask tasks to yield. When a high-priority hardware interrupt fires at the 10-millisecond mark, the scheduler violently suspends whatever background task is running, stores the memory state, and forces the CPU to read the pulse. Let's rewrite the toy model to use hyper-rigid, preemptive blocking. We'll swap asyncio for standard OS threads to simulate our preemptive scheduler, and introduce an artificial 5-millisecond sleep to represent the massive context-switching overhead of a legacy RTOS.[^2]

import threading
import time
import random

def background_sync():
    while True:
        time.sleep(0.005)
        # The exact same 15ms tail-spike
        if random.random() < 0.0001:
            time.sleep(0.015)

def read_pulses():
    latencies = []
    for pulse_id in range(1, 10001):
        start = time.perf_counter()
        # Simulated rigid context-switching overhead
        time.sleep(0.005) 
        
        latency = (time.perf_counter() - start) * 1000
        latencies.append(latency)
        
        if latency > 10.0:
            print(f"Patient died at pulse {pulse_id}.")
            return
            
    avg = sum(latencies) / len(latencies)
    worst = max(latencies)
    print(f"Survived 10,000 pulses! Avg: {avg:.1f}ms, Worst: {worst:.1f}ms")

# OS-level threads will preempt each other
sync_thread = threading.Thread(target=background_sync, daemon=True)
sync_thread.start()
read_pulses()

Unsurprisingly, if we run this architecture, we get a totally different result:

Survived 10,000 pulses! Avg: 5.2ms, Worst: 6.8ms

We completely destroyed our beautiful 1.2-millisecond average! A modern backend developer looking at that benchmark graph would laugh at the massive, seemingly unnecessary loss of CPU efficiency. But the variance on the pulse reader practically shrinks to zero. Even when the background thread stalls for 15 milliseconds, the OS violently interrupts it to ensure read_pulses runs on time.

The device meets 100 percent of its deadlines, forever.

When high-level developers look at low-level code, it is easy to assume the legacy approaches are just primitive. But the continued dominance of RTOS isn't developers being lazy or ignoring modern paradigms. Web tools are exquisitely designed for web problems. An async loop servicing ten thousand network connections prioritizes total statistical throughput because a dropped request is just an annoyed user hitting refresh. We intentionally sacrifice massive CPU efficiency and throughput in embedded domains to buy an unbreakable deterministic guarantee.

It all comes down to the math we choose to believe in. Web developers live in a world of statistical throughput, where a 99th-percentile drop is just an irritated user hitting refresh. Embedded engineers live in a world of absolute combinatorial bounds, where the 100th percentile is the only thing keeping the plane in the sky.

[^1]: (Yes, I know you don't write embedded async loops in CPython, and the Rust embassy runtime is vastly more sophisticated than this script. That’s not the point: we are just looking at the underlying math of cooperative multitasking.) [^2]: (Yes, I know standard OS threads are not a true Real-Time Operating System scheduler. We are just using standard thread preemption as a mathematical stand-in to demonstrate how priority interruption solves the stall.)