why is alpine linux slower? (a tiny musl vs glibc experiment)

Until recently, the exact physical mechanics of Alpine Linux's performance penalties were completely missing from my mental model. I’ve seen developers getting stuck on Alpine performance over and over again. Usually, it happens when their web apps mysteriously grind to a halt after being containerized. I previously assumed the main difference was just that Alpine uses musl libc instead of glibc, making the base image much smaller and thus better for containers. While the reduced image size is fantastic for storage, the massive 26% performance gap on heavy workloads remained a total mystery to me. Why are Python Docker builds getting way slower on Alpine?

Imagining a tiny C script is a great way to think about this. A hypothetical script simulating what a heavy runtime like Python does constantly—allocating memory and copying stuff around—reveals exactly what the CPU is dealing with.

#include <stdlib.h>
#include <string.h>

int main() {
    int size = 1024 * 1024; // 1MB
    char *src = malloc(size);
    char *dst = malloc(size);
    memset(src, 'A', size);

    for (int i = 0; i < 50000; i++) {
        memcpy(dst, src, size);
    }
    return 0;
}

This is super simple—just grabbing memory and copying a 1MB chunk fifty thousand times in a tight loop.

If you were to compile and run this on a standard Ubuntu box with glibc under perf (a magical Linux tool for seeing what the CPU is actually doing), the resulting numbers typically look something like this:

$ gcc -O2 test.c -o test_glibc
$ perf stat ./test_glibc

 Performance counter stats for './test_glibc':

          1,203.45 msec task-clock                #    0.998 CPUs utilized
             2,845      context-switches          #    2.364 K/sec
     4,510,230,192      instructions              #    1.52  insn per cycle

Okay, it takes about 1.2 seconds. But what is physically happening inside that memcpy call? If someone runs objdump -d to peek at the machine code inside glibc's memory copy implementation, it produces this dense block of vector instructions:

00000000000213b0 <__memcpy_avx_unaligned_erms>:
   213b0:  vmovdqu %ymm0,(%rdi)
   213b4:  vmovdqu %ymm1,0x20(%rdi)
   213b9:  vmovdqu %ymm2,0x40(%rdi)
   213be:  vmovdqu %ymm3,0x60(%rdi)
   213c3:  vzeroupper
   213c6:  ret

Let's translate these literal bytes! The vmovdqu command means Vector Move Double Quadword Unaligned, which physically tells the CPU to copy an outrageously massive chunk of data in a single instruction. The real magic happens inside those %ymm0 through %ymm3 AVX registers. A standard CPU register typically holds 64 bits of data. These special AVX registers hold a whopping 256 bits at once!

So glibc grabs 32 bytes of memory simultaneously per instruction, blasting them to the destination address in parallel using specialized vector hardware. The memory addresses on the right, like 0x20(%rdi), show it jumping forward 32 bytes at a time. The vzeroupper command acts as a cleanup crew, clearing out the upper bits of those giant registers so older non-vectorized instructions don't get confused. This block of code is pure, platform-specific wizardry designed exclusively for raw execution speed.

Now, if that exact same script were compiled and run inside an Alpine container (which uses musl instead of glibc), the math completely changes:

$ gcc -O2 test.c -o test_musl
$ perf stat ./test_musl

 Performance counter stats for './test_musl':

          2,845.12 msec task-clock                #    0.999 CPUs utilized
             1,210      context-switches          #    0.425 K/sec
    18,410,110,200      instructions              #    2.10  insn per cycle

Wait, what?! It takes 2.8 seconds—more than twice as long! And look at the instruction count: it jumps from 4.5 billion to 18.4 billion instructions! (how does that WORK??)

The assembly for musl's version of memcpy shows that the AVX vector instructions are completely gone. Musl relies on a standard C loop to copy memory forward word by word. Looking straight at the musl source code reveals a straightforward while loop handling the bytes.

This is a deliberate design choice! The maintainers of musl are trading blazing-fast, CPU-specific vector math for extreme portability and a tiny footprint. Alpine loves small static binaries, and the corresponding reality is that heavy data-copying operations require billions of extra CPU instructions to get the exact same amount of work done.

Memory copying is only half the battle. For a long time, musl's memory allocator (malloc) essentially used a giant global lock. You can visualize this like a single tollbooth on a massive highway. If a modern web server spins up dozens of threads, and they all try to allocate memory at the exact same time, they hit that global lock and have to wait in a single-file line. This lock contention absolutely decimated multi-threaded server performance (though thankfully newer releases use mallocng to fix this exact bottleneck!). It is amazing how one little synchronization primitive can bottleneck an entire web server architecture.

There is a LOT I still don't know about how SIMD assembly works, and very likely some of my assumptions about this objdump output are wrong! If you know more about how glibc magically detects CPU architecture at runtime to load the right vector instructions, please tell me :)