Backend
Don't put getaddrinfo on your proxy's hot path
Efrain Garay DEV Community
1 views
I set out to benchmark Pingora 0.9.0 — the Rust proxy library Cloudflare runs at its edge — against nginx, in the environment I actually care about: a small pod with a CPU limit, two vCPUs, everything in containers so no number leans on the host.
I wrote the simplest possible reverse proxy with Pingora, hit it, and got 21k requests/sec against nginx's 126k in the same setup. Six times slower.
That number was a lie, and the fault was mine.
Six times is too much — Cloudflare would not replace nginx with something 6x slower. So I isolated it. Not CPU throttling (nr_throttled was zero). Not thread oversubscription (exactly two workers). Not connection reuse (both held ~130 upstream connections). The clue was latency: 4.78 ms in the container, 0.45 ms on the host with the backend at 127.0.0.1. The only thing that changed was how I named the backend.
My upstream_peer built the destination on every request:
// (&str, u16) → to_socket_addrs() → getaddrinfo, blocking, on the tokio worker
let peer = HttpPeer::new(("backend", 80), false, String::new());
HttpPeer::new over a (&str, u16) calls getaddrinfo synchronously, inside the tokio worker thread. In a container, every request fired a DNS query to Docker's resolver and blocked the thread. On the host, an IP literal is just a parse — no syscall — so it barely showed.
The fix is one line: resolve once at startup, keep the SocketAddr, pass that on the hot path.
let addr = ("backend", 80).to_socket_addrs()?.next().unwrap();
// ...on every request, no lookup:
Ok(Box::new(HttpPeer::new(self.addr, false, String::new())))
Pingora went from 21k to 89k req/s in the same pod. The 6x was a getaddrinfo per request, not the framework.
The lesson isn't about Pingora. A microbenchmark punishes any clumsiness of whoever writes it, and a blocking name resolution hidden on the hot path disguises itself perfectly as "the framework is slow."
With the proxy written properly, the real gap in a 2 vCPU pod is 1.49x — and perf pins it down to the cycle: Pingora runs 1.7x more instructions per request. I also found Pingora's thread default is 1 (a whole idle core if you don't set it), and that in a CPU-limited pod, more threads than cores wrecks the p99 through CFS throttling.
I documented the whole thing — the container setup, the full concurrency matrix with repetitions, the perf/strace profile, the flamegraph, and the thread-throttling numbers — with live diagrams and a reel, here:
👉 The full write-up on efraingaray.com
If you benchmark a Rust proxy in a container: resolve upstreams by IP or once, and set threads equal to your CPU quota. Those two lines matter more than the framework you pick.
Read original: https://dev.to/efraingaray/dont-put-getaddrinfo-on-your-proxys-hot-path-i31
← Previous
ECC: An Open-Source Performance Harness and Operating System for AI Coding Agents
Next →
Vercel Sandbox now provides 64 GB of storage
Related
Protocol Divergence Localization: Finding WHERE Firewalls Block Your Traffic
Backend
2
Dev.to (EN Zone)
Dockerizing ROS 2 and AI Robotics Applications for NVIDIA Jetson
Backend
2
Dev.to (EN Zone)
Building a Vision-Language Robot with Jetson + ROS 2
Backend
2
Dev.to (EN Zone)
Edge LLMs on NVIDIA Jetson: Building an AI Assistant for Robots
Backend
2
Dev.to (EN Zone)
Comments0
No comments yet — be the first