Tailored Inference: Why One Engine Won't Be Fastest for Everything

Z.ai got 3.22 times the throughput for GLM-5.3-Flash in 13 days, with the same model on the same chips, by changing only the software that serves it.

I made a 6-minute video that explains how they did it, and why I think every team that serves models at scale should pay attention. The idea is called tailored inference: fitting the serving software to one model, on one kind of hardware, for the workload it will serve.

In the video, I cover:

• Why standard engines like vLLM and SGLang leave speed unused. They’re built to run almost anything, so they can’t take full advantage of any one combination.
• What the gains look like. DeepSeek’s DeepInfer runs one model and nearly halves the work of processing a prompt. vLLM got about 5 times the agent throughput on DeepSeek-V4.1-Flash after launch day, mostly from code for that one model. Databricks reports agent-written kernels 1.8 to 5.2 times faster than vLLM’s best.
• Why it’s happening now. AI agents do most of the tuning, so tailoring each new model is finally affordable.
• Where it goes wrong. Researchers at NVIDIA and UC Berkeley found that about a third of agent patches that passed unit tests failed in a real serving setup.

The question to ask isn’t which engine is fastest. It’s what’s fastest for this model, on this hardware, for this workload.

At Makora, we find that speed with agents that search for optimizations for each model and hardware combination, checked against representative workloads before they ship.