QDA: Closed-Loop LLM Quantization Design with Agents
TL;DR
QDA brings a coding agent into LLM quantization design. For each model and precision, the agent designs the post-training quantization recipe on its own, with no human in the loop. Its checkpoints beat the strongest public recipe on four reasoning models, by 0.9, 1.4 and 2.2 points on average at W4A16, W4A8 and W3A16, and run on existing kernels with no inference overhead.
The agent works the way a quantization expert does: it proposes a recipe, builds it, measures it and decides what to change next. Two things make this loop fast enough for up to 36 candidates in one eight-hour search: a KL score that takes minutes instead of dozens of GPU-hours, and a library of 21 PTQ methods that stack in any order. We find that no method wins everywhere: the best recipe differs for every model and precision, and the agent finds it by searching.
On the 320B-parameter GLM-5.3-Flash at NVFP4, one 6.5-hour, $69 search recovers 98.3% of the FP8 model's accuracy, against 96.9% for NVIDIA's release. The tools matter too: given NVIDIA ModelOpt's tools instead of ours, the same agent falls 0.5–1.3 points behind on Qwen3.6-35B. A search costs $19–75 of agent tokens.
1An old question
In 2024, Yujun Lin, Xingyang's wonderful mentor at MIT HAN Lab, wrote DeepCompressor, the quantization toolbox behind QServe and SVDQuant. He once told us that the results sometimes puzzled him: a new method that looks strong in its paper often adds little once it is stacked on a few basics, such as AWQ's smoothing, a Hadamard rotation and GPTQ. Moreover, nobody knew which methods work well together, because each is tested on its own.
Answering that by hand means trying combinations model by model, which no one has time for. Today, however, LLMs can act as optimizers[1, 2], and agent harnesses such as Humanize[3] keep a coding agent working for hours with no one watching. So QDA lets an agent run exactly these experiments, for every model and precision.
The question is real. Across four reasoning models and three precisions, no public recipe is the most accurate everywhere (Figure 1). The newest method wins three of the four models at W3A16, yet plain GPTQ is still the best on R1-Distill-Llama-8B at every precision.
Stacked well, however, known methods go far. On Qwen3.5-9B at W4A8, GPTQ alone sits 17 points below BF16. A Hadamard rotation wins back 9.9 points, and the agent's recipe wins back another 5.2 with self-generated data, an FP8-aware scale search and scale distillation. None of these steps is new; the gain comes from choosing them for this model.
2Why the loop is slow
An agent designs a recipe the way an expert does: propose a recipe, build the checkpoint, measure it, and decide what to change next. The more rounds it fits into a budget, the better the recipe, but two steps of this loop are slow (Figure 2, left).
Measuring is slow, because reasoning models think for thousands of tokens per answer and each problem must be sampled many times, so a full evaluation of one checkpoint takes dozens of GPU-hours. Building is slow, too, because most PTQ methods live in separate codebases. Left to port them itself, the agent wrote 3.1k lines of code per search and still ended up less accurate. QDA makes both steps fast (Figure 2, right). Figure 3 shows how the pieces fit: the agent works in a sandbox and reaches the library, the evaluator and the GPUs only through four commands. The harness around it is built on Humanize, our agent flow system, which fences the agent's files and network, restarts a stalled turn and keeps a long search running unattended.
qda to build, kl to score, qsched for other GPU jobs such as long generations, and handin to submit. Reference traces and test benchmarks stay outside its sandbox, so it cannot tune on them, and every GPU job goes through one scheduler that keeps the search within budget.2.1Measuring: a score in minutes
QDA scores a candidate in minutes, 350–990× faster than a full evaluation, by how far its next-token predictions drift from the original model's. A quantized model is good when it behaves like the original one, so instead of grading answers, QDA measures the KL divergence between the two models' predictions on reasoning traces that the original model wrote. The original model's side is computed once and cached, so a score needs only one pass of the candidate. The traces stay outside the agent's sandbox, so it cannot tune on them.
The score ranks checkpoints the way accuracy does. Most PTQ methods minimize the error of each layer on its own, which cannot see errors that build up across layers. On Qwen3.5-9B at W4A16, this layer-wise error ranks AWQ last of three public recipes although it is second in accuracy, whereas KL gets the order right (Figure 4). Over 44 pairs of checkpoints, KL picks the more accurate one 82% of the time, against 73% for the layer-wise error, though it does not catch every failure (§3).
2.2Building: parts that snap together
QDA's library implements 21 PTQ methods behind one interface. Because every method takes a checkpoint and returns one, the agent can stack them in any order, and a method written once runs on new architectures. Each method changes one part of the math behind a quantized layer, which Figure 5 lets you assemble.
It is also fast enough for the loop. On one H100, a GPTQ build of Qwen3.8-27B takes 191 s against 501 s with llm-compressor, and the 35B MoE builds in 181 s, where llm-compressor runs out of memory.
3One search, start to finish
Each search runs Claude Opus 5.5 in Claude Code at high effort, driven by Humanize, on eight GH200 GPUs for eight hours, with no human in the loop, for $19–75 of agent tokens per search (37–122M tokens, mostly read from cache). Figure 6 replays the one on Qwen3.6-35B at W4A16.
The agent first rebuilds GPTQ with different clip and solver settings, none of them better than the public checkpoint. At 1.5 hours, it calibrates on the original model's own reasoning traces instead of generic text, and the KL drops by 23% in one step. Its notes, where the “anchor” is the original model, spell out both the bet and the risk:
Calibrating GPTQ on the anchor's own generated thinking traces is the biggest lever so far (…), but it raises ref KL: it specialises on anchor-style text.
Its KL looked fine, but KL scores the model on text it is given, not on text it writes. So at 3.1 hours the agent had this candidate write full answers to 191 hard problems of its own choosing, and 12 were cut off at the length limit, most of them code answers stuck in a loop, against 3 for the original model. The agent starts watching truncations (“the truncation count is the signal I watch”) and turns to distillation on the quantized model's own long answers. Its hand-in scores 0.0088 against 0.0153 for the best public recipe and cuts off only 6 answers.
4Results
We ran one search per model and precision on four reasoning models (dense, linear-attention hybrid and MoE) and evaluated the final checkpoints on math, GPQA-Diamond, LiveCodeBench and IFBench.
4.1Over the best public recipe
QDA is more accurate than the best public recipe on 11 of the 12 pairs of model and precision, by 0.9, 1.4 and 2.2 points on average at W4A16, W4A8 and W3A16 (Figure 7). The largest gains, +5.2 on Qwen3.5-9B at W4A8 and +6.1 on R1-Distill-Llama-8B at W3A16, come where the best public recipe is far below BF16, by 7.1 and 11.7 points.
4.2A 320B model at NVFP4
On GLM-5.3-Flash, a 320B MoE that none of the main searches saw, one 6.5-hour search for $69 recovers 98.3% of the FP8 model's accuracy, against 96.9% and 96.3% for the NVFP4 checkpoints that NVIDIA and Red Hat AI released (Figure 8). It used the same agent, library and evaluator as the main searches, with no public recipe to start from.
4.3Over an agent with NVIDIA ModelOpt
A strong agent alone is not enough. NVIDIA's ModelOpt ships a recipe-search skill for coding agents, so we gave the same agent ModelOpt and that skill instead of our tools, with the same 64 GPU-hours. It falls 0.5–1.3 points behind QDA, and even below the best public recipe at W4A8 and W3A16 (Figure 9), although it picks its candidates on the test benchmarks themselves.
4.4Free at inference
QDA's checkpoints decode as fast as GPTQ's (Figure 10). ParoQuant, the strongest public recipe on Qwen3.5-9B, earns part of its accuracy with a rotation applied to each layer's input at serving time, a sensible trade that comes with its own kernel; at batch sizes 16 and 64, QDA's checkpoints decode 1.3–1.6× faster. QDA only uses steps that fold into the weights, so its checkpoints keep the standard formats and kernels.
5What we learned, and what's next
PTQ methods are parts, not rivals. Papers pit one method against another, as if the field were a contest with a single winner. Yet each method fixes one kind of error, and which error dominates depends on the model; so the best checkpoints came from stacking known methods, and a method that loses in isolation can still be the step a recipe needs.
Agents are smart enough already; they need fast feedback and good tools. With a score that takes minutes, each search tried 5 to 36 candidates, using only 4% of its GPU time on scoring. Without the library, by contrast, the agent spent its effort on porting code and never tried a rotation at W4A8, the step that gave QDA its largest gain there.
What's next: agents that make their own tokens cheaper. A search spends tokens to make tokens cheaper: $19–75 of agent tokens buys a checkpoint that serves every later token with about a quarter of the weight memory. Pointed at the open models that agents themselves run on, this becomes a loop that feeds itself, and we plan to close it in three steps. First, day-one quantization: every new open model gets searched recipes for every format on release day. Second, a library that grows itself: our agents already invented steps of their own, such as training on the quantized model's answers and averaging checkpoints, and the ones the evaluator confirms can join the library, so each search starts from more than the last. Third, the same loop beyond weights, for KV caches, sparsity and new low-bit formats co-designed with their kernels, where coding agents such as NVIDIA's Kernel Design Agents[4] already write faster kernels. We hope QDA answers the question DeepCompressor raised, with a tool anyone can run.
Acknowledgements
We thank Han Cai, Yi Pan, Haocheng Xi, Yujun Lin, Ziyi Xu, Mark Kim and Enrico Shippole for their feedback. This research was supported by SK Telecom and the MIT Generative AI Impact Consortium (MGAIC). Jenia Jitsev and LAION provided access to compute, and the Gauss Centre for Supercomputing e.V. (GCS) provided computing time on the JUPITER supercomputer at the Jülich Supercomputing Centre (JSC).
References
[1] Yang et al. Large Language Models as Optimizers. ICLR 2024.
[2] Novikov et al. AlphaEvolve: A coding agent for scientific and algorithmic discovery. 2025.
[3] Humanize: the agent flow system. github.com/humanfia/humanize.
[4] KDA: Kernel Design Agents, an agent-centric workflow for high-performance CUDA kernels. github.com/NVlabs/kda.
Citation
Please cite this work as:
Xingyang Li, Tianlang Zhao, Song Han and Ligeng Zhu. “QDA: Closed-Loop LLM Quantization Design with Agents.” arXiv:2611.23388, 2026.
Or use the BibTeX citation:
@article{li2026qda,
title = {QDA: Closed-Loop LLM Quantization Design with Agents},
author = {Li, Xingyang and Zhao, Tianlang and Han, Song and Zhu, Ligeng},
journal = {arXiv preprint arXiv:2611.23388},
year = {2026}
}