Huawei open-sources MACaQ, quantizing a 122B model in 64 minutes
Huawei's cloud and AI lab released MACaQ, an open-source quantization tool, under Apache 2.0.
· Last verified: 13 Nov 2026 (Huawei (huawei-csl GitHub repository), arXiv, AI Weekly)

Summary
- Huawei's cloud and AI lab released MACaQ, an open-source quantization tool, under Apache 2.0.
- It covers text, image, video and speech models across llama.cpp, vLLM, Diffusers and PyTorch.
- Developers can compress large models, including Qwen, and run them locally on lighter hardware.
The latest
A quantization tool that its authors say compressed a 122-billion-parameter model in 64 minutes using 32.5GB of device memory is now open source. Huawei's cloud and AI lab published MACaQ under the huawei-csl account on GitHub with an Apache 2.0 license, bundled external components keeping their own terms. The repository positions it as model-agnostic quantization that goes beyond text.
Details
- The tool: MACaQ stands for Model Agnostic Calibrated Quantization. The repository tagline advertises high-quality GGUF for local inference and quantization that is not tied to a single model family or modality. Reports logged the release on 12 November 2026, with AI Weekly timestamping its alert at 09:29 GMT.
- The modalities: Four are supported: text, image, video and audio, with audio covering speech generation. That breadth is the pitch — a single pipeline rather than separate per-modality quantization toolchains, which until now has forced developers to stitch together different tools or wait for labs to publish compressed checkpoints.
- Frameworks and outputs: llama.cpp handles text models at 2-, 3- and 4-bit in GGUF format with CPU execution support. vLLM covers text generation and serving in W4A4 mode, the default, or weights-only. Diffusers handles image, video and audio generation in W4A4 with safetensors output.
- Speech path: PyTorch carries speech generation in W4A4 mode, with Qwen3-TTS and Chatterbox Turbo named among the supported models — placing text-to-speech inside the same quantization workflow as text and diffusion models.
- The hardware: Ascend support arrives through pending merge requests in CANN covering dense and grouped W4A4 kernels. Full model execution, the repository states, still requires integration with the Ascend inference runtime, meaning end-to-end deployment on Huawei silicon is not yet complete.
- The model list: The profile names Qwen3 at 4B, 27B, 30B, 35B and 80B, alongside OLMoE-1B-7B, FLUX.2-klein-4B, Wan2.2-TI2V and Stable Audio 1.0. A pre-quantized checkpoint, Qwen3.6-35B-A3B-MACaQ-R, ships with the code in GGUF format.
- The benchmarks: The authors report lower perplexity than MR-GPTQ and DuQuant++, or matched-to-better perplexity on Llama-3-8B at 11 times faster quantization. On Qwen3.8-27B in vLLM, BF16 perplexity of 2.8463 compares with 2.8994 for MACaQ W4A4, alongside 2.33x prefill and 2.58x decode speedups.
- The paper: The method is described in arXiv 2611.10649, "MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations," listed as published 7 November 2026. The authors are Muller, Bich, Boretti, Chang, Zhuang and Cavigelli.
- The method: Each linear layer is quantized independently. Blockwise Hadamard transforms are applied to activations with the inverse transforms folded into the weights, and a Flip-Refinement component improves MXFP4 codes. Repository statistics at reading stood at 16 commits and 35 stars.
Between the lines
The claimed 11x quantization speedup and the 64-minute, 32.5GB figure target cost rather than accuracy alone: compression that fits on modest hardware lets developers produce their own low-bit builds instead of waiting on labs. The early repository activity — 16 commits, 35 stars — and the unmerged CANN kernels suggest a project still at its opening stage.
What's next
Watch whether the CANN merge requests for dense and grouped W4A4 kernels land, and whether Ascend inference runtime integration closes the gap to full model execution. Independent reproduction of the Qwen3.8-27B perplexity and speedup figures is the other test.
Source: Huawei (huawei-csl GitHub repository), arXiv, AI Weekly