Quantization: GPTQ vs AWQ
Compare post-training quantization models for squeezing massive LLMs onto standard GPUs.
Jul 2, 20261 min read
Command Palette
Search for a command to run...
#quantization
Articles tagged with #quantization
Types of LLM Quantization: By Timing, Scope, and Mapping
PTQ, QAT, INT8, INT4, and NF4 explained through timing, scope, and mapping choices.
Mar 14, 202617 min read118
Practical LLM Quantization in Colab: A Hugging Face Walkthrough
A Colab-first Hugging Face guide to quantize open LLMs and run real inference code.
Mar 12, 202615 min read346GPTQ vs AWQ vs NF4: Choosing the Right LLM Quantization Pipeline
A practical comparison of GPTQ, AWQ, and NF4 quantization pipelines for LLM inference.
Mar 12, 202615 min read88LLM Model Quantization: Why, When, and How to Deploy Smaller, Faster Models
Cut GPU memory and latency by converting FP16 weights to INT8 or INT4 — without retraining from scratch.
Mar 8, 202613 min read43