gptq
Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.
来源信息
- 仓库
- synthetic-sciences/openscience
- 最近来源活动
- 2026年9月16日 06:26
- 检测到的 SKILL.md 语言
- 英语
- 星标
- 4,400
- Forks
- 555
安装方式
默认使用会先检查来源的 Prompt;你也可以切换为直接命令,或下载本地副本。
检查来源文件
决定是否安装前,请先阅读 SKILL.md,以及 SkillsMP 当前展示的配套文件。
正在显示 SKILL.md
| name | gptq |
|---|---|
| description | Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning. |