メニュー
Load large LLMs with 4-bit NF4 quantization and optional double quantization via BitsAndBytes to reduce GPU memory by 4x while preserving inference quality
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Memory-efficient Keras generator that streams sharded CSV files in chunks, renders strokes to images on-the-fly, and yields batches for training on datasets too large for memory
Stack 1D convolutions for local feature extraction before bidirectional LSTMs to classify variable-length stroke sequences into hundreds of doodle categories
Partition a large dataset into N balanced shards using integer key modulo arithmetic for reproducible, class-interleaved splits across CSV files
Normalize raw stroke coordinates to 0-255 range, resample at uniform arc-length spacing, then apply Ramer-Douglas-Peucker simplification
Render stroke sequences to grayscale images with temporal intensity encoding where earlier strokes are brighter and later strokes fade to encode drawing order
Score each face in a video as anomalous by computing L2 distance from the embedding centroid of all faces, then convert to probability via logistic function
| name | llm-4bit-nf4-double-quantization |
| description | Load large LLMs with 4-bit NF4 quantization and optional double quantization via BitsAndBytes to reduce GPU memory by 4x while preserving inference quality |
NormalFloat4 (NF4) quantization maps weights to a 4-bit data type optimized for normally-distributed neural network weights. Double quantization further compresses the quantization constants themselves. Together they reduce a 7B model from ~14GB (fp16) to ~4GB, fitting on a single consumer GPU. Quality loss is minimal for inference tasks like scoring, generation, and classification.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16,
bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(
"mistralai/Mistral-7B-Instruct-v0.1",
quantization_config=bnb_config,
device_map="auto",
torch_dtype=torch.bfloat16,
)
tokenizer = AutoTokenizer.from_pretrained(
"mistralai/Mistral-7B-Instruct-v0.1")
BitsAndBytesConfig with NF4 quant type and bfloat16 compute dtypedouble_quant=True for additional memory savings (~0.4GB on 7B)device_map="auto" for automatic GPU placement