low-resource-ai-researcher

name	low-resource-ai-researcher
description	Train high-performance medical LLMs on consumer GPUs using parameter-efficient fine-tuning
version	1.0.0
category	Research
tags	[]
author	AIPOCH
license	MIT
status	Draft
risk_level	Medium
skill_type	Tool/Script
owner	AIPOCH
reviewer
last_updated	2026-02-06

Skill: Low-Resource AI Researcher

ID: 215
Category: AI/ML Research
Language: Python
Framework: PyTorch + PEFT (LoRA/QLoRA) + Transformers

Overview

Based on Parameter-Efficient Fine-Tuning (PEFT) technology, trains high-performance medical domain large language models on consumer-grade GPUs or single A100. Supports advanced fine-tuning methods such as LoRA, QLoRA, optimized for medical text understanding and generation tasks.

Features

• 🚀 Parameter-Efficient Fine-Tuning: LoRA, QLoRA, DoRA support
• 🏥 Medical Domain Optimized: Pre-configured for medical QA, diagnosis, clinical notes
• 💻 Low-Resource Ready: Optimized for consumer GPUs (RTX 3090/4090) and single A100
• 📊 Quantization: 4-bit/8-bit quantization with bitsandbytes
• 🔄 Multi-Task: Supports SFT, DPO, and medical instruction tuning
• 📝 Medical Datasets: Built-in support for PubMedQA, MedQA, MIMIC-III

Installation

# Core dependencies
pip install torch transformers datasets accelerate peft bitsandbytes

# Optional for training optimization
pip install flash-attn --no-build-isolation
pip install wandb tensorboard

# Medical NLP utilities
pip install scispacy scikit-learn

Quick Start

from skills.low_resource_ai_researcher.scripts.main import MedicalPEFTTrainer

# Initialize trainer
trainer = MedicalPEFTTrainer(
    model_name="meta-llama/Llama-2-7b-hf",
    task="medical_qa"
)

# Train with LoRA
trainer.train(
    output_dir="./medical_lora_model",
    num_epochs=3,
    batch_size=4,
    use_qlora=True  # 4-bit quantization
)

Configuration

Hardware Profiles

Profile	GPU Memory	Quantization	Max Model Size