additivellm2-domain-adaptation

name	additivellm2-domain-adaptation
title	AdditiveLLM2: Multi-Modal Language Models for Additive Manufacturing
version	0.0.3
engine	skillxiv-v0.0.3-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.22017
keywords	["Domain Adaptation","Language Models","Additive Manufacturing","Knowledge Extraction","Visual Understanding"]
description	Adapt general LLMs to specialized manufacturing domains via domain-adaptive pretraining on open-access journals and visual instruction tuning. Extract 50M tokens and 24K images from peer-reviewed papers, achieve >90% accuracy on domain knowledge tasks, and enable real-time defect identification from manufacturing images.

AdditiveLLM2: Domain Adaptation for Manufacturing

Domain Problem: Knowledge Gap in General LLMs

Additive manufacturing (AM) involves specialized terminology (LPBF, FDM, melt pool dynamics, material properties) and visual patterns (layer defects, porosity) that general LLMs rarely encounter. Manufacturing practitioners need models that understand equipment specifications, material behavior, and quality control—not general knowledge about 3D printing.

Gap Analysis: General Gemma-3-12B achieves <50% accuracy on manufacturing knowledge tasks; domain-specific terminology is treated as out-of-vocabulary or incorrectly classified.

Source Method & Adaptation Recipe

Foundation Model: Gemma-3-12B (open-weights baseline)

Domain Data Curation: Extract text and images from 1,704 peer-reviewed papers across four open-access journals:

• Journal of Additive Manufacturing
• Rapid Prototyping Journal
• Specialized AM conferences (via arXiv)

Dataset Composition:

• 29 million text tokens (50M target after tokenization)
• 24,000 images with captions
• Focus on: process parameters, material properties, defect modes, quality metrics

Three-Stage Training Pipeline:

1. Text Domain-Adaptive Pretraining (DAPT): Unsupervised MLM on AM corpus; trains terminology and concept associations specific to manufacturing.

# Domain-adaptive pretraining: continuous pretraining on domain data
# Train masked language modeling on AM corpus
# This builds internal representations for:
# - Equipment types (LPBF, FDM, SLM, DMLS)
# - Material properties (viscosity, thermal conductivity, porosity)
# - Defect modes (layer adhesion, warping, spatter)

2. Image Domain-Adaptive Pretraining: Vision encoder fine-tuning on AM images; learns visual patterns specific to manufacturing artifacts and defects.

3. Visual Instruction Tuning: Supervised fine-tuning on (image, question, answer) triples extracted from papers; teaches model to answer AM-specific questions about images.

# Instruction tuning examples:
# Q: "What defects are visible in this LPBF part cross-section?"