ffacenerf-few-shot-face-editing

name	ffacenerf-few-shot-face-editing
title	FFaceNeRF: Few-shot Face Editing in Neural Radiance Fields
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2503.17095
keywords	["neural-radiance-fields","face-editing","few-shot-learning","3d-face-modeling","latent-mixing"]
description	Edit 3D faces with flexible mask layouts using only a few training samples. FFaceNeRF employs geometry adapters with feature injection and latent mixing for tri-plane augmentation, enabling rapid NeRF adaptation without fixed segmentation masks. Ideal for personalized medical imaging, creative face editing, and applications requiring user-defined mask control.

Core Concept

FFaceNeRF tackles a fundamental limitation in NeRF-based 3D face editing: existing methods rely on fixed segmentation masks predefined during training, severely restricting user control. Changing mask layouts requires retraining with large datasets—impractical for personalized applications like medical imaging or creative editing. FFaceNeRF introduces a flexible NeRF editing framework that adapts to arbitrary mask layouts using only a handful of training images. The key innovation combines two complementary techniques: geometry adapters with feature injection (for precise geometric control) and latent mixing for tri-plane augmentation (enabling training with few samples). This empowers applications where custom masks and rapid adaptation are critical.

Architecture Overview

The system integrates four essential components:

• Geometry Adapter with Feature Injection: Parameterized module that modulates geometry attributes without full NeRF retraining, enabling manipulation of face shape while preserving appearance
• Latent Mixing for Tri-Plane Augmentation: Augmentation strategy in latent space that synthesizes diverse training views from few samples, expanding effective training data
• Mask-Guided Editing Pipeline: Accepts user-defined masks and applies geometry changes selectively to masked regions
• Rapid Model Adaptation: Few-shot fine-tuning that converges quickly on new mask layouts and editing targets

Implementation Steps

1. Initialize NeRF and Geometry Adapter

Set up base NeRF model and attach geometry adapter for targeted manipulation:

import torch
import torch.nn as nn
from typing import Tuple

class GeometryAdapter(nn.Module):
    """
    Learnable adapter for geometry modification in NeRF.
    Uses feature injection to modulate density and position shifts.
    """
    def __init__(self, hidden_dim: int = 64, input_dim: int = 3):
        super().__init__()
        self.input_dim = input_dim
        .hidden_dim = hidden_dim

        
        .fc1 = nn.Linear(input_dim, hidden_dim)
        .fc2 = nn.Linear(hidden_dim, hidden_dim)
        .fc3 = nn.Linear(hidden_dim, input_dim)  

        
        .density_head = nn.Linear(hidden_dim, )

     () -> [torch.Tensor, torch.Tensor]:
        
        x = torch.cat([positions, features], dim=-)  features     positions
        h = torch.relu(.fc1(x))
        h = torch.relu(.fc2(h))

        
        position_offset = torch.tanh(.fc3(h)) *   

        
        density_modulation = .density_head(h)

         position_offset, density_modulation