Generate smooth morphing sequences between images without fine-tuning or alignment. Uses guidance-aware spherical interpolation and step-oriented attention blending to handle diverse semantic and layout scenarios, completing morphs 50× faster than fine-tuning methods.
Standardmäßig ist der Prompt ausgewählt, der zuerst die Quelle prüft. Sie können zu einem direkten Befehl wechseln oder eine lokale Kopie herunterladen.
Quelldateien prüfen
Lesen Sie SKILL.md und alle von SkillsMP angezeigten Begleitdateien, bevor Sie sich für eine Installation entscheiden.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Ein direkter Befehl überspringt den Prüf-Prompt. Prüfen Sie die Quelle, bevor Sie ihn ausführen.
Generate smooth morphing sequences between images without fine-tuning or alignment. Uses guidance-aware spherical interpolation and step-oriented attention blending to handle diverse semantic and layout scenarios, completing morphs 50× faster than fine-tuning methods.
FreeMorph: Morphing Between Any Two Images Without Training
Image morphing traditionally requires either careful manual alignment or extensive fine-tuning per image pair. This limits practical application—each new pair means retraining. FreeMorph solves this by working within standard diffusion models' self-attention mechanisms. By blending features from both images and carefully managing attention across diffusion steps, it generates smooth morphing sequences in under 30 seconds without any per-image training.
The key insight: diffusion models' attention mechanisms can naturally blend images if guided properly. Early denoising steps preserve input image structure; later steps add variation. By modulating which input image dominates at each step, smooth transitions emerge automatically.
Core Concept
Morphing requires three capabilities: (1) feature-level understanding of both images, (2) smooth interpolation between their features, and (3) handling semantic differences (objects in different positions or shapes). FreeMorph achieves this through:
Guidance-Aware Spherical Interpolation: Blend self-attention Key/Value features from both images using spherical geometry, avoiding the "shortest path" problem where linear interpolation creates unrealistic intermediate states
Step-Oriented Variation Trend: Rather than uniform weight throughout diffusion, gradually shift emphasis from left image (early steps) to right image (late steps), creating directional morphing
Improved Diffusion Process: Strategic application of different attention mechanisms at forward (corruption) and reverse (denoising) stages, plus high-frequency noise injection for flexibility
This enables smooth, realistic morphing across diverse image pairs without training.
Architecture Overview
The FreeMorph system consists of these components:
Feature Extraction from Both Images: Initial encoding of left and right images into latent space
Spherical Feature Aggregation: Geometric blending of Key/Value attention features
Prior-Driven Self-Attention: Modulation emphasizing interpolated features during forward diffusion, input images during reverse
Step-Oriented Weight Scheduling: Temporal weighting that shifts from left to right image
Attention Mechanism Switching: Different strategies for forward vs reverse diffusion processes
Inference Without Fine-tuning: Complete morphing in single forward pass with no training
Implementation
This section demonstrates how to implement FreeMorph morphing.
Step 1: Implement spherical feature interpolation
This code blends features from two images using spherical geometry:
import torch
import torch.nn as nn
import torch.nn.functional as F
import math
classSphericalFeatureInterpolation:
"""
Blend features from two images using spherical interpolation (slerp).
Avoids "shortest path" artifacts that come from linear interpolation.
""" @staticmethoddefspherical_interpolate(feat_left, feat_right, t):
"""
Spherical linear interpolation between two feature vectors.
feat_left: (B, N, D) features from left image
feat_right: (B, N, D) features from right image
t: scalar in [0, 1], interpolation parameter (0=left, 1=right)
Returns: (B, N, D) interpolated features
"""# Normalize features to unit sphere
feat_left_norm = F.normalize(feat_left, dim=-1)
feat_right_norm = F.normalize(feat_right, dim=-1)
# Compute angle between features
cos_omega = (feat_left_norm * feat_right_norm).sum(dim=-1, keepdim=True)
# Clamp to avoid numerical issues with arccos
cos_omega = torch.clamp(cos_omega, -1.0, 1.0)
omega = torch.acos(cos_omega)
# Spherical interpolation# If features are nearly parallel, fall back to linear interpolation
sin_omega = torch.sin(omega)
sin_omega = torch.where(sin_omega < 1e-6, torch.ones_like(sin_omega), sin_omega)
# Slerp formula: (sin((1-t)*w) * a + sin(t*w) * b) / sin(w)
weight_left = torch.sin((1 - t) * omega) / sin_omega
weight_right = torch.sin(t * omega) / sin_omega
# Blend using computed weights
interpolated = weight_left * feat_left_norm + weight_right * feat_right_norm
# Scale by average magnitude to preserve signal
magnitude = 0.5 * (feat_left.norm(dim=-1, keepdim=True) + feat_right.norm(dim=-1, keepdim=True))
interpolated = interpolated * magnitude
return interpolated
@staticmethoddefinterpolate_attention_features(kv_left, kv_right, t):
"""
Interpolate Key/Value attention features from both images.
kv_left: (B, N, 2*D) Key-Value features from left image
kv_right: (B, N, 2*D) Key-Value features from right image
t: interpolation parameter
Returns: blended KV features
"""# Split Key and Value
K_left, V_left = kv_left[..., :kv_left.shape[-1]//2], kv_left[..., kv_left.shape[-1]//2:]
K_right, V_right = kv_right[..., :kv_right.shape[-1]//2], kv_right[..., kv_right.shape[-1]//2:]
# Interpolate Key and Value separately
K_interp = SphericalFeatureInterpolation.spherical_interpolate(K_left, K_right, t)
V_interp = SphericalFeatureInterpolation.spherical_interpolate(V_left, V_right, t)
# Concatenate back
kv_interp = torch.cat([K_interp, V_interp], dim=-1)
return kv_interp
# Test spherical interpolation
feat_left = torch.randn(1, 64, 768) # 64 patches, 768-dim features
feat_right = torch.randn(1, 64, 768)
for t in [0.0, 0.25, 0.5, 0.75, 1.0]:
interp = SphericalFeatureInterpolation.spherical_interpolate(feat_left, feat_right, t)
# At t=0: close to left, at t=1: close to right
dist_to_left = (interp - feat_left).norm(dim=-1).mean()
print(f"t={t}: distance to left = {dist_to_left:.3f}")
This blends features using geometry instead of linear interpolation.
Step 2: Implement step-oriented weight scheduling
This code creates temporal weighting that shifts from left to right image:
classStepOrientedMorphing:
"""
Gradually shift emphasis from left image (early diffusion) to right image (late diffusion).
Creates directed morphing: left → morphing sequence → right.
""" @staticmethoddefcompute_morphing_weights(current_step: int, total_steps: int, morphing_direction: str = 'left_to_right'):
"""
Compute interpolation weight as function of diffusion step.
current_step: 0 = start (noise), total_steps-1 = end (clean image)
total_steps: total diffusion steps
morphing_direction: 'left_to_right' or 'right_to_left'
Returns: t in [0, 1], interpolation parameter
"""# Normalize step to [0, 1]
progress = current_step / (total_steps - 1)
if morphing_direction == 'left_to_right':
# Early steps: t=0 (left image)# Late steps: t=1 (right image)
t = progress
else:
# Early steps: t=1 (right image)# Late steps: t=0 (left image)
t = 1.0 - progress
return t
@staticmethoddefapply_prior_driven_attention(
self_attention_output,
interpolated_features,
current_step,
total_steps,
forward_pass: bool = True):
"""
Modulate attention output based on diffusion direction and step.
forward_pass: True during corruption (forward diffusion), False during denoising (reverse).
During forward: emphasize interpolated features (generate morphing structure)
During reverse: emphasize input images (preserve original content)
"""# Get weight for this step
t = StepOrientedMorphing.compute_morphing_weights(current_step, total_steps)
if forward_pass:
# Forward diffusion: blend toward interpolated features# This makes the model learn to generate morphing transition
blend_weight = 0.7# Emphasize interpolation
output = (1 - blend_weight) * self_attention_output + blend_weight * interpolated_features
else:
# Reverse diffusion: smoothly transition which image to emphasize# Early: use left features; late: use right features# This is implicitly handled by interpolated_features changing over time
output = self_attention_output
return output
@staticmethoddefcreate_gradual_transition(
image_left,
image_right,
num_frames: int = 30,
diffusion_steps: int = 50):
"""
Generate a morphing sequence by gradually changing interpolation weight.
Returns: list of num_frames morphed images
"""
morphing_sequence = []
for frame_idx inrange(num_frames):
# Interpolation parameter for this frame
frame_t = frame_idx / (num_frames - 1)
# During diffusion, interpolate based on step
morphed_frame = {}
morphed_frame['t'] = frame_t
morphed_frame['left_weight'] = 1.0 - frame_t
morphed_frame['right_weight'] = frame_t
morphing_sequence.append(morphed_frame)
return morphing_sequence
# Test step-oriented morphing
frames = StepOrientedMorphing.create_gradual_transition(None, None, num_frames=20)
print("Morphing frame schedule:")
for i, frame inenumerate(frames[:5]):
print(f" Frame {i}: t={frame['t']:.2f}, left={frame['left_weight']:.2f}, right={frame['right_weight']:.2f}")
This creates a temporal schedule that smoothly transitions from one image to the other.
Step 3: Implement improved diffusion with attention modulation
This code integrates spherical interpolation into the diffusion process:
classFreeMorphDiffusionModule(nn.Module):
"""
Diffusion module with morphing-aware attention modulation.
Handles both forward (corruption) and reverse (denoising) processes.
"""def__init__(self, diffusion_model):
super().__init__()
self.diffusion_model = diffusion_model
self.spherical_interp = SphericalFeatureInterpolation()
self.step_morph = StepOrientedMorphing()
defencode_images(self, image_left, image_right):
"""Encode both images into latent space."""# Use diffusion model's VAE encoder
latent_left = self.diffusion_model.vae_encoder(image_left)
latent_right = self.diffusion_model.vae_encoder(image_right)
return latent_left, latent_right
defforward_diffusion_step(
self,
x_t,
t,
latent_left,
latent_right,
timestep_in_sequence
):
"""
Single forward diffusion step with morphing guidance.
"""# Compute interpolation weight for this step
morph_t = self.step_morph.compute_morphing_weights(timestep_in_sequence, 50)
# Get interpolated features at this morphing parameter# Extract attention features from diffusion model
kv_left = self.diffusion_model.extract_attention_features(latent_left, t)
kv_right = self.diffusion_model.extract_attention_features(latent_right, t)
kv_interp = self.spherical_interp.interpolate_attention_features(kv_left, kv_right, morph_t)
# Inject interpolated features into attention
x_t_morphed = self.diffusion_model.forward_with_custom_attention(
x_t,
t,
kv_interp,
is_forward_pass=True
)
return x_t_morphed
defreverse_diffusion_step(
self,
x_t,
t,
latent_left,
latent_right,
timestep_in_sequence
):
"""
Single reverse diffusion step (denoising) with morphing guidance.
"""
morph_t = self.step_morph.compute_morphing_weights(timestep_in_sequence, 50)
# During denoising, use prior-driven attention
x_t_denoised = self.diffusion_model.denoise_step(
x_t,
t,
condition=None,
prior_guidance=morph_t # Gradually shift from left to right
)
return x_t_denoised
defgenerate_morphing_sequence(
self,
image_left,
image_right,
num_morphs: int = 20,
diffusion_steps: int = 50):
"""
Generate complete morphing sequence between two images.
"""# Encode input images
latent_left, latent_right = self.encode_images(image_left, image_right)
morphing_sequence = []
for morph_idx inrange(num_morphs):
# Initialize noise
x_t = torch.randn_like(latent_left)
# Forward diffusionfor t inrange(1, diffusion_steps):
x_t = self.forward_diffusion_step(
x_t, t, latent_left, latent_right, morph_idx
)
# Reverse diffusion (denoising)for t inreversed(range(1, diffusion_steps)):
x_t = self.reverse_diffusion_step(
x_t, t, latent_left, latent_right, morph_idx
)
# Decode to image space
morphed_image = self.diffusion_model.vae_decoder(x_t)
morphing_sequence.append(morphed_image)
return torch.stack(morphing_sequence)
# Test morphing module (placeholder)# In practice, this would use actual diffusion modelprint("FreeMorph diffusion module ready for morphing")
This integrates morphing guidance into the diffusion process.
Step 4: Add high-frequency noise injection
This code improves morphing flexibility through adaptive noise:
classHighFrequencyNoiseInjection:
"""
Inject high-frequency noise adaptively to allow greater morphing flexibility.
Helps bridge semantic gaps between very different images.
""" @staticmethoddefcompute_frequency_content(image_tensor):
"""Analyze frequency content of image using FFT."""
fft = torch.fft.fftn(image_tensor.float(), dim=(-2, -1))
magnitude = torch.abs(fft)
# High frequency: edges of spectrum
high_freq_mask = magnitude > magnitude.median()
return high_freq_mask
@staticmethoddefadaptive_noise_injection(
x_t,
image_left,
image_right,
injection_strength: float = 0.1):
"""
Inject noise proportional to semantic difference between images.
Greater difference → more noise allowed.
"""# Compute semantic difference (simplified: L2 distance)
diff = (image_left - image_right).abs().mean()
# Adapt noise strength based on difference
noise_strength = injection_strength * (1.0 + diff.item())
# Generate Gaussian noise
gaussian_noise = torch.randn_like(x_t) * noise_strength
# Add to current diffusion state
x_t_injected = x_t + gaussian_noise
return x_t_injected
@staticmethoddefhigh_frequency_component_injection(
x_t,
image_left,
image_right,
diffusion_step: int,
total_steps: int):
"""
Inject high-frequency components to preserve detail during morphing.
Especially important for handling layout differences.
"""# Extract high-frequency components from both images
freq_left = HighFrequencyNoiseInjection.compute_frequency_content(image_left)
freq_right = HighFrequencyNoiseInjection.compute_frequency_content(image_right)
# Blend frequency masks based on morphing progress
progress = diffusion_step / total_steps
freq_blend = (1 - progress) * freq_left + progress * freq_right
# Create high-freq noise from blend
hf_noise = torch.randn_like(x_t) * 0.05
hf_noise = hf_noise * freq_blend.unsqueeze(1)
# Add high-frequency component
x_t_enhanced = x_t + hf_noise
return x_t_enhanced
# Test noise injection
x = torch.randn(1, 4, 64, 64)
img_left = torch.randn(1, 3, 256, 256)
img_right = torch.randn(1, 3, 256, 256)
x_noisy = HighFrequencyNoiseInjection.adaptive_noise_injection(x, img_left, img_right)
print(f"Original x range: {x.min():.3f} to {x.max():.3f}")
print(f"Noisy x range: {x_noisy.min():.3f} to {x_noisy.max():.3f}")
This injects adaptive noise to handle diverse image pairs.
Step 5: Complete end-to-end morphing pipeline
This code combines all components:
classFreeMorphPipeline:
"""
Complete FreeMorph pipeline: morph between any two images without fine-tuning.
"""def__init__(self, diffusion_model):
self.morph_module = FreeMorphDiffusionModule(diffusion_model)
self.noise_injection = HighFrequencyNoiseInjection()
defmorph(
self,
image_left,
image_right,
num_keyframes: int = 20,
num_interpolations_per_keyframe: int = 1,
diffusion_steps: int = 50,
guidance_scale: float = 7.5):
"""
Generate complete morphing sequence between two images.
Returns: sequence of morphed images
"""print(f"Morphing {image_left.shape} → {image_right.shape}")
print(f"Generating {num_keyframes} keyframes with {num_interpolations_per_keyframe}× interpolation")
start_time = time.time()
# Generate morphing sequence
morphing_frames = self.morph_module.generate_morphing_sequence(
image_left,
image_right,
num_morphs=num_keyframes,
diffusion_steps=diffusion_steps
)
# Optionally interpolate between keyframes for smoothnessif num_interpolations_per_keyframe > 1:
# Linear interpolation in latent space
smooth_frames = []
for i inrange(len(morphing_frames) - 1):
frame1 = morphing_frames[i]
frame2 = morphing_frames[i + 1]
for j inrange(num_interpolations_per_keyframe):
alpha = j / num_interpolations_per_keyframe
interpolated = (1 - alpha) * frame1 + alpha * frame2
smooth_frames.append(interpolated)
morphing_frames = torch.stack(smooth_frames)
elapsed = time.time() - start_time
print(f"Morphing complete in {elapsed:.1f} seconds")
print(f"Generated {len(morphing_frames)} frames")
print(f"50× faster than fine-tuning methods (30s vs 25min)")
return morphing_frames
# Example usage
pipeline = FreeMorphPipeline(pretrained_diffusion_model)
img_left = torch.randn(1, 3, 512, 512)
img_right = torch.randn(1, 3, 512, 512)
morphing_sequence = pipeline.morph(
img_left,
img_right,
num_keyframes=20,
num_interpolations_per_keyframe=2,
diffusion_steps=50
)
print(f"Output shape: {morphing_sequence.shape}")
This provides a complete morphing pipeline.
Practical Guidance
When to use FreeMorph:
Interactive morphing applications requiring fast turnaround
Content creation where morphing many image pairs
Scenarios where images have different semantics or layouts
Web/mobile applications with latency constraints
Batch morphing of image collections
When NOT to use:
Ultra-high-quality studio morphing (fine-tuning methods may be better)
Real-time applications (30 seconds still substantial)
Situations where perfect semantic alignment is critical
Very low-resolution images (morphing quality depends on detail)
Domains very different from diffusion training data
Hyperparameters and Configuration:
Parameter
Recommended
Notes
Num Keyframes
20-50
More keyframes = smoother but slower
Interpolations per Keyframe
1-3
Subframe interpolation for smoothness
Diffusion Steps
50-100
More steps improve quality; 50 is good default
Guidance Scale
7.5
Standard for diffusion guidance
Noise Injection Strength
0.1
Adapt to semantic difference
High-Frequency Injection
0.05
Preserve detail across diverse images
Spherical Interpolation
always
Use spherical, never linear, for features
Common Pitfalls:
Using too few keyframes (jerky morphing)
Linear instead of spherical interpolation (shortcuts create artifacts)
Ignoring high-frequency components (detail loss)
Applying noise injection too aggressively (blurry output)
Not stepping down diffusion steps during speed-sensitive applications
Trying to morph completely unrelated images (fundamental limits exist)
Key Design Decisions:
FreeMorph works by modulating diffusion model attention without fine-tuning. Spherical interpolation preserves feature geometry better than linear blending. Step-oriented weights gradually shift emphasis from one image to the other, creating directional morphing. High-frequency noise injection handles semantic differences between images. The method requires no per-image training because it operates within the pretrained diffusion model's existing machinery.
