Geometry-guided initialization
Feature matching and homography estimation establish the panorama layout. Feather blending provides a stable coarse result instead of asking the diffusion model to infer the entire geometry from scratch.
Abstract
Image stitching usually relies on geometric alignment and handcrafted blending. While geometry can place two overlapping images on a shared canvas, the resulting panorama may still contain visible seams, exposure inconsistencies, and small residual misalignments. ISS addresses this limitation with a two-stage framework. It first estimates a homography from local feature correspondences and produces aligned image conditions, validity masks, and a coarse feather-blended panorama. A conditional latent diffusion model then refines the aligned result using both source images as explicit spatial conditions. This design preserves the dependable global structure provided by classical geometry while using generative priors to improve local appearance around difficult overlap regions.
Method
01
Input Pair
Two images with an overlapping field of view.
02
Geometry
SIFT or ORB matching, RANSAC homography, warping, and masks.
03
Diffusion
Seam-aware latent refinement conditioned on both aligned images.
04
Panorama
A coherent stitched image with improved local transitions.
Multi-image conditioning
The default U-Net input contains a 4-channel noisy target latent, two 4-channel aligned image latents, and two 1-channel validity masks, for a total of 14 channels.
Controlled refinement
At inference time, ISS can refine a noised coarse panorama while preserving known regions, or perform complete conditional generation from random latent noise.
Training Objective
ISS is trained with the scheduler-native diffusion target together with optional pixel-space objectives for reconstruction, seam correction, gradient consistency, and content preservation.
ℒ = λdiffℒdiff + λrecℒrec + λseamℒseam + λgradℒgrad + λpreserveℒpreserve
Qualitative Results
The following slots are reserved for trained results. Each row is ready for a source pair, the geometry-only baseline, and the final ISS output.
Example 01
Input A
Input B
Coarse
ISS
Example 02
Input A
Input B
Coarse
ISS
Implementation
The full backend is built on Stable Diffusion 2. ISS reuses its frozen VAE, OpenCLIP conditioning space, U-Net, and scheduler configuration while introducing its own geometric alignment, multi-image conditioning, training losses, and sampling flow. The repository includes commands for data preparation, training, evaluation, inference, and a lightweight CPU smoke-test pipeline.
View the implementation on GitHub →