SceneAligner: 3D-Grounded
Floorplan Localization in the Wild

Junhyeong Cho1 Ruojin Cai2 Hadar Averbuch-Elor1
  • 1Cornell University
  • 2Kempner Institute, Harvard University
Paper arXiv Code Visualization BibTeX

TL;DR Our method localizes in-the-wild photos within a reference floorplan by reconstructing a gravity-aligned 3D scene and aligning it with the floorplan.

Given a collection of in-the-wild images and a rasterized floorplan, SceneAligner reconstructs a gravity-aligned 3D point cloud from the images and globally aligns this reconstruction to the 2D floorplan map, thereby localizing the images within the floorplan. As illustrated above, our approach successfully aligns images capturing large-scale 3D environments, including exterior scenes (Doddabasappa Temple, left) and interior spaces (Église Saint-Martin d’Agonac, right).

Abstract

Many public buildings provide floorplans with a “you are here” indicator to help visitors orient themselves. Floorplan localization seeks to computationally replicate this capability by determining where visual observations were captured within a floorplan. However, existing methods typically assume controlled small-scale environments and precise vectorized floorplans, limiting their ability to operate in large-scale buildings and rasterized floorplans. In this work, we present an approach for performing floorplan localization in the wild by grounding the task in a reconstructed 3D representation of the scene. Given an unconstrained image collection, our method reconstructs a gravity-aligned 3D scene and projects it into a 2D density map that serves as a floorplan proxy. Floorplan localization is then formulated as aligning this proxy with the input floorplan via a 2D similarity transform. To bridge the appearance gap between density maps and architectural floorplans, we adapt a 2D foundation model to learn cross-modal correspondences, introducing a fine-tuning scheme that encourages semantically aligned matches while preserving structural consistency. Extensive experiments demonstrate substantial improvements over prior methods, including in extremely sparse settings with as little as a single input image. Our code and data will be publicly available.


Method Overview

Given in-the-wild images and a floorplan, SceneAligner reconstructs a gravity-aligned 3D scene, extracts a 2D density map via projection, and solves for a 2D similarity transform 𝐌 via correspondence estimation between the density map and floorplan using a shared encoder. Reliable correspondences used to compute 𝐌 are overlaid (in orange) on the aligned density map.


Experimental Results

Interior 3D Scene
Exterior 3D Scene

Floorplan-aligned 3D Scene Reconstructions. Our method effectively aligns in-the-wild 3D scene reconstructions with their corresponding floorplans.


360° View Renderings of Floorplan-Aligned Scenes
In-the-Wild Floorplan Localization Results

We show localization results on the C3 dataset, with one random sample per scene. We visualize correspondences (photo ↔ density map ↔ floorplan) estimated by Ours, and the camera poses estimated by Ours, C3Po, DUSt3R, and LoFTR, alongside Ground Truth.

Image
Floorplan
LoFTR DUSt3R C3Po Ours GT
Correspondence (Ours)
Camera Pose (All)
| Scene | Abbaye de Chaalis - Abbatiale
1 / 111

Quantitative Evaluation on In-the-Wild Data

We evaluate camera pose and cross-modal correspondence estimation on the C3 dataset. Our fine-tuning scheme substantially improves the 2D foundation model (DINOv3). Ours outperforms LoFTR, DUSt3R, and C3Po by a large margin across all evaluation metrics.

LoFTR DUSt3R C3Po DINOv3 Ours

Applications

Using the floorplan as a shared geometric bridge, our approach enables separate reconstructions to be registered into a common coordinate system even when direct 3D alignment is not possible. Two such challenging examples are provided below.

Alignment of interior and exterior 3D scenes, registered despite minimal visual overlap and large viewpoint differences between indoor and outdoor environments.


Alignment of disjoint 3D scenes, registered from disjoint image collections with no visual overlap.

BibTeX

@article{cho2026scenealigner,
  title={SceneAligner: 3D-Grounded Floorplan Localization in the Wild},
  author={Junhyeong Cho and Ruojin Cai and Hadar Averbuch-Elor},
  journal={arXiv preprint arXiv:2605.22581},
  year={2026}
}