Comparison of Gaussian Splatting via MASt3R vs. SfM-based Methods

Introduction

In typical pipelines for generating 3D Gaussian Splatting (3DGS) from images, Structure from Motion (SfM) is used to estimate sparse point clouds and camera poses. These are then used as geometric initial values for Gaussian optimization. This design is extremely effective when there is sufficient textural correspondence and viewpoint overlap between images.

However, in cases where viewpoint differences are extreme or data contains many featureless regions, the fundamental assumptions of SfM break down, making it impossible to obtain an initial geometry. This note focuses on this "SfM Failure Regime" and constructs a Gaussian Splatting pipeline using MASt3R—an SfM-free geometry estimation method—as a geometry initializer, comparing it with traditional SfM-based methods.

What is MASt3R?

MASt3R (Multi-view Asymmetric Stereo Transformer) is an SfM-free reconstruction method that directly regresses dense 3D geometry from image pairs without requiring prior info like camera calibration. Unlike traditional SfM, which relies on step-by-step processing (feature extraction, matching, geometric verification), MASt3R uses pre-trained 3D spatial priors to estimate geometric relationships end-to-end.

This design makes MASt3R particularly resilient to typical SfM weaknesses:

• Wide-baseline pairs with extreme viewpoint changes.
• Repetitive patterns or low-texture areas.
• Sparse view graphs where sufficient matching points cannot be obtained.

Pipeline Configurations

This study compares the following three pipelines:

Results & Comparison

Dataset 1: Cyprus (SfM Failure Case)

A dataset of 30 images of ruins with scattered subjects and limited overlap. Result: SfM-based pipelines failed to find sufficient matches and could not generate a 3DGS. The MASt3R-based pipeline successfully generated a 3DGS by providing stable initial geometric values.

Dataset 2: Fountain (Standard Case)

A dataset of 30 images with abundant overlap. Result: Both methods succeeded. However, the SfM-based method was slightly superior in terms of geometric precision and texture sharpness due to the high number of correspondence points.