MambaSplat-4D: Rotation-Invariant 4D Classification of Gaussian Splatting Sequences

TL;DR: MambaSplat-4D classifies sequences of 3D Gaussian Splats (bird, drone, airplane, helicopter) with predictions that are rotation-invariant by construction, even when every frame is rotated independently.

Abstract

While 3D Gaussian Splatting (3DGS) has revolutionized real-time rendering, its potential for spatial-temporal understanding remains underexplored. Feed-forward 3DGS now enables near-real-time per-frame reconstruction from multi-view images, making temporal sequences of Gaussian Splats a practical representation for dynamic scene modeling, yet existing methods either use point clouds, lack guaranteed rotation invariance, or cannot handle temporal sequences.

We present MambaSplat-4D, a rotation-invariant architecture operating directly on the full 14-parameter feed-forward 3DGS representation (position, quaternion, scale, opacity, and DC color) in three stages: a Gaussian Lifting procedure maps all coefficients into an equivariant feature space, where a Vector Neuron transformer captures inter-Gaussian relationships; the VN-In bridge, a learned equivariant-frame projection at the spatial–temporal boundary, converts these features into rotation-invariant scalars, recovering up to the 3C−3 independent SO(3) invariants when the pooled feature is full-rank; and a Mamba state-space model encodes temporal dynamics in linear time, enabling efficient classification of dynamic Gaussian Splat sequences. The projection is invariant for any learned frame, so predictions are invariant by construction under arbitrary (per-frame) SO(3) rotations. On static 3DGS benchmarks VN-In is level with the VNStdFeature bridge at a matched ~27k bridge budget, and on temporal 4D benchmarks it reaches VNStdFeature's accuracy with 75× fewer bridge parameters. To enable evaluation of temporal 3DGS classification, we release AeroSplat-4D, a synthetic dataset pairing temporal 4DGS sequences with aerial object classes, with its generation pipeline, the MambaSplat-4D architecture, and evaluations.

In short: 3D Gaussian Splatting represents an object as thousands of small coloured 3D ellipsoids; Vector Neurons are network layers whose features rotate together with the input; Mamba is a state-space sequence model whose cost grows linearly with the number of frames.

Architecture

MambaSplat-4D extracts rotation-invariant representations from the full 3DGS parameterization in four stages. Invariance rests on a single mechanism, the equivariant-to-scalar projection at the spatial–temporal boundary; everything before it is equivariant and everything after it acts on scalars.

1. Input. A sequence of T frames, each a set of N Gaussians with 14 parameters: centre (3), rotation quaternion (4), log-scale (3), opacity (1) and the DC spherical-harmonic colour (3). The paper's operating point is N = 512 and T = 16.
2. Gaussian lifting. The position and the three scaled principal axes from the quaternion and scale form four vectors that co-rotate with the object. Opacity and colour, which are rotation-invariant, are fused alongside them.
3. VN-Transformer. Vector Neuron attention and feed-forward blocks encode relationships between Gaussians while keeping every feature equivariant.
4. VN-In bridge. The pooled per-frame features are projected onto a learned equivariant frame. The resulting 3C scalars are invariant to any rotation of that frame's input, so each frame may be rotated independently.
5. Mamba temporal encoder and head. Four Mamba blocks with sinusoidal positional encodings model the scalar sequence in linear time; mean pooling over frames and a two-layer MLP produce the class logits.

AeroSplat-4D

AeroSplat-4D is a synthetic 4D Gaussian-splat benchmark for airborne-object classification. Animated 3D assets are rendered in NVIDIA Isaac Sim on a blank stage by four static cameras 90° apart, at two elevations and ten radii from 2 m to 280 m, for 30 frames at 30 FPS. Every frame is reconstructed into 3D Gaussians by a feed-forward reconstructor and cached as a fixed-size tensor. Splits are identity-disjoint: no asset appears in more than one split.

The assets

Eight assets per class shown; every asset is animated (wing-beat, rotors, propellers).

From four views to 3D Gaussians

Each frame is lifted to 3D Gaussians in one forward pass of LGM from the four camera views. One frame per class, orbited:

What the network sees

Each reconstruction is cut down to a fixed set of N = 512 Gaussians per frame (random draw, farthest-point sampling, random draw) and normalised to the unit sphere. The classifier never sees pixels, only these Gaussians and how they change over the 30 frames.

How each class moves

The temporal encoder classifies from how the Gaussian set changes over time. The Chamfer distance of every frame to the first makes that change visible: a wing-beat returns periodically, rotors flicker, a rigid airplane stays flat.

Acknowledgements

This project was sponsored by Wim Bos, CEO of Lumiad, and carried out within the Lumiad Extended Technologies (LXT) department in collaboration with Spacecific. And to Remi, for the fruitful discussions that shaped this work.

BibTeX

@inproceedings{verdiesen2026mambasplat4d,
  title     = {{MambaSplat-4D}: Rotation-Invariant {4D} Classification of {Gaussian} Splatting Sequences},
  author    = {Verdiesen, Alessandro and Hofstee, H. Peter and Al-Ars, Zaid},
  booktitle = {Asian Conference on Computer Vision (ACCV)},
  year      = {2026}
}