STAR: A Spatial-Topology Aware Routing Framework for Generalizable 3D Scene Understanding
*Equal contribution †Corresponding author
Abstract
Constructing a unified 3D scene understanding model has long been hindered by the significant topological discrepancies across different sensor modalities. While applying the Mixture-of-Experts (MoE) architecture is a flexible approach for multi-domain 3D understanding, we observe that conventional feature-only MoE routers may underrepresent local sampling topology under semantic supervision, making expert allocation difficult when semantic consistency coexists with geometric heterogeneity. To overcome this challenge, we propose STAR (Spatial-Topology Aware Routing Framework). Specifically, we introduce a multi-attribute self-supervised pre-training branch, covering topological and textural variations, to anchor cross-domain structural priors. Building upon this, we design a domain-aware expert branch with two mechanisms: Domain-Spatial-Guided Routing (DSR), which captures local topological variations from spatial context, and Entropy-controlled Dynamic Allocation (EDA), which adjusts the number of activated experts according to routing uncertainty. Together, these branches combine stable cross-domain representation learning with adaptive expert allocation. Extensive experiments across various tasks, encompassing both indoor and outdoor scenes, demonstrate the effectiveness of STAR. It achieves 80.1% mIoU on the ScanNet validation set and 77.2% mIoU on S3DIS, consistently improving over strong baselines.
Introduction Insight
Compared with vanilla MoE that tends to reuse similar experts for semantically similar objects, STAR adaptively changes expert combinations according to topology variations across datasets.
Results
Performance. STAR achieves consistent gains across indoor benchmarks, improving key metrics such as mIoU while maintaining strong generalization.
Segmentation visualization. Compared with previous methods, STAR yields cleaner semantic boundaries.
| Method | F1@0.25 | F1@0.5 |
|---|---|---|
| SpatialLM + Sonata | 58.9 | 49.5 |
| SpatialLM + STAR (Ours) | 60.8 | 51.9 |
On ARKitScenes validation, STAR improves multimodal detection by +1.9 at F1@0.25 and +2.4 at F1@0.5 over the Sonata-based baseline.
Detection visualization. The model captures fine-grained spatial cues more robustly, improving localization quality under cross-domain scene variations.
Expert Activation Under Topology Variations
Under dropout and mask perturbations, vanilla MoE keeps highly homogeneous top-2 activation patterns, while STAR shows larger distribution shifts, indicating stronger topology sensitivity and adaptive routing.
STAR dynamically expands from top-2 to top-k expert combinations when topology becomes more uncertain, supporting robust performance under severe structural changes.
Pretraining Strategy Comparison
| Method | 2D | 3D | mIoU | mAcc | allAcc |
|---|---|---|---|---|---|
| Sonata | ✗ | ✓ | 79.4 | 86.1 | 92.5 |
| Sonata + Ours | ✗ | ✓ | 79.7 | 87.0 | 92.8 |
| STAR (Ours) | ✗ | ✓ | 80.1 | 87.3 | 93.1 |
| Concerto | ✓ | ✓ | 80.7 | 87.4 | 93.1 |
| Concerto + Ours | ✓ | ✓ | 81.2 | 89.1 | 93.2 |
STAR is compatible with different pretraining paradigms and consistently improves performance on top of both 3D-only and joint 2D-3D pretrained representations.
BibTeX
@article{xing2026star,
title={STAR: A Spatial-Topology Aware Routing Framework for Generalizable 3D Scene Understanding},
author={Xing, Mingwei and Wang, Xinliang and Shi, Yifeng},
journal={arXiv preprint arXiv:2608.11699},
year={2026}
}