Learning Adaptive Safety Margins for Visual Navigation

Real-world demo

A Unitree G1 in a cluttered lab, trained only in simulation

The policy proposes many candidate trajectories from egocentric RGB-D, then a learned critic scores them and executes the best one — with no map building and no task-specific tuning.

Navigating clutter. The humanoid moves through a lab with narrow passages and irregular obstacles, driven end-to-end from RGB-D.

Proposing trajectories. From the robot's first-person view, the diffusion generator samples K = 16 candidate paths under the same observation.

Selecting the safest. The critic ranks every candidate and the highest-scoring trajectory is executed.

Motivation

How much room to leave is a property of the situation

How much clearance a robot needs is not a constant a designer gets to choose in advance. It is a property of the space the robot is in at that moment: set the margin high and it detours and times out in open corridors, set it low and it cuts corners in clutter under perception bias. A single global threshold cannot be right everywhere — which means the margin should be read off what the robot currently sees, not fixed in a configuration file.

Comparison between a fixed-margin critic and the adaptive safety critic: fixed margins cause conservative detours or brittle near-boundary choices, while the learned critic adapts its clearance budget. — Fixed margins cause conservative detours or brittle near-boundary choices. Our critic learns a clearance preference for navigation in clutter.

Method

Generate, then select with a learned critic

Separate proposing from judging. A diffusion generator proposes candidate trajectories from RGB-D and the goal, and a learnable safety critic — carrying a trajectory-dependent clearance budget rather than a global one — scores and ranks them. Geometry enters only as a teacher: ESDF is privileged information used offline to supervise training, so what conditions the deployed policy is the interaction it can actually observe. It never builds a map.

System architecture: a diffusion policy samples candidate trajectories from RGB-D observations and the goal; a critic-function based selector scores them. Training is two-stage, with ESDF supervision used offline only. — During inference the diffusion policy samples candidates from RGB-D and the goal, and the selector scores them without map building. ESDF supervises training offline.

Results

PointGoal navigation in HM3D and MP3D

300 episodes per policy, identical start–goal pairs across all methods, 500-step timeout. Cross-dataset transfer is HM3D → MP3D.

Success rate (SR) and success weighted by path length (SPL); higher is better.
Method	HM3D		MP3D		Cross-dataset
Method	SR	SPL	SR	SPL	SR	SPL
NavDP	0.710	0.529	0.627	0.431	0.523	0.349
iPlanner	0.653	0.573	0.603	0.492	0.570	0.436
ViPlanner	0.683	0.508	0.577	0.407	0.553	0.413
PointNav	0.547	0.406	0.390	0.251	0.340	0.216
Ours	0.783	0.611	0.680	0.523	0.597	0.465

Real-world PointGoal on a Unitree G1, 10 episodes per scene (successes and rate).
Method	Corridor (easy)	Apartment (medium)	Lab (hard)
NavDP	10/10 (1.0)	9/10 (0.9)	6/10 (0.6)
ViPlanner	9/10 (0.9)	6/10 (0.6)	4/10 (0.4)
PointNav	5/10 (0.5)	5/10 (0.5)	0/10 (0.0)
EGO-Planner	5/10 (0.5)	4/10 (0.4)	1/10 (0.1)
Ours	10/10 (1.0)	10/10 (1.0)	8/10 (0.8)

Qualitative

What the critic sees

The highest- and lowest-scored trajectories back-projected onto the robot's first-person view across all three environments.

Real-world results: highest- and lowest-scored trajectories projected onto the robot first-person view across corridor, apartment and lab scenes. — Blue-to-red encodes the critic score from low (dangerous) to high (safe). Across corridor, apartment and lab, the policy keeps consistent margins as scene difficulty increases.

A real-world test at checkpoints A and B with Degree of Narrowness near 0.75: NavDP fails at both while our policy passes the cluttered obstacles. — At Degree of Narrowness ≈ 0.75, NavDP with a fixed d_min = 0.5 m judges the corridor impassable and stalls (left), while our policy tightens its learned constraint and passes (right).

Open source

Code

The training code for the diffusion generator and the safety critic is available now; the Habitat-based data collection follows.

Data collection

data-collection

Collects paired RGB-D observations and smooth navigable trajectories from HM3D, MP3D and Gibson scenes.

Random start–goal sampling with floor detection
Shortest-path planning over the habitat-sim navmesh
Cubic-spline smoothing and obstacle-clearance refinement
Domain randomization of camera height and pitch

Repository — coming soon

Training

safety-critic

Trains the diffusion generator, then the safety critic as a teacher, then distils it into the RGB-D selector.

Conditional diffusion generator (DDPM, K=16 candidates)
Context-conditioned margin head q_η, CBF residual, gated detour ratio
Adversarial teacher training against A*-sampled non-expert trajectories
Student distillation: RGB-D only, no ESDF at inference

Repository →

Citation

BibTeX

@inproceedings{hu2026adaptive,
  title     = {Learning Adaptive Safety Margins for Visual Navigation},
  author    = {Hu, Junyi and Yuan, Shuaihang and Bethala, Geeta Chandra Raju
               and Fang, Yi and Tzes, Anthony},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year      = {2026}
}