Urban Autonomous Rover
Heterogeneous Embedded System for Real-Time Visual Navigation Under Constraints
Project Overview & Description
Most open-source ground robots handle basic path-following and obstacle avoidance, but few can reason about nuanced urban scenarios like navigating a controlled intersection. This project pairs the general reasoning ability of an LLM/VLM with a dedicated low-level navigation model to close that gap — while working under a strict budget and power constraint that rules out LiDAR, relying instead on a single commercial webcam. To make rapid prototyping possible without expensive positioning hardware, I engineered iSLAM, a framework that bridges an iPhone's pose estimation and GPS to the onboard Jetson computer. The project is still in active development: autonomy has not yet been fully tested end-to-end, though manual control (driven via an iPhone-based joystick connected to the Jetson's hotspot) is working, along with a wireless dashboard for monitoring status.
Project contributions
- iSLAM: bridges iPhone pose estimation and GPS to a Jetson-class onboard computer, enabling low-cost, LiDAR-free localization.
- A local VLM agent (Qwen3.5:2b via Ollama) with a custom tool-calling harness for scene understanding and navigation decisions, running on-device on the Jetson Orin Nano.
- Integration of an existing end-to-end navigation policy (MBRA), trained on cross-embodiment data, for real-time waypoint following and obstacle avoidance.