FloorPlan2Guide: LLM-Based Indoor Navigation System
AI-Powered Navigation for Individuals with Blindness or Low Vision
Overview
FloorPlan2Guide is a novel AI-powered indoor navigation system that leverages GPT-4 Vision and floor plan understanding to provide real-time, context-aware navigation assistance for visually impaired individuals.
The system integrates multimodal inputs including camera feeds, floor plans, and natural language to generate precise turn-by-turn directions. By combining state-of-the-art vision-language models with spatial reasoning capabilities, FloorPlan2Guide offers an unprecedented level of independence for users navigating unfamiliar indoor environments.
Key Features
GPT-4 Vision Integration
Utilizes advanced vision-language models for real-time scene understanding and spatial reasoning.
Floor Plan Understanding
Processes architectural floor plans to build accurate spatial models of indoor environments.
Accessibility-First Design
Built from the ground up with input from visually impaired users to ensure usability.
Voice-Guided Navigation
Natural language turn-by-turn directions with context-aware announcements.
Technical Approach
1. Floor Plan Processing
The system processes CAD floor plans and building blueprints to extract navigable paths, room locations, and key landmarks. This creates a graph-based representation of the indoor environment.
2. Real-time Scene Analysis
Using GPT-4 Vision, the system continuously analyzes camera input to identify obstacles, doors, signs, and other navigational cues in real-time.
3. Multimodal Fusion
The system fuses information from the floor plan model, real-time camera analysis, and user queries to generate contextually appropriate navigation instructions.
4. Natural Language Generation
Instructions are generated in natural, conversational language that is clear and easy to follow for users with visual impairments.
System Architecture
System Architecture Diagram
View PDFFloorPlan2Guide system architecture showing floor plan parsing and navigation generation pipeline.
Technologies Used
Publications & Presentations
STARS Celebration Conference 2024
Poster presentation on LLM-based indoor navigation for accessibility.
View PosterCVPR 2026
LLM-Guided Agentic Floor Plan Parsing for Accessible Indoor Navigation of Blind and Low-Vision People. Accepted.
IISWC 2026
Characterizing an On-Device Depth-Aware Multimodal Workload in Assistive Navigation for Blind and Low-Vision Users. Accepted. With Tim Oates.
CMD-IT/ACM Richard Tapia Conference 2025
Presentation on accessibility AI research.
Interested in this research?
I'm always open to discussing collaborations, accessibility AI, or potential applications of this work.
Get in Touch