FloorPlan2Guide: LLM-Based Indoor Navigation System
AI-Powered Navigation for Individuals with Blindness or Low Vision
Overview
FloorPlan2Guide is a novel AI-powered indoor navigation system that leverages GPT-4 Vision and floor plan understanding to provide real-time, context-aware navigation assistance for visually impaired individuals.
The system integrates multimodal inputs including camera feeds, floor plans, and natural language to generate precise turn-by-turn directions. By combining state-of-the-art vision-language models with spatial reasoning capabilities, FloorPlan2Guide offers an unprecedented level of independence for users navigating unfamiliar indoor environments.
Key Features
GPT-4 Vision Integration
Utilizes advanced vision-language models for real-time scene understanding and spatial reasoning.
Floor Plan Understanding
Processes architectural floor plans to build accurate spatial models of indoor environments.
Accessibility-First Design
Built from the ground up with input from visually impaired users to ensure usability.
Voice-Guided Navigation
Natural language turn-by-turn directions with context-aware announcements.
Technical Approach
1. Floor Plan Processing
The system processes CAD floor plans and building blueprints to extract navigable paths, room locations, and key landmarks. This creates a graph-based representation of the indoor environment.
2. Real-time Scene Analysis
Using GPT-4 Vision, the system continuously analyzes camera input to identify obstacles, doors, signs, and other navigational cues in real-time.
3. Multimodal Fusion
The system fuses information from the floor plan model, real-time camera analysis, and user queries to generate contextually appropriate navigation instructions.
4. Natural Language Generation
Instructions are generated in natural, conversational language that is clear and easy to follow for users with visual impairments.
System Architecture
System Architecture Diagram
View PDFFloorPlan2Guide system architecture showing floor plan parsing and navigation generation pipeline.
Technologies Used
Publications & Presentations
STARS Celebration Conference 2024
Poster presentation on LLM-based indoor navigation for accessibility.
View PosterCMD-IT/ACM Richard Tapia Conference 2025
Upcoming presentation on accessibility AI research.
Target: NeurIPS 2025 / AAAI 2026
Full paper submission in preparation.
Interested in this research?
I'm always open to discussing collaborations, accessibility AI, or potential applications of this work.
Get in Touch