Data+: AI-Powered Hazard Detection for the Visually Impaired

The 2026 Visual Language Models for Visual Assistance Data+ team led by ECE professor Henry Pfister’s research group is designing and implementing computer vision models that provide real-time audio guidance to the visually impaired. This ranges from hazard alerts, where the models flag obstacles and fast-approaching objects in indoor and outdoor environments, to helping users with more detailed descriptions of scenes to locate specific objects, read signs and mail, and better understand their surroundings. The team’s implementation runs directly on the user’s own phone, paired with a set of camera-equipped smart glasses that stream what the user is looking at to the model.
The team’s work is based on current gaps in assistive technology. While traditional tools like canes and guide dogs remain heavily trusted as navigation aids, head-level injuries from missed obstacles remain common, happening as often as once a month, which can cause harm and lose trust in navigating more crowded spaces. Meanwhile, current mainstream camera-based devices are emerging to act like the user’s eyes, with products like Meta’s Ray-Ban smart glasses showing that this kind of technology can be affordable, stylish, and widely adopted. At the same time, recent advances in vision-language models are opening new possibilities for applying this technology specifically to walking guidance by providing more intelligent responses with models small enough to fit on consumer edge devices. This project sits at the intersection of these three trends, and they want to help advance research in this area.
For an assistive system, a smaller specialized model that responds immediately and works without an internet connection may be more valuable than a larger cloud model that responds several seconds later. “We learned that bigger is not always better. When every second matters, a smaller specialized model running close to the user can be more useful than a much larger model in the cloud.” Says Ziv Ahroni, project manager for the team.
Currently, they have compared their baseline rule-based hazard alerting against a new, compressed 0.5B model distilled from a larger teacher model, designed specifically to run on a phone. This lets them bring learned, context-aware hazard reasoning to lightweight everyday mobile hardware, instead of depending on a heavier cloud-hosted model. Their next step is exporting this model to run on-device. Once this is done, users can receive smarter, more reliable hazard alerts with no dependence on an internet connection or an external server.
We will update you when we hear what the team has done next, in the meantime you can see a demonstration of the tool they have developed here: