Picture This: Open Source AI for Image Description

An image-description service can let a blind user ask about details that an initial caption leaves out. A blind developer at Fly.io built a self-hosted prototype using Ollama to run the LLaVA vision model and PocketBase to manage uploads and conversation history. Each follow-up sends the image, earlier conversation and new question back to the model, preserving the context of what the user wants to understand. The project account explains that interaction.

The author found the generated scene description useful, while the model could not identify the pictured tree species. The prototype illustrates an accessibility benefit from conversational descriptions, with detail still limited by what the model can recognize.