Unlocking the potential of vision language models on satellite imagery through fine-tuning
Source
mistral.ai
Date
Context
Training on labeled aerial images substantially improved Pixtral-12B's scene classification in Mistral AI's reported experiment. The team used 8,000 training images and 2,000 test images covering 30 categories. It first prompted the unmodified model with the permitted labels, then fine-tuned it using images paired with correct answers. Reported accuracy rose from 56% to 91%. The case study demonstrates the value of teaching a general vision model a dataset's distinctions, while acknowledging that smaller specialist vision models could achieve comparable results.
Terms in this piece · Glossary
LoRA — A cheap way to fine-tune a model by training a small add-on layer instead of changing all of the model's weights.
