Semantic Image Search
https://github.com/vercel-labs/semantic-image-search- Category
- AI Tools
- Rank
- No. 1426Tools index
Previous survey · No. 1418 ·
- Pricing
- Open Source
- Type
- TOOL
- Builder
- vercel-labs
- GitHub
- 230 stars
- Date
About
Search images by what they show, not just filenames — reference build using embeddings and the Vercel AI SDK.
What it does
It turns a collection of JPG images into a searchable web gallery. A vision model writes a title and description for each image, then the app embeds that text in Postgres. Searches combine literal title and description matches with cosine similarity, deduplicate results by title, and cache completed queries.
Why it's ranked here
This is a useful reference implementation because it exposes the entire ingestion and retrieval path, not merely a search interface. The schema, indexing scripts, vector query, keyword fallback, caching, and streaming web stack are all present. Its narrow setup and Vercel service dependencies make it better for learning and adaptation than immediate reuse.
What's good
The retrieval design combines exact text matching with semantic similarity, so obvious keyword hits are not sacrificed. An HNSW cosine index supports vector lookup, while stored results omit embeddings from returned image records. Runtime validation protects database inserts, and the three-stage indexing process makes failures easier to isolate.
Tradeoffs
Indexing is an offline, sequential process limited to JPG files, and every image receives generated metadata before embedding. Metadata generation and embeddings are hardwired to OpenAI in the supplied code despite broader provider claims. Setup requires Blob storage, KV, Postgres, pgvector, numerous credentials, and a manually tuned similarity threshold.
How to use it well
Use it when prototyping a private image catalogue or studying a complete semantic retrieval pipeline in a modern React application. Replace the sample storage hostname, tune similarity against your collection, and automate the indexing scripts for repeated imports. It does not provide a general image-search library, hosted API, or broad media ingestion system.
Technical notes+
lib/ai/0-upload.ts, lib/ai/1-generate-metadata.ts, and lib/ai/2-embed-and-save.ts implement upload, GPT-4o metadata generation, and embedding persistence as separate batch scripts. lib/ai/utils.ts fixes embeddings to text-embedding-3-small; lib/db/schema.ts stores 1,536-dimensional vectors with an HNSW cosine index. lib/db/api.ts merges ILIKE matches with vector results above 0.28, limits semantic results to 10, deduplicates by title, and caches queries in Vercel KV. next.config.mjs permits images from one fixed Vercel Blob hostname, so cloned deployments must update that configuration.
Observed
- Primary language
- TypeScript and TSX
- Package surface
- pnpm-installed Next.js application template with development, build, database, and indexing scripts
- Interfaces
- Web application plus command-line batch scripts for image ingestion and indexing
- Storage stack
- Vercel Blob for images, Vercel KV for query caching, and Vercel Postgres with pgvector
- Search methods
- Case-insensitive title and description matching combined with cosine vector similarity
- Deployment surface
- One-click Vercel deployment and local Next.js development server
Read from README.md, package.json, lib/utils.ts, lib/db/api.ts, lib/db/index.ts, lib/ai/utils.ts, lib/db/schema.ts, lib/ai/0-upload.ts, lib/ai/2-embed-and-save.ts, lib/ai/1-generate-metadata.ts, lib/hooks/use-shared-transition.tsx, next.config.mjs, drizzle.config.ts, postcss.config.mjs, tailwind.config.ts.
What it can do
Search images by semantic content
Natural language query describing image content → Relevant images matching the semantic description
Generate image embeddings
Image files → Vector embeddings representing image content
Index images for semantic search
Collection of images → Searchable image database with embeddings
Find visually similar images
Reference image or image description → List of semantically similar images
Search images beyond filename matching
Descriptive text query about image content → Images containing the described visual elements
Tags
Tech Stack
Media
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.