OpenAI Knowledge Retrieval Starter

Why it's ranked here

This is a strong reference implementation because it connects configuration, ingestion, retrieval, cited synthesis, and evaluation instead of stopping at a search demo. The included Qdrant path and plugin protocol provide an escape hatch from managed storage. Some rough edges, particularly inconsistent backend ports and partially implemented store choices, keep it from feeling turnkey.

What's good

YAML controls chunking, retrieval, synthesis, prompts, models, filters, and evaluation without routine code edits. Chunking includes recursive, heading-aware, hybrid, XML-aware, and custom strategies. Retrieval can add query expansion, hypothetical-document embeddings, similarity filtering, and reranking. Evaluations accept curated data or generate questions from the corpus, then produce local reports and optionally mirror results to OpenAI Evals.

Tradeoffs

The backend still requires an OpenAI key for embeddings and answer generation, including the Qdrant route. The optional web interface adds a separate Node.js toolchain and process. Configuration recognizes several custom store kinds, but the bundled factory implements only Qdrant and dynamically loaded plugins. Skipping ingestion preflight can create duplicate uploads.

How to use it well

Use it when an engineering team wants a configurable baseline for testing retrieval choices against a local document corpus. Start with managed File Search, establish cited answers and evaluation data, then switch to Qdrant or a plugin when storage control matters. It does not provide managed document connectors, deployment automation, or a complete production operations layer.

Observed

License

MIT

Primary language

Python 3, requiring Python 3.10 or later

Packaging

Setuptools project installable with pip in editable mode; development extras include pytest, Ruff, and coverage support

Interfaces

Typer command-line interface, FastAPI backend API, and Vite plus React web interface

Storage backends

OpenAI File Search and Qdrant are bundled; custom stores can be loaded as Python plugins

Platform support

Package metadata declares operating-system independence

Optional UI requirement

Node.js 18.18 or later is required for the web interface

Evaluation surface

Supports curated or synthesized JSONL datasets, local grading, Markdown and HTML reports, and optional OpenAI Evals mirroring