Vibeleaderboard
Index / tool
Visit github.com
Category
Developer Tools
Rank
No. 1254Tools index
Pricing
Open Source
Type
TOOL
GitHub
146 stars
Latest release
0.7.0
Date

About

VennDB is an append-only in-memory Rust database for rows queried with bit (flag) columns. It suits mostly static data loaded at startup and filtered constantly with simple conditions, and it has a derive macro that generates the API for a struct. The README says it is faster than a naive implementation or SQLite for this narrow use case.

What it does

VennDB turns a plain Rust struct into a purpose-built lookup table that lives in memory. You mark which fields are unique keys and which are filters, and a code generator writes a matching database type with a typed query builder: chain conditions such as admin true, manager false, department engineering, then get every match or one random match. True-or-false fields become bitmaps, so a query is mostly bitwise AND work. Rows can be added but not edited in place. The project's own FAQ calls it "not a database" and says not to replace a usual database with it.

Why it's ranked here

For one narrow job, the numbers are hard to argue with. On the author's benchmark with a million rows, three true-or-false filters took a median 192 nanoseconds, against 13.79 microseconds for a hand-written roaring bitmap index and 42.23 milliseconds for in-memory SQLite. Building that table took 121 milliseconds against SQLite's 3.8 seconds. The core crate pulls in one hashing library and forbids unsafe code outright, and it is dual licensed MIT or Apache 2.0. It came out of the rama proxy project, where it backs an in-memory upstream proxy database, so the design reflects a real workload rather than a demo.

What's good

The query planner makes a sensible choice per query: when one filter matches very few rows it walks those rows and checks the rest, otherwise it combines whole bitmasks. Asking for a single random match often skips the full query entirely by sampling random rows first, which suits load balancing. Keys are checked before anything is written, so a duplicate key rejects the row without leaving half of it indexed. Filters can be optional, can mark a row as matching any value through a wildcard variant, and can hold a set of values. An optional validator function can refuse bad rows at insert time.

Tradeoffs

This is a library you compile into one Rust program, not a server, so other processes and languages cannot query it. Data lives only in memory and is rebuilt from your source at startup. Rows cannot be updated or removed in place, which rules it out for anything that changes often. Queries are equality filters only: no ranges, sorting or joins. A table tops out at about four billion rows. The benchmarks come from one laptop run by the author, who writes that they are "not database nor hardware experts". At a million rows the hand-written roaring index used slightly less index memory than VennDB.

How to use it well

Reach for it when a Rust service loads a mostly fixed list at startup and answers many small yes-or-no questions about it per second: proxy pools, feature flag targets, routing tables, device inventories. Prefer enums over free-text strings for filter fields, as the FAQ advises, because "BE" and "Belgium" hash differently. Add it with cargo, derive it on your struct, then build the table once from an iterator; a duplicate key comes back as an error you must handle. When the data changes, rebuild the whole table. If you need persistence, ad hoc queries or writes from many places, use SQLite or a real database instead.

Technical notes+

src/lib.rs declares #![forbid(unsafe_code)], re-exports the Any and VennDB derives from venndb-macros behind the default derive feature, and defines the Any trait (blanket impls for &T, Option, Arc, Rc, Box) plus FilterSet, implemented for HashSet and BTreeSet and deliberately not for Vec (a compile_fail doctest enforces this). venndb-macros/src/generate_db.rs emits a <Name>DB struct holding rows: Vec<T> plus one KeyIndex, BoolColumn or ValueColumn per field; from_rows sizes only key indexes, appends each row, then calls shrink_to_fit; append checks every key for duplicates before committing any column, and an optional validator path runs first. The generated column(predicate) scans all rows and excludes rows appended later. src/plan.rs picks a driver set when its walk cost times DRIVE_RATIO (256) is at most the row count, else ANDs dense bitmasks; random() first tries up to min(rows/1024, 1024) random rows when the estimated match share times that count is at least 4.0, falling back to execute(). Plan::new asserts rows <= u32::MAX. Cargo.toml: edition 2024, rust-version 1.96.0, sole runtime dependency foldhash plus optional venndb-macros; workspace includes a fuzz crate; roaring and sqlite are dev-dependencies for the bench. examples/usage.rs mirrors the README and runs as a test.

Observed

License
MIT OR Apache-2.0 (Cargo.toml, LICENSE-MIT)
Language
Rust, edition 2024
Install surface
Rust crate, added with cargo add venndb; derive macro on by default
Interface
Library only: generated Rust types and query builder, no server, CLI or network API
Runtime dependencies
foldhash, plus the optional venndb-macros crate
Memory safety
Crate root forbids unsafe code (src/lib.rs)
Storage model
In-memory, append-only; no update or delete in the generated API that was read
Capacity limit
At most u32::MAX rows per database (src/plan.rs)
Testing
Unit tests in the crate root, an example that runs as a test, and a fuzz crate in the workspace
Benchmark setup
Author-run on a single Apple M4 Max laptop against roaring, SQLite and a naive Vec

Read from README.md, Cargo.toml, LICENSE-MIT, src/lib.rs, venndb-macros/src/generate_db.rs, examples/usage.rs, src/plan.rs.

Tags

rustdatabasein-memorybitsetderive-macrofiltering

Tech Stack

Rust

Media

VennDB

Comments (0)

No comments yet

Editorially curated, with community endorsements as a secondary signal. Corrections welcome.