s3-accelerator
github.com/danthegoodman1/s3-accelerator- Category
- Developer Tools
- Rank
- No. 1892Tools index
Previous survey · No. 1723 ·
- Type
- TOOL
- Use case
- Deployment & Operations
- Interfaces
- API
- GitHub
- 10 stars
- Date
About
s3-accelerator is an S3-compatible NVMe cache that serves hits in under a millisecond and replicates hot objects across the cluster. It uses rendezvous hashing to limit remapping when a node is lost, and asks the previous owner of a key for the object when nodes are added, so the cache stays warm instead of refetching from S3. It caches on the second hit, uses kTLS, splice and sendfile to keep chunk transfers out of userspace, is Parquet-aware, and can use SQS to refresh objects that are overwritten.
At a glance
s3-accelerator puts a per-zone NVMe read cache in front of an S3-compatible origin. It is worth benchmarking if repeated reads of a hot working set are a measured bottleneck.
Good fit
Infrastructure teams that can test object access, consistency and cache behavior under realistic load.
Consider something else when
Keep direct reads if access is mostly one-off or you cannot operate an extra credential and storage layer.
Start here
Use a disposable bucket and compare a cold read, a warm read and a read after replacing an object. Check the bytes and authorization before comparing latency.
Cost and requirements
Deployment needs cache hosts, local NVMe capacity and access to an S3-compatible origin. Check repository reuse terms and storage and network costs.
Limits to know
S3 remains the source of truth. Performance depends on hit rate, object size and deployment. Verify invalidation, credential boundaries and behavior when a cache node is unavailable.
Sources checked 2026-11-05. Source-based assessment.
Why it made the leaderboard
A sub-millisecond S3 cache that stays warm when the cluster resizes and avoids userspace copies. Useful for read-heavy data and agent workloads against object storage.
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.