Spinf-serve
github.com/spinfinc/spinf-serve- Category
- Developer Tools
- Rank
- No. 1598Tools index
- Pricing
- Open Source
- Type
- TOOL
- GitHub
- 12 stars
- Latest release
- 0.1.2
- Date
About
Spinf-serve is an inference server for bulk classification on a single 24 GB NVIDIA GPU. It reads each piece of content once and scores many questions about it in the same pass, returning a probability for each option rather than generated text. It exposes an OpenAI-compatible API, is licensed Apache 2.0, and is built for throughput rather than single-request latency.
Tags
inferenceclassificationgpuopenai-compatiblellmbatch
Tech Stack
PythonDocker
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.