Run a vLLM Server on HF Jobs in One Command
- Source
- huggingface.co
- Date
Context
Hugging Face's tutorial shows how to run vLLM, software that serves language models, as a GPU job and expose its listening port through the Jobs proxy. The resulting address lets a developer query the model remotely, from a laptop or notebook, without first assembling a separate server deployment.
Access still requires credentials: every request must carry a Hugging Face access with read permission for the job's namespace, and a plain browser visit is rejected. The tutorial positions this approach for tests, evaluations and batch generation; its managed Endpoints product adds finer-grained access control and scaling to zero during inactivity. A Jobs server is billed while it runs, so stopping it is part of the developer's work. Access and deployment guidance.
- token — The chunk of text a model reads and writes in — roughly three-quarters of a word — and the unit AI usage is billed in.
- inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
- AI agent — An AI system that doesn't just answer once but works toward a goal in a loop — taking actions, reading the results, and deciding what to do next.
Checking sign-in…
Loading comments…