Fly.io's historical L40S price cut reflected what its customers were buying: the company said its less powerful A10 GPUs were more popular than its higher-end options. It priced the L40S from NVIDIA at $1.25 per hour, matching its A10 rate at the time. The explanation was workload fit: serving model responses also depends on memory, storage and networking, so the fastest available chip need not be the most useful choice.
The later GPU strategy reassessment adds context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition →. In that follow-up, the company said L40S customers were finding uses for the hardware, while GPUs had not become a driver of its core business.
Terms in this piece · Glossary
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
inference — Running a trained model to get answers — the phase where AI is actually used, as opposed to trained.
