Poolside runs the model instances that attempt tasks separately from those being trained, allowing the two workloads to scale independently. That creates a synchronization problem: task attempts should use sufficiently recent model parameters. Within its Model Factory, the company addresses this with direct GPU-to-GPU transfers. Its system maps parameter pieces between potentially different GPU arrangements and transfers updates asynchronously, so training can continue during delivery. This reduces dependence on saving and reloading checkpoints through intermediate storage when refreshing the models that generate training experience.
Terms in this piece · Glossary
pretraining — The first, biggest phase of building a model: training it on enormous amounts of text so it learns language, facts, and reasoning in general.
