MTIA 300: Meta’s First Training Chip with Built-in NICs and Communication-Offloading Engines
Source
engineering.fb.com
Date
Why it matters
Meta's MTIA 300 puts NIC chiplets on the accelerator itself and co-designs the collective library with the silicon, attacking the AllReduce/AllToAll bottleneck that leaves general-purpose GPUs underutilized on embeddingA list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.Full definition →-heavy training.
Terms in this piece · Glossary
embedding — A list of numbers representing a piece of text's meaning, so that similar meanings end up numerically close and can be searched.
