Arcee Ai Intel Gaudi2 Training A Financial Insights Model

This January 2025 post from Arcee AI is a training recipe rather than a model launch. It shows how Arcee trained two existing open models, Llama 3 8B and Qwen2 7B, on Intel's Habana Gaudi2 accelerators to answer questions about financial documents (Arcee blog). The training data was 10 billion tokens of SEC filings and other financial text, combining Arcee's own SEC dataset with open-source financial datasets. The software was optimum-habana, which adapts Hugging Face's transformers library to Gaudi hardware, together with DeepSpeed ZeRO-3, which Arcee used for memory efficiency when training across multiple machines. The Llama training run used eight Gaudi2 accelerators.

Most of the post is configuration. Arcee set training and evaluation batch sizes of 8 and 4 to fit Gaudi2's memory, turned on gradient checkpointing to reduce memory use, and set Gaudi-specific options, including lazy mode, which defers computation until it is needed. The setup targets the Gaudi2 1.17.1 Docker images, and Arcee says the recipe works for both models.

The one measured result concerns serving, not training. Arcee reports that a single Gaudi2 generates about 49 tokens per second with the 8B model, and the same with the 7B model, and calls that comparable to an AWS g6e.2xlarge instance at about half the price. The post gives no throughput or price figures for the AWS instance, no training times and no benchmark scores for the finance models, and its statement that the model showed 'strong accuracy' on financial statements is unquantified. Arcee also describes Intel as a partner, so the cost comparison is the company's own account. The post supports the conclusion that this training workflow runs on Gaudi2. It does not show that the resulting models outperform alternatives.