Arcee.ai

Arcee and the Department of Energy will build Genesis-Science-1, a trillion-parameter-class open-weight model on the next Trinity base, paired with a governed execution system for long scientific tasks. Weights and a technical report are promised later this year.

nac 0.1.3 groups sessions under projects, isolates each one in its own sandboxed git worktree, adds skill prompt expansion with composer autocomplete, reads images, and renders code and diffs with Shiki highlighting.

A working MergeKit walkthrough using Llama-3 8B base and instruct as the subjects: authoring the merge YAML, running the merge on a GPU box, generating a model card, and pushing the result, with the GUI path as an alternative.

Arcee is shutting the hosted Trinity-Large-Preview endpoint on OpenRouter after roughly 3.3T tokens served since January, pointing active pipelines at Trinity-Large-Thinking. The Preview weights stay on Hugging Face for local deployment.

MergeKit gains evolutionary merging, an open implementation of the method Sakana published but never released. The tutorial covers GPU requirements, defining custom lm-eval scoring tasks instead of optimising against leaderboards, and running the search.

DistillKit's technical paper covers two distillation routes: KL divergence over teacher logits, and intermediate hidden-state alignment. It details the constraints of each and the SFT-based experiments Arcee ran to compare them against vanilla training.

Three lessons from published MergeKit use cases, led by Pre-trained Model Average: averaging checkpoints from the stable phase of pre-training improves performance, with merge interval and model count both mattering. Arcee used the method building AFM-4.5B.

Arcee published pre-anneal base checkpoints for its Trinity Nano and Mini models. They retain the math and code exposure of the released bases but sit earlier in the training schedule, which the team says makes continued pretraining and customization easier than starting from post-anneal weights.

MergeKit can decompose the difference between a fine-tune and its base model into a PEFT-compatible low-rank adapter. On OpenHermes 2.5, a rank-8 extraction holding 0.46 percent of the parameters kept a 9.5 percent average gain over base Mistral; rank 512 reached 90 percent of full performance.

Arcee Agent is a 7B open model built on Qwen2 for one job: interpreting and chaining tool calls. It speaks several tool-use formats and can sit as a router in front of bigger models, with the tradeoff of weaker general capability.

Adapting a base model to a specialist domain usually costs you its general ability. This case study pairs continual pre-training with model merging to keep both, applied to medical and patent corpora, with citations to prior domain-adapted models and the cost argument behind the approach.

AFM-4.5B is Arcee's first from-scratch foundation model, trained on 6.58 trillion tokens curated with DatologyAI. The post argues the 3–10B band had gone stale and details why data curation, not architecture, was the hard part.

Arcee is pulling the hosted Trinity Mini checkpoint from OpenRouter and reallocating that serving capacity toward the next Trinity generation. Existing workflows must switch models; the Mini weights stay on Hugging Face for local deployment.

Arcee-SuperNova is a 70B model assembled from three separately trained variants: a distillation of Llama-3.1-405B-Instruct into a 70B, an instruction-tuned model built on synthetic data from the EvolKit pipeline, and a DPO-aligned version given less weight in the final merge.

Arcee launched the Trinity family: Mini as a post-trained reasoning MoE and Nano Preview as an experimental high-sparsity chat model, with Trinity Large in training on 2048 B300 GPUs. The post explains why the team moved from post-training others' checkpoints to owning pretraining.

Arcee ran continual pre-training on Mistral-7B-Instruct over 2.5B tokens of SEC filings twice, once full and once with QLoRA. Perplexity landed at 2.72 for standard CPT against 3.30 for QLoRA and 3.96 for the base, evidence that low-rank updates cannot absorb substantial new knowledge.

Arcee announced that Trinity-Large-Thinking will be retired from its endpoint on October 26. DeepSeek, Kimi and GLM models stay available, and the lab says a successor is in preparation. Whether the open weights remain published was not addressed.

Arcee-Meraj adapts the Qwen2-72B-based Nova model to Arabic through a curated bilingual data mixture, batch translation of English sets, task-specific cleaning, self-curation and iterative training with DPO. Arcee reports it leading the Open Arabic LLM Leaderboard.

Arcee distilled DeepSeek-V3 into a 10B Falcon-based model and a 32B, both Apache-2.0. Virtuoso-Lite trained on the V3 tokenizer to absorb 1.1B tokens of logits from an fp8 V3, then reverted to a Llama-3 tokenizer and merged. The 32B outscores their earlier 72B Nova.

Trinity-Large-Thinking ships under Apache 2.0 with open weights, targeting the failure modes that break long agent runs: context coherence across turns, tool calling that stays disciplined, and instruction following under constraint. Its instruct predecessor stays free on OpenRouter.

A postmortem of Meta's Llama 4 launch: the Maverick checkpoint submitted to LM Arena differed from the public weights, Scout's advertised 10M-token context crashed and diverged between GPU and CPU runs, and denials landed alongside an admitted chat-optimized submission.

A walkthrough of converting AFM-4.5B into a hybrid Kimi Delta Attention and global attention model through knowledge distillation, covering student initialisation, dropping RoPE without replacement, and a modified RADLADS pipeline ending in long-context finetuning.

Arcee reversed the planned Preview sunset: Trinity-Large-Preview stays live at $0.15 in and $0.45 out per million tokens, while Trinity-Large-Thinking falls from $0.90 to $0.80 per million. Both are reachable through OpenRouter or Arcee directly.

A training report on distilling a 405B teacher into 70B and 8B students: why live distillation was infeasible, how compressing extracted logits cut the dataset from petabytes to 50GB, the FSDP training configuration used, and how a parallel SNR-targeted run was merged in.