humans& open-sources a 4-bit hardware-native RL training recipe
- Source
- x.com
- Date
At humans&, we train models from the long-term impacts of their interactions with people. This requires prioritizing long-horizon multi-agent RL. We've developed and are excited to share an open-source, hardware-native 4-bit RL recipe, significantly accelerating training
This recipe resolves instabilities due to policy error (forward) and gradient mismatch (backward) - we share details here: https://t.co/LxUGOabUSd This work was led by @zianglih and big thanks our collaborators at @radixark and @nvidia - without them this would not be possible!
We have a couple surfaces for our models cooking and have started sharing them with a small set of folks - if you're interested in trying them, either individually or as a company, reach out and we'll follow up as soon as we open up more broadly
Context
humans& says it trains models from the long-term impacts of their interactions with people, which it says requires prioritizing long-horizon reinforcement learning (RL), where a model learns from feedback over extended interactions. In a post on X it announced an open-source, hardware-native 4-bit RL recipe that it says significantly accelerates training.
The company says the recipe resolves instabilities due to policy error (forward) and gradient mismatch (backward), and links a write-up of the details. It credits the work to a team member and thanks collaborators at Radix Ark and NVIDIA. It adds that it has started sharing early model surfaces with a small set of people who reach out. The post gives no speedup figure, so the acceleration is the company's claim.
- multi-agent — Using several AI agents on one problem — splitting work in parallel, checking each other, or filling different roles like planner and reviewer.
Checking sign-in…
Loading comments…