EleutherAI
1 Intel
EleutherAI is a non-profit AI research institute that began in 2020 as a Discord server formed to create an open-source version of GPT-3, and formally incorporated as a non-profit research institute in early 2023. It trains and releases open models, datasets and tools, including the GPT-NeoX training framework, the Pythia model suite and the Common Pile corpus, and studies how models learn, how to evaluate them and how to make open-weight systems safe. Stella Biderman is its executive director. The directory credits it with lm-evaluation-harness, its framework for few-shot evaluation of language models.
Intel
lm-evaluation-harness v0.4.12 adds TensorRT-LLM and Megatron-LM backends, tensor-parallel support for transformers models, new benchmarks, a TaskManager refactor and a long tail of task correctness fixes.