Jeffrey Quesnelle

Quesnelle’s work spans machine-learning research and software infrastructure. He earned an MS in computer science from the University of Michigan-Dearborn and an undergraduate degree in computer science and mathematics at Oakland University. His project history includes an OpenAI-compatible server for transformer models, visual podcasts generated with open models and the nds4droid emulator. He now leads technical work as Nous Research’s co-founder and CTO.[1]

Extending the context window

A language model’s context window limits how much text it can process at once. YaRN addresses models that use rotary position embeddings, which encode the positions of tokens but can behave poorly beyond the lengths seen in training. The method adapts those representations and fine-tunes the model to use longer sequences efficiently. The paper tests extended-context LLaMA models and reports reduced training requirements relative to earlier methods.[2]

Communicating less during training

Distributed training usually requires machines to exchange large gradient updates. DeMo separates local momentum updates and transmits a compressed selection of information using a transform and sparsification. Its error-feedback mechanism retains information that was not sent. The paper reports experiments with comparable loss and accuracy while moving substantially less data than its AdamW distributed baseline. Those results concern the evaluated model sizes and setups.[3]

Hermes instruction and reasoning models

Quesnelle co-authored Hermes4, an open model family combining deliberate reasoning with general instruction-following. The report details its data, training and evaluation.[4]