Yi Tay

An NTU-hosted author biography records Tay’s B.E. in Computer Science from the university in 2015 and identifies him as a computer-science PhD student. In his later interview he describes completing a PhD in Singapore and developing an interest in research during that period.[4][5]

Tay’s personal site dates his Google Brain research to 2019–2023, focused on pretraining and scaling. He then spent 2023–2024 as a startup co-founder and chief scientist before returning to Google. In his current account, he works on Gemini post-training and recursive self-improvement, leads the Singapore group and helped establish Google DeepMind Singapore with Quoc Le in 2025. He also describes contributing to the Deep Think system that achieved an IMO gold-medal-level result in 2025.[1]

Combining language-learning objectives

Tay is the first-listed author of UL2, which examines how a model’s architecture and its training objective affect performance across tasks. Its mixture-of-denoisers objective combines different ways of reconstructing or completing text, instead of committing to only one pretraining pattern. The paper studies those choices with controlled comparisons and releases model checkpoints. This gives researchers a concrete way to test how the training task changes what a language model can do later.[2]

Retrieval by generating an identifier

He also co-authored the Differentiable Search Index paper. Instead of storing document representations in a separate retrieval index and comparing a query against them, the proposed model learns to map query text directly to relevant document identifiers. Information about the corpus is encoded in its parameters. The experiments study document representation, training procedure and corpus size, making this an alternative retrieval architecture with specific tested conditions rather than a universal replacement for search indexes.[3]