Patrick Esser is a co-founder of Black Forest Labs and a computer-vision researcher. He co-authored Taming Transformers and Latent Diffusion Models, and helped develop the rectified-flow transformer approach used in high-resolution image generation.[1][2][3][4]
Start with The researcher-to-founder journey →
GitHub 
Professional X account Career
Contributions
Learn an image vocabulary before modeling its composition
In Taming Transformers, Esser and Robin Rombach share first authorship with Björn Ommer as co-author. The approach uses a convolutional network to learn a compact vocabulary of image constituents and a transformer to model their composition. Combining local visual structure with long-range interactions makes high-resolution synthesis more manageable than treating every pixel as an independent sequence element.[2]
Move diffusion into a compressed space
The latent-diffusion paper applies the denoising process to a pretrained autoencoder’s representation. Compression reduces computation while preserving useful detail. Cross-attention lets the model incorporate inputs such as text or bounding boxes. Esser is one of five co-authors; the paper lists his affiliation as Runway ML.[3]
Study how a generative architecture scales
The rectified-flow transformer paper connects data and noise along straight paths and adjusts training-time noise sampling toward perceptually relevant scales. Its architecture gives image and text modalities separate weights while allowing information to flow in both directions. Esser and Rombach share the paper’s equal-contribution designation. The study examines scaling, typography and correspondence between validation loss and image quality.[4]
Tools, writing & talks
Writing, research & talks
Start here
A discussion with the BFL founders about model development, openness and building a research company.
Begin with the founders’ discussion of why they release models and how research became a product. It requires no technical setup; the papers provide a more mathematical follow-up.
Participant[1]- Taming Transformersarxiv.org
Research combining a learned image codebook with transformers for high-resolution synthesis.
Equal-contribution co-author with Robin Rombach[2] - Latent Diffusion Modelsarxiv.org
Research moving diffusion into a compressed autoencoder representation with flexible conditioning.
Co-author[3] - Scaling Rectified Flow Transformersarxiv.org
A study of rectified-flow training and multimodal transformer scaling for image synthesis.
Equal-contribution co-author with Robin Rombach[4]
