Exploring Diffusion Transformer Designs Via Grafting

Liquid AI and collaborators describe a way to test changes inside an already-trained image generator without repeating its full original training. Their grafting method first trains a replacement component to imitate the outputs of the component it will replace. It then briefly trains the modified model to reduce errors that accumulate when several replacements work together. The research article reports experiments replacing components and reorganizing layers in diffusion transformers, and says one image-generation experiment used less than 2% of the original model's pretraining compute. This gives researchers a way to explore architectural alternatives using an existing model as the starting point. The article is a summary of a paper, so its quality results were not examined here.