One pretrained model for score and density, no retraining per dataset
- Source
- x.com
- Date
AI image generators don't "draw"—they follow a compass: the score function, which points toward more probable images. The same compass drives Bayesian sampling and plasma physics. We built DiScoFormer to estimate the score far better when data gets complex. 🧵
Recovering a distribution means estimating its density (where data clusters) + the score. Kernel density estimation (KDE) does both, but when data has many dimensions, it breaks down. Neural score-matching models stay more accurate but need to be retrained per dataset.
DiScoFormer is a single model that gives both density & score on any dataset with no retraining. With 100 numbers per point, it cuts error vs. the best-tuned KDE ~6.5× on score and 37× on density.
DiScoFormer can also handle data unlike anything it trained on—more clusters than it ever saw, and unfamiliar shapes.
Context
DiScoFormer estimates the pattern behind a cloud of numerical observations. It predicts both where observations are concentrated, called density, and which direction leads toward more probable values, called the score. Ai2 describes a trained model that accepts sample points and produces these estimates in a single pass, without fitting a new neural network to each sample. Method overview.
Its training uses synthetic distributions whose correct answers are known. In one test on random two-component Gaussian mixtures with 100 numerical dimensions per point, score-estimation error was about 6.5 times lower than the best tested kernel-density baseline. That supports using learned estimators for difficult distribution-estimation problems; the measurement concerns statistical estimates on synthetic data, rather than improvements to a finished image generator. Paper, Table 5.
Checking sign-in…
Loading comments…