Conditioning almost all stochastic processes on almost anything

Abstract

Sampling from a stochastic process only requires running it forward; outside a small collection of special cases, conditioning has remained the preserve of model-specific constructions. Once conjugacy breaks, we reach for Laplace, EP, variational bounds, or a bespoke sampler, with a fresh derivation for every new problem. Borrowing the architecture of diffusion models, I'll show a general solution for conditioning intractable stochastic processes. Stripped of its hype and branding, a diffusion model is simply a way to sample from an inconvenient distribution. Both sampling and conditioning require the score of the noised process, which in the ML literature, would be approximated with a neural network trained on a huge dataset. I'll show that if the forward model can be written as a deterministic map of simple noise, this score is available in closed form and we can do without the network; sampling and conditioning then reduce to solving a reverse SDE. After discretisation this covers essentially every stochastic process in use: Gaussian, Student-t and Cauchy fields; SDEs and SPDEs; Potts models; or whatever your simulator already produces. Examples are given for conditioning on observed data, censoring, shape constraints, differential-equation constraints, rare events, and (subject to some hand-waving) natural language.