A potent family of generative models that can depict complicated distributions over high-dimensional spaces is score-based generative models (SBGMs), which include diffusion models. The development of a source density, almost always Gaussian, is commonly simulated using SBGMs using a stochastic differential equation (SDE) to produce samples. SBGMs are constrained by their assumption of a Gaussian source, which is necessary for optimization with the simulation-free denoising goal, notwithstanding their empirical success. The use of SBGMs for understanding the underlying dynamics is prohibited since this assumption is frequently broken in the temporal development of physical or biological systems, such as in the case of single-cell gene expression data.
Continuous normalizing flows (CNFs), also known as flow-based generative models, have been the method of choice for solving these issues. The source density is transformed to the target density using an ordinary differential equation (ODE), which is fitted in flow-based models on the assumption of a deterministic continuous-time generating process. Previous work introduced simulation-free training objectives that make CNFs competitive with SBGMs when a Gaussian source is assumed, and these objectives were extended to the case of arbitrary source distributions. Flow-based models were previously constrained by inefficient simulation-based training objectives that demand an expensive integration of the ODE at training time.
However, these aims still need to cover learning stochastic dynamics, which might be useful for both generative modeling and regaining the dynamics of real systems. The Schrödinger bridge problem (SB) considers the most probable development between a source and target probability distributions under a certain reference process. It is the basic probabilistic formulation of stochastic mapping between two arbitrary distributions. Modeling natural stochastic dynamical systems, mean field games, and generative modeling are just a few of the issues for which the SB problem has been used. The SB issue normally lacks a closed-form solution, except for several specific situations (such as Gaussian). Still, it may be approximated using iterative techniques that call for replicating the learned stochastic process.
Although theoretically valid, these approaches have numerical and practical problems that only allow for high-dimensional scaling. Researchers from Mila Québec AI Institute, Université de Montréal, McGill University, University of Toronto and Vector Institute study the simulation-free score and flow matching (2M) goal for the Schrödinger bridge issue. The simulation-free objectives for CNFs and the denoising training target for diffusion models are concurrently generalized by 2M to stochastic dynamics and arbitrary source distributions, respectively. In their approach, the Schrödinger bridge is defined as the Markovization of a collection of Brownian bridges using a relationship between the SB issue and entropic optimum transport (OT).
2M can benefit from static entropic OT mappings between source and target distributions, which are effectively approximated by the Sinkhorn method or stochastic algorithms instead of dynamic SB approaches, which require simulating an SDE on each iteration. They use simulated and real-world datasets to show the utility of 2M. On artificial data, they demonstrate that 2M outperforms comparable earlier work in terms of generative modeling metrics and discovers a more accurate approximation to the real Schrödinger bridge. They investigate modeling cross-sectional measurement sequences (i.e., unpaired time series observations) by a succession of Schrödinger bridges as an application to actual data.
Even though there have been several previous approaches to modeling cells with Schrödinger bridges in static or low-dimensional dynamic settings, 2M is the first approach that can scale to thousands of gene dimensions since its training requires no simulation. They also provide a static manifold geodesic map, illustrating one of the earliest real-world uses of Schrödinger bridge approximations with non-Euclidean costs, which enhances cell interpolations in the dynamic environment. Finally, they demonstrate that, in contrast to the static optimum transport example, they can directly model and reconstruct the gene-gene interaction network that controls the dynamics of the cell. Code and examples are available on GitHub.
Check out the Paper and Github link. Don’t forget to join our 26k+ ML SubReddit, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more. If you have any questions regarding the above article or if we missed anything, feel free to email us at Asif@marktechpost.com
Check Out 800+ AI Tools in AI Tools Club
The post This AI Paper Introduces a Novel Class of Simulation-Free Objectives for Learning Continuous-Time Stochastic Generative Models between General Source and Target Distributions appeared first on MarkTechPost.
A potent family of generative models that can depict complicated distributions over high-dimensional spaces is score-based generative models (SBGMs), which include diffusion models. The development of a source density, almost always Gaussian, is commonly simulated using SBGMs using a stochastic differential equation (SDE) to produce samples. SBGMs are constrained by their assumption of a Gaussian
The post This AI Paper Introduces a Novel Class of Simulation-Free Objectives for Learning Continuous-Time Stochastic Generative Models between General Source and Target Distributions appeared first on MarkTechPost. Read More AI Shorts, Artificial Intelligence, Editors Pick, Machine Learning, Staff, Tech News, Technology, Uncategorized