# Full Text: Catechism for: "Towards Active Diffusion: A Tale of Multiple (den)Cities"

> Extracted from `2022_ActiveDiffusion.pdf`

---

## Page 1

Active Diffusion Catechism
Mission
The Mission of 2023-AD is to characterize mathematical formalisms and computational applications of Active 
Inference and Diffusion Models, and to explore relevant applications.
Broadly, we are interested in a number of research threads that have impacted the development of applied Active 
Inference models, such as: 
, 
, 
 (HGF), 
 (FFG), 
 (PP), 
, and other topics that might become 
relevant over the course of our research. 
Bayesian Graphs Bayesian Physics Hierarchical Gaussian Filter
Forney Factor Graphs
Predictive Processing
entropy production
We will combine both theoretical and applied approaches to elucidate the connection between Diffusion Models 
(DM) used in contemporary Artificial Intelligence research and Active Inference (ActInf) generative models from the 
cognitive sciences. Specifically, we are interested in exploring how DMs can be used to represent internal beliefs 
about an agent’s environment and their role in decision making and planning under the Free Energy Principle — such 
that outputs can actually be strategic, rather than stochastically parroting the syntax of strategy. 
Situation
This initiative will elucidate and develop connections among Active Inference (ActInf), Diffusion Models (DM), Large 
Language Models (LLM), and related topics. 
⁠
 (ActInf) describes perception and action through the lens of Bayesian statistics. In ActInf, each 
autonomous entity performs inference over the causes of its sensory input and adjusts its internal 
representations about itself and environment, called beliefs. This belief adjustment is finessed through the duality 
of perception and action, whereby the agent can either passively alter its model to fit the incoming sensory input 
(change its mind), or equivalently adjust its sensory input to fit the internal model by interacting with the 
environment (change the world).
Active Inference
Project title
​
Towards Active Diffusion
A Tale of Multiple (den)Cities
​
​
Short name
2023-AD
Call opened
December 2022
Project start
January 2023
Team Name
The Diffusion Detectives
Facilitators
Jakub Smékal & Daniel Friedman
To join the project 
as a participant:
Read the entire document and prepare any thoughts or questions you have.
Email 
 to express interest and provide thoughts (use the project short name 
2023-AD in email subject. 
ActiveInference@gmail.com
The details of work, beginning in January 2023, will be communicated to those who have expressed interest. 
 1.
 2.
 3.
Field
Value
2023-AD —— Initiative Overview
 •
 •
 •
 •
 ◦

## Page 2

⁠
 (DM) and the recent 
 (LDM) are deep learning architectures used to 
create stochastic representations of their input data and to equivalently remove noise and produce clean outputs. 
DMs have been successful at learning representations of high-dimensional, complex data and are finding diverse 
applications in synthetic data generation whereby noise is gradually removed from a noisy prior through a Markov 
chain.
Diffusion Models
Latent Diffusion Models
ActInf and DM models display many salient areas of similarity. For example:  
Fundamentals: Both ActInf and DM draw important notions from stochastic thermodynamics in their formulation 
of internal representations.
Implementations: Both ActInf and DM are deployed using modern computational methods, and are undergoing 
rapid advancements.
Applications: Both ActInf and DM have overlapping areas of concern and possible application (e.g. in multimedia, 
cyberphysical settings, conversational systems). 
This project and direction has multiple possible implications in areas where the models are currently being applied 
(e.g. ActInf in behavioral neuroscience, DM in multimedia creation) and in emerging areas (e.g.  Autonomous 
Cognitive Agents using Active Diffusion).
This 
 is being distributed at the end of 2022, to solicit and engage interest in the 2023-AD initiative, which 
will operate during 2023. 
catechism
Avenues of Approach
With the team of committed individuals assembled at the beginning of 2023, we will assess relevant Avenues of 
Approach from two primary angles: Theoretical and Applied
Theoretical
Write an analytical paper on the formal relationships between ActInf and DM.
Provide a thorough review of the relevant literature on LDM and belief propagation in ActInf. We will compare and 
contrast the mathematical formalism and computational implementations of different models within applied 
active inference and predictive processing research and attempt to identify the relevant connections to LDMs.
LDMs succeed in condensing high dimensional input data to latent representations which can be efficiently 
manipulated. This capacity is analogous to the manner in which agents encode their beliefs about their 
environment in ActInf. Such an approach might bridge the gap between the continuous and discrete time 
formulation of generative models in active inference, enabling work intersection of contemporary Artificial 
Intelligence and ActInf research (e.g. “
”).
shared intelligence
The diffusion process itself may have multiple interpretations within the ActInf formalism and we hope to 
investigate its link to different belief propagation schemes such as 
, 
, and 
. The diffusion process may also connect to different techniques used to 
make approximations about the latent states of the environment, including the 
 and 
.
variational message passing
marginal message passing
others
Mean Field approximation
Bethe approximation
Different tasks may result in a different classification of LDMs within the perception-action loop, including 
important questions relating to action selection and planning. The latent state representations in LDMs may also 
offer a novel way to do 
 in high-dimensional state-spaces without performing 
computational operations over all observational modalities.
sophisticated inference
 ◦
 •
 ◦
 ◦
 ◦
 •
 •
 •
 ◦
 ◦
 ◦
 ◦
 ◦

## Page 3

Applied
We can generate 
 that demonstrate concordances among ActInf, DM, and related 
methods. Such repositories serve as didactic examples, as well as kernels for related research work.  
open source repositories
We can demonstrate benchmarks on standard datasets and formats (e.g. 
, discrete-time decision 
tasks, multi-agent simulations).
MNIST diffusion
After creating an initial theoretical synthesis of Active Inference and Latent Diffusion Models, we will integrate 
diffusion models within the active inference formalism, building on top of existing implementations, and devise 
experiments to test the effectiveness of DMs in learning representations of a dynamic environment within the 
action-perception loop.
Explore the applicability of individual components within the LDM architectures in the action-perception loop (e.g. 
UNet architecture) as well as in their theoretical significance (e.g. What is the nature of noising and denoising 
regimes in the context of dynamic, nested Markov Blankets? What does an “Active Diffusion” look like in terms of 
inference about action?)
Leverage the abilities of 
 and related LLM models and tools to develop interactive systems capable of 
inference and action.
ChatGPT
Utilize the complex adaptive dynamics Computer-Aided Design (
) package to develop frameworks, use 
cases, and applications in cognitive ecosystems design, wherein we can leverage the power of diffusion models 
to scale previously limited experiments of multi-agent systems. The use of cadCAD will also open up the 
possibility to explore the connection between DMs and ActInf from the perspective of category theory, in 
particular 
. This work will continue to develop the 
 package 
and extend its capabilities.  
cadCAD
generalized dynamical systems
Active Blockference
Consider and co-create the future of 
, in terms of how such systems 
will be enabled and challenged by rapid technological development.
Decentralized Science (DeSci) ecosystems
Milestones
To be determined in January 2023 by the team assembled. 
 ◦
 ◦
 ◦
 ◦
 ◦
 ◦
 ◦
 •


---
*Extraction method: pymupdf*