Active Inference

Active inference is a theory of adaptive action selection for agents, proposed by Karl Friston and expanded by many others. The core claim is that action selection and decision-making can be usefully understood as inference problems — hence “active” inference — derived from the free energy principle, which states that the existence of any entity presupposes it performing Bayesian inference over its active states.

The theory promises both a descriptive account (explaining how the brain selects actions) and a prescriptive framework (designing effective decision-making agents).

Relationship to Reinforcement Learning

Active inference and reinforcement learning are essentially isomorphic. Both can be derived as consequences of the “control as inference” framework — a general mapping of decision-making problems onto Bayesian inference problems. The key difference is how reward is encoded into the probabilistic graphical model:

• RL (Control as Inference): Reward is treated as an optimality variable — the probability that a state-action trajectory is optimal
• Active Inference: Reward is encoded as a prior over desired outcomes (the agent’s “preferences”)

This difference gives rise to slightly different objectives, but in practice algorithms from one framework can be ported directly to the other with little change. As Millidge (2024) concludes, “RL methods as a subset of active inference.” ¹

Expected Free Energy and Exploration

The Expected Free Energy (EFE) is the objective optimised by active inference agents. It decomposes into:

1. A reward-maximising term (pragmatic value) — seeking preferred outcomes
2. An information-gain term (epistemic value) — seeking to reduce uncertainty about the environment

Active inference claims this exploration bonus arises naturally from first principles, giving it an advantage over standard RL methods.

Critique (Millidge 2024): The EFE is not directly derivable from a standard Bayesian treatment. Existing derivations either assume their conclusion or are incorrect. A more fundamental distinction exists between:

• Divergence objectives — minimise divergence between desired and actual states → produces information-seeking exploratory behaviour
• Evidence objectives — standard utility maximisation

See reinforcement-learning and free-energy-principle for related frameworks.

Neuroscientific Plausibility

The claim that brain function is well-explained by active inference has weak empirical support. While the brain performs tasks isomorphic to Bayesian inference in some sense, Millidge argues for a bottom-up empirical approach: understand brain algorithms on their own terms first, then examine their Bayesian nature, rather than fitting top-down Bayesian algorithms to neural data. ¹

Relevance to AI Alignment

Active inference contributes to alignment discourse through:

• Uncertainty calibration: Problems like misspecified utility functions, extreme Goodharting, and squiggle maximisation stem from poor uncertainty specification. Bayesian frameworks with sensible priors address these.
• Divergence vs evidence objectives: The distinction is relevant for regularised/soft optimisation in alignment, where we want bounded, corrigible agents rather than hard utility maximisers.
• Preference aggregation as Bayesian inference: Modelling CEV-like solutions as Bayesian inference is a promising direction.

Relationship to Other Frameworks

Active inference is one of several computational neuroscience frameworks for understanding brain function. Unlike spiking-neural-networks, which focuses on biologically realistic spike-based computation at the neuronal level, active inference operates at the algorithmic/computational level of decision-making and Bayesian inference. The two frameworks are complementary — SNNs could implement active inference at the neural level, though this has not been a focus of either field. Other related frameworks include reinforcement-learning (decision-making via reward maximisation), the free-energy-principle (the unifying thermodynamic/Bayesian principle from which active inference is derived), and analog-cognition-consciousness (Miller et al., 2026 — a hybrid synapse/wave theory that grounds predictive processing in specific physical mechanisms of oscillatory electric field dynamics and ephaptic coupling). The metaphysical backdrop for such multi-level descriptions — properties of organized systems that are physically realized without being reducible to their components — is treated in emergence, with james-woodward’s interventionist account as the causal criterion for which level’s variables answer to interventions.

Assessment

Strengths:

• Unifies perception and action under a single theoretical framework
• Focuses on structured generative models for discrete systems
• Drives research into principled Bayesian inference methods beyond SGD
• Provides a modelling language for understanding agent behaviour

Weaknesses:

• Practically indistinguishable from standard deep RL on most tasks
• No demonstrated “secret sauce” or consistent performance advantage
• Neuroscientific evidence is weak
• Most practical implementations still rely on black-box neural network variational inference

Verdict (Millidge 2024): “Theoretically interesting but practically limited vs standard RL methods.”

References

• Millidge, B. (2024). A Retrospective on Active Inference. Blog post. https://www.beren.io/2024-07-27-A-Retrospective-on-Active-Inference/ ¹
• Friston, K. et al. — Foundational papers on the free energy principle and active inference
• Levine, S. (2018). Reinforcement learning and control as probabilistic inference: Tutorial and review. — The “control as inference” framework

Footnotes

1. raw/articles/millidge-2024-retrospective-active-inference.md ↩ ↩² ↩³