so in principle the agent can calculate the probability of a hidden state of the world under its internal model by multipling the likelihood by the prior and normalising by the marginal . The challenge is that computing the marginal term requires integrating over all of the possible hidden states, which tends to be intractable unless the internal model is chosen to have a form that has a closed form integral e.g., a Gaussian.

The consequence is that if an Agent want to do Bayesian inference they face a choice:

1. large computational cost to do exact inference on arbitrary internal models.
2. restrict the internal model to a family that is tractable to integrate in closed form.
3. do some form of approximate inference e.g., variational inference.

Depending on the compute time budget available, exact inference is likely to be infeasible, and limiting ourselves to families of distributions that have closed form integrals is quite restrictive, which leaves us with branch 3 to explore.

One of the approaches to approximate inference that I like is the Free Energy principle by Friston. The idea is to first rearrange Bayes’ rule to focus on the intractable marginal .

Let be an arbitrary probability distirbution parameterised by . We have the freedom to insert this distribution into the top and bottom of the fraction on the RHS

Next we take logs of both sides and negate, converting the LHS into surprise

Then taking the expectation of both sides over we have

The LHS is not a function of so the integral simplifies

On the RHS we can choose how to split the terms inside the logarithm