Bayes_InfoTheory #

This is a repository of Jupyter notebooks used by Florent Leclercq during lectures on Bayesian statistics and Information Theory. The homepage of the lectures is accessible here.

Contents ###

• Ignorance priors (exemplified with the lighthouse problem) and the maximum entropy principle
• Gaussian random fields:

* Examples and a digression on non-Gaussianity * Bayesian signal processing and reconstruction: de-noising * Bayesian signal processing and reconstruction: de-blending

• Bayesian decision theory
• Markov Chain Monte Carlo:

* Monte-Carlo integration, importance sampling, rejection sampling * Metropolis-Hastings algorithm & Gelman-Rubin test (MCMC) * Slice sampling * Gibbs sampling * Hamiltonian sampling (HMC), and comparison of 2nd and 4th order integrators for HMC

• Approximate Bayesian Computation:

* Likelihood-free rejection sampling * Synthetic likelihood (parametric approximation) * Discrepancy and effective likelihood (non-parametric approximation) * Implicit likelihood inference with Gaussian signals

• Information theory:

* The noisy binary symmetric channel * Supervised Machine Learning basics: Titanic example

• Cosmological and physical examples:

* Wiener filtering for the Cosmic Microwave Background * Bayesian decision theory for Cosmic Web classification * Supernova cosmology: data and simulations (preliminary exercise) and inference with MCMC and HMC * The 1919 Eclipse: parameter inference and model comparison

Acknowledgments ###

I thank Benjamin Wandelt for his own lectures, which have inspired a fraction of this material, and the SOC/LOC of the ICIC Data Analysis workshop 2021 and STFC Summer School on Data Intensive Science 2021.

Warranty ###

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.