Basic Statistics:
- Random numbers. Probability. Moments. Generating function.
- Independence. Law of large numbers (strong and weak versions). Central Limit Theorem. Large Deviation and Cramer function. Sigma algebra operations.
Foundations of information theory:
- Multivariate distributions. Marginalization. Conditional Probabilities. Bayes theorem.
- Entropy, independence, mutual information, comparison of probabilities (Kullback-Leibler)
- Probabilistic inequalities for entropy and mutual information
- Information channel. Shannon Theorem.
- Thermodynamic potentials and free energies, thermodynamic formalism in information theory.
Markov Chains [discrete space, discrete time]:
- Transition probabilities. Properties of Markov Chains. Steady State Analysis.
- Spectrum of the Transition Matrix & Speed of convergence (to steady state)
- Reversible and Irreversible Markov Chains. Detailed vs Global Balance.
- Exactness and convergence. Perron-Frobenius (connect to linear algebra) and implications for Markov chains.