Statistics & Probability

Basic Statistics:

  • Random numbers. Probability. Moments. Generating function.
  • Independence. Law of large numbers (strong and weak versions). Central Limit Theorem. Large Deviation and Cramer function. Sigma algebra operations.

Foundations of information theory:

  • Multivariate distributions. Marginalization. Conditional Probabilities. Bayes theorem.
  • Entropy, independence, mutual information, comparison of probabilities (Kullback-Leibler)
  • Probabilistic inequalities for entropy and mutual information
  • Information channel. Shannon Theorem.
  • Thermodynamic potentials and free energies, thermodynamic formalism in information theory.

Markov Chains [discrete space, discrete time]:

  • Transition probabilities. Properties of Markov Chains. Steady State Analysis.
  • Spectrum of the Transition Matrix & Speed of convergence (to steady state)
  • Reversible and Irreversible Markov Chains. Detailed vs Global Balance.
  • Exactness and convergence. Perron-Frobenius (connect to linear algebra) and implications for Markov chains.