This page contains resources about Bayesian Inference and Bayesian Machine Learning.

Bayesian Networks do not necessarily follow Bayesian approach, but they are named after Bayes' Rule.

Bayes' Rule can be used at both the parameter level and the model level.

Subfields and ConceptsEdit

  • Bayesian Parameter Estimation
  • Bayesian Feature Selection / Bayesian Variable Selection
    • Spike and Slab Method
    • Kuo & Mallick
    • Gibbs Variable Selection (GVS)
    • Stochastic Search Variable Selection (SSVS)
    • Adaptive shrinkage with Jeffreys' prior or Laplace prior
    • Reversible jump MCMC
  • Variational Bayesian Inference
  • Approximate Bayesian Computation (ABC)
    • Automatic Variational ABC
    • Variational Bayes with Intractable Likelihood (VBIL)
  • Bayesian Information Theory
    • The Principle of Maximum Entropy
    • Bayesian Occam's Razor
    • Minimum Message Length (MML)
  • Bayesian Linear (Regression) Model
  • Bayesian Naive Bayes
  • Bayesian Mixture Models
  • Sparse Bayesian Models
    • Bayesian LASSO
    • Bayesian Ridge Regression
    • Relevance Vector Machine (RVM) / Automatic Relevance Determination  / Sparse Bayesian Learning
    • The Spike and Slab Model
  • Bayesian State Space Models
    • Bayesian Linear Dynamical System
    • Bayesian Time Series
    • Bayesian Structural Time Series (BSTS)
  • Probabilistic Matrix Factorization
  • Bayesian Multitask Learning 
  • Bayesian Optimization
  • Bayesian Reinforcement Learning
  • Bayesian Neural Network
  • Bayesian Deep Learning

Online CoursesEdit

Video LecturesEdit

Lecture NotesEdit

Books and Book ChaptersEdit

  • Davidson-Pilon, C. (2015). Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional.
  • Koduvely, H. M. (2015). Learning Bayesian Models with R. Packt Publishing.
  • Theodoridis, S. (2015). "Section 12: Bayesian Learning". Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
  • Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2014). Bayesian data analysis. 3rd Ed. Chapman & Hall/CRC.
  • Lee, P. M. (2012). Bayesian statistics: an introduction. 4th Ed. John Wiley & Sons.
  • Barber, D. (2012). Bayesian Reasoning and Machine Learning. Cambridge University Press.
  • Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern Classification. John Wiley & Sons.
  • Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press.
  • Berger, J. O. (2010). Statistical decision theory and Bayesian analysis. 2nd Ed. Springer Science & Business Media.
  • Koller, D., & Friedman, N. (2009). Probabilistic Graphical Models. MIT Press.
  • Barber, D., Cemgil, A. T., & Chiappa, S. (2011). Bayesian time series models. Cambridge University Press.
  • Bishop, C. M. (2006). "Section 2.2: The beta distribution". Pattern Recognition and Machine Learning. Springer.
  • MacKay, D. J. (2003). "Chapter 37: Bayesian Inference and Sampling Theory". Information Theory, Inference and Learning Algorithms. Cambridge University Press.
  • Mitchell, T. M. (1997). "Chapter 6: Bayesian Learning". Machine Learning. McGraw Hill.
  • Pearl, J. (1988). "Chapter 2: Bayesian Inference". Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann.

Scholarly ArticlesEdit

See also Further Reading.

  • Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & de Freitas, N. (2016). Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE104(1), 148-175.
  • Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature521(7553), 452-459.
  • Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015). Weight Uncertainty in Neural Network. In Proceedings of The 32nd International Conference on Machine Learning (pp. 1613-1622).
  • Snoek, J., Rippel, O., Swersky, K., Kiros, R., Satish, N., Sundaram, N., & Adams, R. P. (2015, February). Scalable bayesian optimization using deep neural networks. In International Conference on Machine Learning.



See alsoEdit

Other ResourcesEdit