Bayesian Machine Learning

This page contains resources about Bayesian Inference and Bayesian Machine Learning.

Bayesian Networks do not necessarily follow Bayesian approach, but they are named after Bayes' Rule.

Subfields and Concepts[]

Bayesian Model Selection / Bayesian Model Comparison
- Bayes Factor
- Bayesian Model Averaging (in Ensemble Learning)
- Bayesian Information Criterion (BIC)

Bayesian Parameter Estimation
- Bayesian Decision Theory
- Bayesian Signal Processing
- Bayesian Nonparametric Methods
Bayesian Feature Selection / Bayesian Variable Selection
- Spike and Slab Method
- Kuo & Mallick
- Gibbs Variable Selection (GVS)
- Stochastic Search Variable Selection (SSVS)
- Adaptive shrinkage with Jeffreys' prior or Laplace prior
- Reversible jump MCMC
Variational Bayesian Inference
Approximate Bayesian Computation (ABC)
- Automatic Variational ABC
- Variational Bayes with Intractable Likelihood (VBIL)
Bayesian Information Theory
- The Principle of Maximum Entropy
- Bayesian Occam's Razor
- Minimum Message Length (MML)

Bayesian Linear (Regression) Model
Bayesian Naive Bayes
Bayesian Mixture Models
Sparse Bayesian Models
- Bayesian LASSO
- Bayesian Ridge Regression
- Relevance Vector Machine (RVM) / Automatic Relevance Determination / Sparse Bayesian Learning
- The Spike and Slab Model
Bayesian State Space Models
- Bayesian Linear Dynamical System
- Bayesian Time Series
- Bayesian Structural Time Series (BSTS)
  - Kalman filter
  - Spike and Slab Method
  - Bayesian Model Averaging
Probabilistic Matrix Factorization
Bayesian Multitask Learning
Bayesian Optimization
Bayesian Reinforcement Learning
Bayesian Neural Network
Bayesian Deep Learning

Davidson-Pilon, C. (2015). Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional.
Koduvely, H. M. (2015). Learning Bayesian Models with R. Packt Publishing.
Theodoridis, S. (2015). "Section 12: Bayesian Learning". Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2014). Bayesian data analysis. 3rd Ed. Chapman & Hall/CRC.
Lee, P. M. (2012). Bayesian statistics: an introduction. 4th Ed. John Wiley & Sons.
Barber, D. (2012). Bayesian Reasoning and Machine Learning. Cambridge University Press.
Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern Classification. John Wiley & Sons.
Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press.
Berger, J. O. (2010). Statistical decision theory and Bayesian analysis. 2nd Ed. Springer Science & Business Media.
Koller, D., & Friedman, N. (2009). Probabilistic Graphical Models. MIT Press.
Barber, D., Cemgil, A. T., & Chiappa, S. (2011). Bayesian time series models. Cambridge University Press.

Bishop, C. M. (2006). "Section 2.2: The beta distribution". Pattern Recognition and Machine Learning. Springer.

MacKay, D. J. (2003). "Chapter 37: Bayesian Inference and Sampling Theory". Information Theory, Inference and Learning Algorithms. Cambridge University Press.

Mitchell, T. M. (1997). "Chapter 6: Bayesian Learning". Machine Learning. McGraw Hill.

Pearl, J. (1988). "Chapter 2: Bayesian Inference". Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann.

Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., & de Freitas, N. (2016). Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1), 148-175.
Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature, 521(7553), 452-459.
Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015). Weight Uncertainty in Neural Network. In Proceedings of The 32nd International Conference on Machine Learning (pp. 1613-1622).
Snoek, J., Rippel, O., Swersky, K., Kiros, R., Satish, N., Sundaram, N., & Adams, R. P. (2015, February). Scalable bayesian optimization using deep neural networks. In International Conference on Machine Learning.