# 1.1. Generalized Linear Models¶

The following are a set of methods intended for regression in which the target value is expected to be a linear combination of the input variables. In mathematical notion, if $$\hat{y}$$ is the predicted value.

$\hat{y}(\beta, x) = \beta_0 + \beta_1 x_1 + ... + \beta_p x_p$

Where $$\beta = (\beta_1, ..., \beta_p)$$ are the coefficients and $$\beta_0$$ is the y-intercept.

To perform classification with generalized linear models, see Bayesian Logistic regression.

## 1.1.1. Bayesian Linear Regression¶

To obtain a fully probabilistic model, the output $$y$$ is assumed to be Gaussian distributed around $$X w$$:

$p(y|X,w,\alpha) = \mathcal{N}(y|X w,\alpha)$

Alpha is again treated as a random variable that is to be estimated from the data.

References

• A good introduction to Bayesian methods is given in C. Bishop: Pattern Recognition and Machine learning
• Original Algorithm is detailed in the book Bayesian learning for neural networks by Radford M. Neal

## 1.1.2. Bayesian Logistic regression¶

Bayesian Logistic regression, despite its name, is a linear model for classification rather than regression. Logistic regression is also known in the literature as logit regression, maximum-entropy classification (MaxEnt) or the log-linear classifier. In this model, the probabilities describing the possible outcomes of a single trial are modeled using a logistic function.

The implementation of logistic regression in pymc-learn can be accessed from class LogisticRegression.