qi

(ML 13.1) (ML 13.2) Directed graphical models - introductory examples (Directed) Graphica “Models” aka “Bayesian” networks Better name would be “conditional independence diagrams” of probability distributions Key notions: factorization of probability distributions notational device useful for visualization of (a) conditional independence properties (b) inference algorithms (DP, MCMC) Why conditional independence...

(ML 12.1) Model selection - introduction and examples “Model” selection really means “complexity” selection! Here, complexity $\approx$ flexibility to fit/explain data Example (Linaer regression with MLE for $w$) $f(x) = w^T\varphi(x)$ Given data $x \in \mathbb{R}$, consider polynomial basis $\varphi(x) = x^k$, $\varphi = (\varphi_0, \varphi_1, \ldots, \varphi_B)$ Turns out...

(PP 6.8) Marginal distributions of a Gaussian One of the weird and wonderful things is: the family of Gaussians is preserved under many different operations. Proposition.(Marginalization) (PP 6.9) Conditional distributions of a Gaussian Proposition.(Conditional) (PP 6.10) Sum of independent Gaussians Proposition. If $X \sim N(\mu_X, C_X)$ and $X \sim N(\mu_X,...

(ML 11.1) Estimators Model the data as random variables: $D = (X_1, \ldots, X_n)$. Definition. A statistic is a random variable $S = f(D)$ that is a function of the data $D$. Terminology. An estimator is a statistic intended to approximate a parameter governing the distribution of $D$. Notation. $\hat{\theta}$...

(ML 10) Baeysian linear regression Linear regression is one of the most powerful general purpose tools. Using MLE for ligression often leads to overfitting, which can be a sever problem. Using MAP can solve the problem of overfitting, but we don’t have rpresentation of uncertainty in this case (not only...