Monday, February 15, 2016

DL, the new Bohr model: The thoughts of an irreducible DL troll

In the dawn of the 20th century, most physicists thought Physics had been solved! Thusly, chicken feasts with exquisite wine were organized in labs all around the globe. Then it turned out the Bohr model had serious gaps, and suddenly the party was over, and all lights were put out. DL is the new Bohr model (assuming it is a model at all...), and ICLR2016 looks like a chicken feast :p

Sunday, January 3, 2016

Random thoughts on the Lasso (to be continued...)

For any $n$-by-$p$ matrix $X$ (with non-zero rows), consider the following objects
$$\mathcal P_X := \{z \in \mathbb R^n | \|X^Tz\|_\infty \le 1\},\; D_X := \text{diag}(\|X_1\|_\infty,\ldots, \|X_n\|_\infty),\; Z_{D_X} := D_X^{-1}\mathbb B_1,$$
where $\mathbb B_1$ is the unit-ball for the $\ell_1$-norm.  Note that $Z_{D_X} \subseteq \mathcal P_X$.



Given a closed convex set $K \subseteq \mathbb R^n$, we've defined the euclidean projection
$$\text{proj}_K(a) := \text{the unique point of }K\text{ minimizing distance from }a \text{ to } K.$$
It's not (too) hard prove that $0 \le QP \le QA$.

What more (of geometric taste) can be said about the picture ?