In this talk, we consider multivariate regression with hidden variables, $Y = \Theta^TX + B^TZ + E$, where $Y$ is a $m$-dimensional response vector, $X$ is a $p$-dimensional vector of observable features, $Z$ represents a $K$-dimensional vector of unobserved hidden variables, possibly correlated with $X$, and $E$ is an independent error. The number of hidden variables $K$ is unknown and both $m$ and $p$ are allowed (but not required) to grow with the sample size $n$. We address several fundamental challenges of this problem, (1) parameter identification, (2) estimation methods/non-asymptotic analysis, (3) asymptotic inference, and (4) generalizations to heteroscedastic errors and GLM. This talk is based on a sequence of works primarily with my students. Related papers include  https://arxiv.org/abs/2003.13844,  https://arxiv.org/pdf/2201.08003, https://arxiv.org/abs/2509.00196.