Indeed the Linear Least Squares uses the Covariance (Which resembles Cross Correlation) and the Variance (Which resembles the Auto Correlation) for parameter estimation.
Let's see that using simple example - Linear Function fitting.
Assume our model is given by (Simple Polynomial Model of Order 2):
$$ {y}_{i} = a {x}_{i} + b, \; i = 1, 2, \cdots, n $$
In this model we assume knowing $ \left\{ \left( {x}_{i}, {y}_{i} \right) \right\}_{i = 1}^{n} $ and we're after estimating $ a, b $.
So, we could build this in the form of Ordinary Least Squares:
$$ \boldsymbol{y} = X \boldsymbol{\theta} \Rightarrow \hat{\boldsymbol{\theta}} = {\left( {X}^{T} X \right)}^{-1} {X}^{T} y $$
Let's examine it using basic Math.
Estimating $ b $ using the Least Squared Error:
$$ \hat{b} = \arg \min_{b} \frac{1}{2} \sum_{i = 1}^{n} {\left( {y}_{i} - a {x}_{i} + b \right)}^{2} $$
Taking the Derivative with respect to $ b $ will yield:
$$ \hat{b} = a \sum_{i = 1}^{n} {x}_{i} - \sum_{i = 1}^{n} {y}_{i} = a \bar{x} - \bar{y} $$
Where $ \bar{x} = \sum_{i = 1}^{n} {x}_{i} $ and $ \bar{y} = \sum_{i = 1}^{n} {y}_{i} $.
It suggests that the constant $ a $ is set such that the line must go through the mean of $ \left\{ {x}_{i} \right\}_{i = 1}^{n} $ and $ \left\{ {y}_{i} \right\}_{i = 1}^{n} $. Which indeed makes sense, as this is the mean of the 2D data.
Estimating $ a $ using the Least Squared Error:
$$ \hat{a} = \arg \min_{a} \frac{1}{2} \sum_{i = 1}^{n} {\left( {y}_{i} - a {x}_{i} + b \right)}^{2} $$
Taking the Derivative with respect to $ a $ will yield:
$$\begin{aligned}
\frac{\mathrm{d} \frac{1}{2} \sum_{i = 1}^{n} {\left( {y}_{i} - a {x}_{i} + b \right)}^{2} }{\mathrm{d} a} & = -\sum_{i = 1}^{n} {x}_{i} \left( {y}_{i} - a {x}_{i} + b \right) && \text{} \\
& = -\sum_{i = 1}^{n} {x}_{i} {y}_{i} + a \sum_{i = 1}^{n} {x}_{i}^{2} -b \sum_{i = 1}^{n} {x}_{i} && \text{} \\
& = -\sum_{i = 1}^{n} {x}_{i} {y}_{i} + a \sum_{i = 1}^{n} {x}_{i}^{2} - \left( a \bar{x} - \bar{y} \right) \sum_{i = 1}^{n} {x}_{i} && \text{Inserting estimation of $ b $} \\
& = -\sum_{i = 1}^{n} {x}_{i} {y}_{i} + a \sum_{i = 1}^{n} {x}_{i}^{2} - a \sum_{i = 1}^{n} {x}_{i} \bar{x} + \sum_{i = 1}^{n} {x}_{i} \bar{y} && \text{} \\
\Rightarrow a & = \frac{\sum_{i = 1}^{n} {x}_{i} {y}_{i} - \sum_{i = 1}^{n} {x}_{i} \bar{y}}{\sum_{i = 1}^{n} {x}_{i}^{2} - \sum_{i = 1}^{n} {x}_{i} \bar{x}}
\end{aligned}$$
It looks like we're close but not right there.
The trick is noting these equations:
$$ \sum_{i = 1}^{n} \left( \bar{x} \bar{y} - {y}_{i} \bar{x} \right) = 0 , \; \sum_{i = 1}^{n} \left( \bar{x}^{2} - {x}_{i} \bar{x} \right) = 0 $$
So now:
$$\begin{aligned}
a & = \frac{\sum_{i = 1}^{n} {x}_{i} {y}_{i} - \sum_{i = 1}^{n} {x}_{i} \bar{y}}{\sum_{i = 1}^{n} {x}_{i}^{2} - \sum_{i = 1}^{n} {x}_{i} \bar{x}} && \text{} \\
& = \frac{\sum_{i = 1}^{n} \left( {x}_{i} {y}_{i} - {x}_{i} \bar{y} \right) + \sum_{i = 1}^{n} \left( \bar{x} \bar{y} - {y}_{i} \bar{x} \right)}{\sum_{i = 1}^{n} \left( {x}_{i}^{2} - {x}_{i} \bar{x} \right) + \sum_{i = 1}^{n} \left( \bar{x}^{2} - {x}_{i} \bar{x} \right)} && \text{} \\
& = \frac{ \sum_{i = 1}^{n} \left( {x}_{i} - \bar{x} \right) \left( {y}_{i} - \bar{y} \right) }{ \sum_{i = 1}^{n} \left( {x}_{i} - \bar{x} \right) \left( {x}_{i} - \bar{x} \right) } && \text{} \\
& = \frac{ \frac{1}{n} \sum_{i = 1}^{n} \left( {x}_{i} - \bar{x} \right) \left( {y}_{i} - \bar{y} \right) }{ \frac{1}{n} \sum_{i = 1}^{n} \left( {x}_{i} - \bar{x} \right) \left( {x}_{i} - \bar{x} \right) } && \text{} \\
& = \frac{ \operatorname{Cov} \left( x, y \right) }{ \operatorname{Var} \left( x \right) }
\end{aligned}$$
Where we have the sample Covariance and the Sample Variance of the data.
P. S.
Found also this great answer https://math.stackexchange.com/a/717552/33 (Which is similar).