In the Fourier series, $x(t) = \sum\limits_{-\infty}^{\infty} C_n e^{jnwt}$ where $C_n =\frac{1}{T} \int\limits_0^T x(\lambda)e^{-jnwt} d\lambda$. $T$ is the period, $w$ the frequency, $j = \sqrt{-1}$.
So if we plug in $n=0$, we get the following: $x(t) = \sum\limits_{-\infty}^{\infty} C_n e^{0}$, with $C_n =\frac{1}{T} \int\limits_0^T x(\lambda)e^{0} d\lambda$. Any number raised to the $0$th power is $1$, so $C_n =\frac{1}{T} \int\limits_0^T x(\lambda) d\lambda$. This is exactly how we define the average value. So in image processing, $F(0)$ corresponds to the average value of all the pixels.
And yes, this is the term that looks at a frequency of 0. It's simply an offset to be added. It's not the whole image, it's just a coefficient. So yeah, I guess it could be the whole image, just a horrible approximation. It would have every pixel equal to the average.