Before starting: I am really a beginner in statistical process in time. I mainly do quantum information and while learning aspect of quantum noise I realized that I am actually too weak on basics of statistical process.
Thus, I would like to first tell you what I basically understand and that you confirm or correct me. Then I have some more specific questions.
Allright, here is what (I think) I understood, and I would like to check
Let's consider a random process. I consider a random variable $X$. First, I can fix a given time $t$, and ask questions about the statistical properties of $X(t)$. For example its mean value $<X(t)>$, its standard deviation $<(X(t)-<X(t)>)^2>$, and I could imagine many other quantities. In principle without further assumptions I need a lot of information (an infinite one ?) to specify all the characteristics of the statistics of $X(t)$.
Furthermore: here it is what is happening at a given time $t$. Even if I fully described the statistics of $X(t)$ I would lack information. For example there might have some correlations between what happens at $t$ and what happens at $t'$. Thus I also need to specify some properties linking those different times. They are not given by what happens at a given time $t$.
An example I have in mind is a variable that verifies: $X(t) \in [0,1]$, uniformously distributed. But the correlations are such that $X(t'>t)=X(t)$. Then if I do a first experiment I will have $X(t)=0.2$ for example. And for all further times $X(t')=0.2$ as well because of correlations. If I re-run the experiment I can have $X(t)=0.87$ for example but again the values for further times will be the same as $X(t)$. Do you confirm my understanding ?
To specify what happens in time we thus need other information like autocorrelation functions: $C(t,t')=<X(t)X(t')>$. And probably many other quantities that I am not aware of.
More specific questions
What is a rigorous definition of stationnary statistical process ? What I basically understand is that the statistical properties of $X(t)$ actually do not depend on $t$. And the statistical properties of $X(t)X(t')$ only depend on $t-t'$. But I guess there is a more precise definition than this.
In a paper I am reading, they assume a gaussian, stationnary process. They say that knowing the average value $<X>$ and the autocorrelation function allows to fully specify the process. I do not understand this. I agree that it will fully describe the statistical behavior at a given time (because from autocorrelation function we have access to the variance in addition to the mean we already knew). But how is the autocorrelation function the only information we need to specify the statistical relationship between different times ?
Finally, basic question but... what is the name of the precise field that deals with all this ? It is not statistics (this is way too broad), I tried some keyword with "statistical noise" but I didn't find something good. Related: is there a good reference to really give the basics ? I don't think I need much.