10

A random process is a collection of random variables, one random variable for each time instant. It is best to write the random process as $$\{X(t)\colon -\infty < t < \infty\} \tag{1}$$ where the $\{$ and $\}$ indicate that a set (or collection) of objects is being defined, and the interior says that a typical member of this set is denoted by $X(t)$ ...


10

when we observe the Random Process at a specific time $t_k$, that is the value at $X(s_1,t_k), X(s_2,t_k),\ldots,X(s_n,t_k)$, if we denote them by $(a_1,a_2,\ldots, a_n)$. Now the mapping between the outcomes $(s_1,s_2,s_3,\ldots,s_n)$ and its probabilities $(a_1,a_2,\ldots,a_n)$ are collectively called as Random Variable. Not true. Random variable ...


7

The definition of the autocorrelation function $R_x(\tau)$ depends on the nature of your $x$. If $x$ is a deterministic signal with finite energy then: $$R_x(\tau)=\int_{-\infty}^{+\infty}x(t)x^*(t-\tau)dt$$ If $x$ is a deterministic signal with finite average power$^{(1)}$ then: $$R_x(\tau)=\lim_{T\to+\infty}\frac{1}{T}\int_{-T/2}^{+T/2}x(t)x^*(t-\tau)dt$$...


4

If you sample a random process for a specific t, you will get one realization of a random variable. For another t, you get another realization of that random variable. This random variable has its statistics which is almost impossible to learn in real world because not all sample paths are observable. See the brown rectangle in the figure below. That ...


4

In easy words: A process is stationary if its stochastic properties are independent of the time you look at it. Think of it like this: A stochastic process is just a Random Variable (RV) that, instead of giving you e.g. a real value gives you a function every time you look at it. We call that realizations. If you now take a lot of these realizations, and ...


4

It's basically just about using the definitions and doing the math: $$\begin{align}R_{WW}[n]&=E\{W^*[k]W[k+n]\}\\ &=E\left\{\sum_mU^*[k-m]h^*[m]\sum_lU[k+n-l]h[l]\right\}\\ &=\sum_m\sum_lh^*[m]h[l]E\left\{U^*[k-m]U[k+n-l]\right\}\\ &=\sum_m\sum_lh^*[m]h[l]R_{UU}[n+m-l]\\ &=\sum_mh^*[m]\sum_lh[l]R_{UU}[n+m-l]\\ &=\sum_mh^*[m]\left(h\...


4

You are mixing up two different notions. Your random process is a collection of random variables $\{X(t)\colon -\infty < t < \infty\}$, one random variable for each time instant. The autocorrelation function of the process is $$R_X(t, \hat{t}) = E[X(t)X(\hat{t})], -\infty < t, \hat{t} < \infty$$ and is a two-dimensional function (meaning it has ...


4

The answer to the OP's question is more straightforward than rb-j's comments make it out to be. $\{X(t)\colon -\infty < t < \infty\}$ is a continuous-time WSS random process with autocorrelation function $$R_X(\tau) = E[X[t)X(t+\tau)], -\infty < \tau < \infty.$$ On the other hand, $\{Y(n)\colon n \in \mathbb Z\}$ where $Y[n] = X(nT)$ is a ...


4

For the case of input process $\{X(t)\}$ being white Gaussian noise with two-sided power spectral density $\frac{N_0}{2}$, the output process $\{Y(t)\}$ is a strictly stationary zero-mean Gaussian process in which all the random variables have the same variance $\frac{N_0}{2}\int_{-\infty}^\infty |H(f)|^2 \,\mathrm df$ almost as you say. But the key point ...


4

Think it this way; assuming $x[-1] = 0$, then recusively compute the output $x[k]$ for $k \ge 0$ such as $$\begin{align} x[0] &= v[0] \\ x[1] &= a x[0] + v[1] \\ x[2] &= a x[1] + v[2] = a^2 x[0] + a v[1] + v[2] = a^2 v[0] + a v[1] + v[2] \\ x[2] &= \left( a^2 v[0] + a v[1] \right) + v[2] \\ ... &= ... \\ x[k] &= \left( a^k v[0] + a^{...


4

You got some definitions wrong. It's correct that orthogonality means that $E[XY]=0$. Uncorrelated means that $X-\mu_X$ and $Y-\mu_Y$ are orthogonal, i.e., $E[(X-\mu_X)(Y-\mu_Y)]=0$. If you work that out you should arrive at the equivalent condition $E[XY]=\mu_X\mu_Y$ for uncorrelatedness (not for independence!). Consequently, if at least one of the two RVs $...


3

First of all lets state more correctly that a discrete-time 1D auto-correlation sequence (ACS), $\phi_{XX}[\kappa]$, of a single parameter $\kappa$, of a discrete-time random process $X[n,s]$ will exist as long as the process is stationary or at least WSS (wide sense stationary). And therefore the power spectral density (PSD), $\Phi_{XX}(e^{j2\pi fT})$, of ...


3

In addition to stationarity, your process must be ergodic to relate these two definitions. Ergodicity tells us joint probability of your signal's value at two instant of time (which only depend on their time difference) is equal to the number of instants of time which those combination with the same delay appeared in your signal divided by the total time.


3

Let me explain it in another way. Consider you have 6 different function of time. You only throw your dice once and regarding the outcome you choose on of six functions and the chose one is one outcome of your random process. Now expand this concept to all possible functions of time instead of only 6 functions and instead of dice throwing consider another ...


3

Regardless of what your book might say, a random process is a collection of random variables that is described in mathematical notation as $\{X(t) \colon t \in \mathbb T\}$ or $\{X_t\colon t \in \mathbb T\}$ where $\mathbb T$ is called the index set (typically, $\mathbb T$ is $(-\infty,\infty)$ or $\{\cdots, -2, -1, 0, 1, 2, \cdots\}$ or subsets thereof) ...


3

A white noise sequence is one for which each (random) element is uncorrelated from every other element: $$ E[y[n]y[m]] = \left \{ \begin{array}{ll} 0 & \mbox{for } n\not=m\\ \sigma_y^2 & \mbox{for } n = m \end{array} \right . \\= \sigma_y^2 \delta[n-m] $$ where $\sigma^2_y$ is the variance of $y$. Note that I am assuming (because of the whiteness) ...


3

Joint behavior cannot always be deduced from individual behavior. For example, if $X$ and $Y$ are (nondegenerate) random variables with finite means, then $P\{X<E[X]\}$ and $P\{Y<E[Y]\}$ both are nonzero. This is because there must be probability mass to the left of the mean (and to the right of the mean too) except in the degenerate case when $X$ has ...


3

1 Proof You're misusing the word proof. Remember that a proof, even one led with stochastic methods, always leads to an absolute "If A, then B, no doubt" statement. Since your $x$ is a realization of a stochastic process of which you don't know many properties, pretty much everything is possible – for example, a perfectly white process might have a ...


3

Well, getting a bit linguistic, according to the Oxford dictionary: stochastic (adj.): Having a random probability distribution or pattern that may be analysed statistically but may not be predicted precisely. So the definition would be the first one (I don't know where you might have found the second one as you didn't put any source about it). ...


3

Consider the following LTI system with impulse response $h[n]$ $$ \{v[n]\} \longrightarrow \boxed{H(z)} \longrightarrow \{u[n]\} $$ From the analysis of LTI systems with WSS random inputs, the following results deduced. Given the input $v[n]$ which is zero mean white WSS random process, its ACS (Auto-Correlation Sequence) sequence is: $$\phi_{vv}[m] = \...


3

The mean of $X(n)$ is always $\mu_X =0$, because the noise has zero mean. Thus we should check whether the autocorrelation corresponds to a WSS process. $$\begin{align} R_X(n_1,n_2) &=\mathbb{E}[X(n_1)X(n_2)]\\ &=\mathbb{E}[u(n_1)e^{-kn_1}F(n_1)u(n_2)e^{-kn_2}F(n_2)]\\ &=u(n_1)u(n_2)e^{-k(n_1+n_2)}\mathbb{E}[F(n_1)F(n_2)]\\ &=u(n_1)u(n_2)e^{-...


3

If E[X(t)] is constant and RX(t+τ,t)=0 can I say the process is WSS? Can I say RX(t+τ,t)=0=RX(τ) and therefore is WSS? Two times the same question. It fulfills the definition (as you noticed yourself), so why even ask? yes. What does RX(t+τ,t)=0 mean? It means the process is uncorrelated ($R_X(\tau \ne 0)=0$), but also that its variance is 0 ($R_X(0) = \...


2

An eternal well-balanced dice has 1/6 probability for each facet $f$, each time. This uniform probability law yields a mean expectation of 3.5: $\sum_{f=1}^{6} \frac{1}{6}\times f$. Each time you cast the dice, you get this expectation. Of course, for each throw you'll only get an integer 1, 2, 3, 4, 5, or 6, never a decimal like 3.5. So there is an ...


2

I never saw the second definition, but the example of SGD actually fits the first definition. SGD works with a random estimator of the gradient, and just GD is using the real gradient (calculated). To sum up, as far as I know and saw stochastic relates to randomness and not iterative (step by step).


2

I wanted to chime in a little only because I think I see why someone would have given you the obviously flawed definition of sample-by-sample. Adaptive algorithms like the LMS filter are based on a stochastic gradient update. For LMS in particular, the updates are typically made on a sample-by-sample basis. In other words, you don't try to do a lot of ...


2

Question 1: If we have a PAM sequence like this: $a[k]=[0, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0]$, then the PAM signal (Waveform) must hold frequency components which cover these fast transitions from 0 to 1 to 0 to 1. The oscillation speed (frequency) of the transitions are basically given by the time $T$ between the PAM symbols in the waveform. If $G(f)$ ...


2

The issue is possibly that the input signal you have chosen is not persistently exciting. This means that the signal doesn't "excite" enough modes of the filter in order to be able to accurately estimate its parameters. Another way to think about it is that it doesn't have enough energy in enough places in the spectrum: just at the frequency of the cosine, ...


2

As far as I can see your discrete-process noise covariance matrix Qk is wrong (as it's based on a polynomial Kalman filter model) whereas your system dynamics matrix A is based on a sinusoidal Kalman filter model. You should use the following to find the correct discrete-process noise covariance matrix: $$ Q_k = \int_{0}^{T_s} \Phi(\tau) Q \Phi(\tau)^T d\...


2

An innovations filter of a WSS process $X(t)$ is a causal and stable minimum phase filter that can be used to generate $X(t)$ from a white noise input $N(t)$: $$X(t)=\int_0^{\infty}h(\tau)N(t-\tau)d\tau\tag{1}$$ where $h(t)$ is the impulse response of the innovations filter. The (causal and stable) inverse filter of $h(t)$ is called the whitening filter ...


Only top voted, non community-wiki answers of a minimum length are eligible