# What is the principle behind my voice echoing perpetually when 2 separate devices are used for playback and recording on video calls?

I had to use 2 separate devices for a video call, desktop for video/audio output and smartphone for recording, since the desktop microphone was not working properly. Unfortunately for me, when I started speaking, my first few words got echoed perpetually, and my following words were ignored by the microphone. What could be the reason behind this phenomenon?

More importantly,

Why does this phenomenon not occur in video calls if the same device is used for recording and playback? How is this phenomenon avoided?

The Explanation from @ScienceGeyser provides a good explanation to the phenomenon.

There are two more things to address the question on how is this phenomenon avoided.

1. The feedback read by the microphone is not identical to the audio sent to the speakers. There is the physical response from the speakers, the acoustic of the device and the environment, if microphone and speakers fixed as parts of the same rigid device like in a cell phone or a headset.

2. For a situation where there is wires or wireless connection between speaker and microphone, the position from one in relation to the other may change.

In any case the system can be modeled as follows, $$k(z)$$ is the speaker signal, $$m(z)$$ is the microphone signal, and $$s(z)$$ is the response from the sources other than the speaker to the microphone. We can say assume that the response of the microphone to the speaker sound is $$F(z)\, k(z)$$ in which $$F(z)$$ is a slowly varying system, so if we process short segments of the signal we may assume $$F(z)$$ to be time invariant.

Then in order to successfully cancel the response from the system you have to compute $$F(z)$$.

## Particular case

To simplify the notation of the formulas in the frequency domain we $$z = exp(j \omega T_s)$$ where $$T_s$$ is the sampling period, and $$\omega$$ some angular frequency, under these conditions that discrete Fourier transform coincides with z transform .

If you assume $$F(z)$$ to have a short input response and to change slowly enough that you can use it as a time invariant filter in a block of $$N$$ samples.

We have $$m(z) = F(z) k(z) + s(z)$$, in your device you have both $$k(z)$$ and $$m(z)$$ (in the time domain).

You can compute the energy in the frequency domain as $$m(z) m(z)^*$$, I am using the $$*$$ suffix to denote complex conjugation.

$$\begin{eqnarray} m(z)^*m(z) &=& (F(z) k(z) + s(z))^*(F(z) k(z) + s(z)) \\ &=& \underbrace{|F(z)k(z)|^2}_{\textrm{echo energy}} + 2 Re(F(z)k(z)s(z)^*) + \underbrace{|s(z)|^2}_{\textrm{external sound}} \end{eqnarray}$$

It is reasonable to assume that $$k(z)$$ and $$s(z)$$ must be uncorrelated, and one can see that the echo will increase the energy of the signal from the microphone.

The corrected signal will be given by

$$\hat s(z) = m(z) - \hat{F}(z) k(z) = s(z) + k(z)(F(z) - \hat{F}(z))$$

And $$\hat{F}(z)$$ is chosen in a way that minimizes the energy of $$\hat s(z)$$, that in the frequency domain consists in a linear regression problem. Let $$\hat F_i$$, $$m_i$$ and $$k_i$$ be the coefficients of $$\hat F(z)$$, $$m(z)$$ and $$k(z)$$, respectively

$$\hat F_i = \frac{m_i k_i^*}{\sum |k_i|^2}$$

Using big number of samples will give better estimate of $$\hat F(z)$$ but the time invariability assumption is more strongly violated. What can be done is to take advantage of the previous estimates. To address this we can filter $$\hat F(z)$$ itself.

$$\tilde{F}_i \leftarrow \tilde{F}_i \alpha + (1 - \alpha) \frac{m_i k_i^*}{\sum |k_i|^2}$$

And you can increase alpha for a more stable estimate, or decrease it to have a faster estimate.

### Notes on complex arithmetic

To answer the question about the term 2 Re(.) left on the comment. The short answer is this is the crossed terms in the expansion of the product in the previous.

I skipped some steps there to avoid making long arrays of equations since this is a very common result in complex manipulations.

$$\begin{eqnarray} (x + y)^* (x + y) &=& (x^* + y^*)(x + y) \\ &=& (x^* x + y^* y + x^*y + y^*x \\ &=& x^* x + y^*y + (x^*y) + (x^*y)^* \end{eqnarray}$$

Let $$x = a+bj$$, with $$j^2=-1$$, term $$x^*x=|x|^2$$,

$$\begin{eqnarray}x^*x &=& (a-bj)(a+bj) \\&=& (a^2 - b^2j^2 -baj +abj) \\&=& (a^2 -b^2(-1)) \\&=& (a^2 + b^2) \\&=& |x|^2\end{eqnarray}$$

Similarly, $$y^*y = |y|^2$$.

The terms $$x^*y + y^*x$$ are conjugate from each other, thus they could be written as $$w^* + w$$, since $$w = Re(w) + j Im(w)$$ and $$w^* = Re(w) - j Im(w)$$, $$w + w^* = 2 Re(w)$$, where $$Re(w)$$ denotes the real part, and $$Im(w)$$ denotes te imaginary part.

### Why the energy is the square of the amplitude

I will explain this using a Mass-spring system, let a mass $$m$$ attached to a sprint of constant $$k$$ oscillating with amplitude $$A$$. The maximum potential energy of the spring is achieved when it is deformed by $$A$$, and can be calculated as $$\int_{0}^{A} k \, x\, dx = k A^2 / 2$$. Also, if there are multiple oscilation modes (different frequencies), each one conserves energy separately, this is why we can talk about power at specific frequencies.

• Why do you have an asterisk in the second line of the expression for m(z)*m(z)? You have written 2Re(F(z)k(z)s(z)∗). Is that correct? I understand Re stands for real part, but what does the asterisk stand for? Mar 3, 2021 at 6:07
• Why is the echo energy squared and the sound energy not squared? Mar 3, 2021 at 6:26
• The asterisk I used to denote complex conjugation (the bar was not expanding over the complete sub-expression). I added two small sections addressing your questions.
– Bob
Mar 3, 2021 at 11:33
• I read that, but it does not seem right to me. Please check line 2 of m(z)*m(z). Mar 3, 2021 at 12:45
• :D thank you, when I arrived @ScienceGeyser had already nailed it. I tried to not repeat him. At least I think it was close!
– Bob
Mar 3, 2021 at 13:24

On each individual device, the speaker output can get subtracted from the microphone before it gets sent to other locations. This prevents others from hearing themselves through your microphone. When using two devices in within audible range of each other, the devices cannot subtract the speaker audio from the microphone audio because the information path does not exist. The delay time is the determined by the network delays between the two devices and if the device with the speaker audio is very loud, you can get a gain of more than 1 to the device with the microphone. If the device with the microphone is using automatic gain control and you are not the loudest thing in the room, then you will eventually only hear the loudest thing in the room. In your case, the echo must have been very loud.

This is due to the delay that occurs between the recording process and the playback process. The voice audio signal has to be recorded by the microphone and then played back by the speaker.

If 2 separate devices are used, then recording and playing back the audio are 2 different events, so your own voice will be played back to you.

In the same device the mic input signal is the sum of the input voice signal and the speaker output signal, so the speaker output signal is continuously subtracted from the input signal to get the input voice signal back. This subtraction occurs on the device, so echoes are avoided.

A detailed explanation of the phenomenon:

• When I spoke the first few words, the microphone recorded it and the audio was played through the desktop's speakers.
• Since the audio output of the desktop has more amplitude than my voice, the microphone picked up that audio signal, and my voice signal got drowned.
• This recorded audio would be played on the desktop again, and it would get recorded by the microphone again. This cycle repeats, with the same audio being echoed perpetually.
• From a signal processing perspective, this forms a closed loop feedback system, and the oscillations are sustained since the desktop amplifies it every time.

Feedback without a gain factor (or a very high one).

Incidentally, this is the principle behind Schroeder reverberators.

• In my opinion, there is a gain factor here since the desktop speaker amplifies it. The gain factor would be quite high. Sep 11, 2020 at 3:18
• Sorry, should clarify: an uncontrollable gain factor! Unless you use your hands to cover the mic/speaker. Sep 11, 2020 at 21:46