2
$\begingroup$

I have two audio tracks (instrumental recordings) named track A, and track B. The tracks will be played in an application and the user of this application has the ability to change the balance of these tracks. The user controls a slider ranging from $x=0$ to $x=1$ with which the balance can be changed from hearing track A only, to hearing track A and B mixed equally, to hearing track B only. Mathematically, I have volume functions for track A and B, $V_A(x)$ and $V_B(x)$ respectively, that should satisfy:

  • $V_A(0) = 1$
  • $V_A(1) = 0$
  • $V_A(0.5)=V_B(0.5)$
  • $V_B(0) = 0$
  • $V_B(1) = 1.$

The resulting track R is calculated as follows:

$R(t) = A(t) V_A(x) + B(t) V_B(x)$ where $0 \le x \le 1.$

A naive approach would be the to let $V_A(x) = 1 - x$ and $V_B(x) = x$ but I don't want the volume of track A to go down drastically as the user moves to the equal mix at $x = 0.5$. A better approach is $V_A(x) = \sqrt{1 - x}$ and $V_B(x) = \sqrt{x}$ (see How to mix two signals without changing the overall loudness?). But ideally, I would have the following function (with the non-constant parts possibly some other power function):

$V_A(x) = \left\{ \begin{array}{ll} 1 & \mbox{if } x \le 0.5 \\ 2 (1 - x) & \mbox{if } x > 0 \end{array} \right.$

$V_B(x) = \left\{ \begin{array}{ll} 2x & \mbox{if } x \le 0.5 \\ 1 & \mbox{if } x > 0.5 \end{array} \right.$

The problem is that both track A and track B are mastered at -0.1 dBFS (they will be played on a device and need to be as loud as possible), so obviously the last two functions will (most likely in my case) cause clipping.

I have two ideas on how to handle this:

  1. Use my ideal function and apply some type of limiter (or would it be better to limit track A and B first, and then mix?)
  2. Find the highest value for $V_A(0.5) = V_B(0.5)$ such that $R(t)$ never clips and then fill in the volume curve somehow. I can even limit some peaks so that this value could probably be quite high.

What are people's thoughts on this? My specific question is about idea 1. How does one go about creating a limiter? Does anyone have a good reference to some algorithms?

$\endgroup$
  • $\begingroup$ Could you precalculate loudness maximized files for a few intermediate values of $x$? $\endgroup$ – Olli Niemitalo Oct 28 '16 at 6:30
  • $\begingroup$ I could precalculate parameters for intermediate values of $x$ if that's what you mean. $\endgroup$ – Joris Weimar Oct 28 '16 at 7:12
  • $\begingroup$ I mean that you could use a loudness maximizer implemented by someone else if you have the space for the extra files. $\endgroup$ – Olli Niemitalo Oct 28 '16 at 7:16
  • $\begingroup$ The files will need to be downloaded over the internet, possibly 3G, so I cannot afford that. Precalculating on the device will probably take some time and I prefer not to let the user wait too long. I'm not sure I even understand what it is that you're trying to do. $\endgroup$ – Joris Weimar Oct 28 '16 at 7:19
  • $\begingroup$ Just in case you are curious, I mean that with enough intermediate $x$ files you could arbitrarily well approximate precalculated loudness maximization for all $x$ by simple linear interpolation which is guaranteed not to overshoot. $\endgroup$ – Olli Niemitalo Oct 28 '16 at 7:44
2
$\begingroup$

You can keep the mixing itself linear but add a mapping to the mixing variable $x$ so that you can control its dynamics. The closest this is to reality is the use of logarithmic potentiometers.

So, given $C_1, C_2$ being your mixing channels and $x$ being your mixing variable, the mixed track $M$ at some time instance $n$ is:

$$ M[n] = C_1[n] \cdot x + C_2[n] \cdot (1-x)$$

At $x=0.5$, both tracks are contributing equally, provided that they are mixed to similar levels. If the tracks are not mixed to similar levels, then this inserts a bias on $x$ on either side.

Anyway, so now another "problem" arises which is how to control $x$ and even how to limit it.

To achieve this, we setup a mapping via a function of the form $y = f(x)$. Ideally, $x,y \in \left[0 \ldots 1 \right]$ So now our mixing process looks like this:

$$ M[n] = C_1[n] \cdot f(x) + C_2[n] \cdot (1-f(x))$$

(The right part could also have been $f(1-x)$)

If you want $f$ to limit your values in a subset of $x$, then you could use a sigmoid function. The sigmoid function has a linear part (where the mapping between $y,x$ is almost identical) and two "non-linear" parts called the shoulder (at the top) and the knee (at the bottom). These are the points where the function starts to plateau and no matter what the value of $x$ is, the value of $y$ is pinned down to a specific limit.

Now, obviously, for some sigmoid function $f$, its output $y \in \left[-1 \ldots 1\right]$. But that is not much of a problem because you can simply $y = \frac{f(x)+1}{2}$ and now your $y$ goes from $0$ to $1$. If you don't want it to hit $1$, then you can now multiply the output of your $f$ with another scaling factor. So $y = q \cdot \frac{f(x)+1}{2}$, where $q$ could be $0.8$ in your case.

Another thing you can do with this scheme is control the "dynamics" of the mixing. So, the user might be moving the slider linearly but their effect taking place non-linearly or with some other rate. This could account for the fact that perception is not linear (for example).

For this purpose, you can set up a logarithmic function $f$ that goes from 0 to 1. The logarithmic also has a "limiting" region towards its maximum value.

The general idea is to keep the mixing linear but not necessarily the way that the mixing variable varies.

The other thing you might want to consider if your tracks are "difficult" might be the use of a compressor before the mixing, so that the levels of both audio channels are kept within reasonable limits before they get mixed down. But, inserting a compressor introduces more controls (more variables) that someone would have to fiddle with depending on the tracks. Maybe you don't want this (?)

Hope this helps.

$\endgroup$
  • $\begingroup$ Thanks for your extensive answer. It looks like the sigmoid function is always positive: $f(x) \in (0, 1)$. Anyway, I was actually thinking about an S-shaped function! Good to know my intuition relates to some science :) I will try what you suggest and let you know how it works. Yes, I want to stay away from compressors as much as possible (my tracks are classical music). $\endgroup$ – Joris Weimar Oct 28 '16 at 7:56
  • $\begingroup$ Ah, regarding the sigmoid function range: I guess it's a matter of convention. :) $\endgroup$ – Joris Weimar Oct 28 '16 at 7:58
  • $\begingroup$ You are welcome. The "classic" sigmoid's range is $\left[-1 \ldots 1\right]$ but the limits do not matter too much if you know them because then you can scale the function and bring it exactly where you want it to be. $\endgroup$ – A_A Oct 28 '16 at 8:00
1
$\begingroup$

If you don't want to use a compressor or a limiter because your material is classical music, this pretty much rules out everything except for choosing gains $V_A(x)$ and $V_B(x)$ and sticking to them for the full length of the audio. You would then mix with:

$$R(t, x)=A(t)V_A(x)+B(t)V_B(x)$$

One way to relate $V_A$ and $V_B$ to $x$ is:

$$x = \frac{V_B(x)}{V_A(x)+V_B(x)} = \frac{V_B(x)}{V_\text{tot}(x)}\\ \Rightarrow V_A(x) = V_\text{tot}(x)(1-x) \quad\text{and}\quad V_B(x) = V_\text{tot}(x) x$$

You can precalculate the factor $V_\text{tot}(x) = V_A(x)+V_B(x)$ for a few values of $x$, say $x=0,$ $x=0.25,$ $x=0.5,$ $x=0.75,$ and $x=1$ as the inverse of the maximum of $|A(t)(1-x) + B(t)x|$ over $t$. This way mixing is guaranteed not to clip. Linear interpolation does not overshoot so if you obtain $R(t, x)$ by linearly interpolating between two non-clipping $R(t, x_0)$ and $R(t, x_1),$ then $R(t, x)$ is guaranteed not to clip. If we denote the interpolation position between $x_0$ and $x_1$ as $\alpha = (x - x_0)/(x_1-x_0)$ then: $$R(t, x) = R(t, x_0)(1-\alpha) + R(t, x_1)\alpha\\ =\big(A(t)V_A(x_0)+B(t)V_B(x_0)\big)(1-\alpha) + \big(A(t)V_A(x_1)+B(t)V_B(x_1)\big)\alpha \\ =\big(A(t)V_\text{tot}(x_0)(1-x_0)+B(t)V_\text{tot}(x_0)x_0\big)(1-\alpha) + \big(A(t)V_\text{tot}(x_1)(1-x_1)+B(t)V_\text{tot}(x_1)x_1\big)\alpha \\ = A(t)\big(V_\text{tot}(x_0)(1 - x_0 )(1 - \alpha) + V_\text{tot}(x_1)(1 - x_1)\alpha\big) + B(t)\big(V_\text{tot}(x_1)x_1\alpha + V_\text{tot}(x_0)x_0(1 - \alpha)\big)$$ $$\begin{array}{rl}\Rightarrow &V_A(x) = V_\text{tot}(x_0)(1 - x_0)(1 - \alpha) + V_\text{tot}(x_1)(1 - x_1)\alpha\\ \text{and}&V_B(x) = V_\text{tot}(x_0)x_0(1 - \alpha) + V_\text{tot}(x_1)x_1\alpha.\end{array}$$ $$\begin{array}{rl}\Rightarrow &V_A(x) = V_A(x_0)(1 - \alpha) + V_A(x_1)\alpha\\ \text{and}&V_B(x) = V_B(x_0)(1 - \alpha) + V_B(x_1)\alpha.\end{array}$$

That is to say, rather than linearly interpolating between $R(t,x_0)$ and $R(t,x_1)$ you can linearly interpolate between gains $V_A(x_0)$ and $V_A(x_1)$ and between gains $V_B(x_0)$ and $V_B(x_1)$.

$\endgroup$
  • $\begingroup$ Thanks. That makes sense. I like this relation and this interpolating trick (and the fact that i can just interpolate the gains without worry). There is just one caveat, and I didn't mention this in the post (because I only thought of it now), but my audio will also be stretched and pitch shifted possibly. So I think the precalculating idea is most likely out the window since I can't precalculate the $V_A$ and $V_B$ functions for each stretch and pitch combination. So it looks like I'm going to have to implement a limiter. I don't mind a limiter at the end, just to keep it from peaking. $\endgroup$ – Joris Weimar Oct 28 '16 at 14:39
  • $\begingroup$ It's basically a last resort. In normal usage, the user would mostly be listening to track A, sometimes with a bit of track B mixed in. $\endgroup$ – Joris Weimar Oct 28 '16 at 14:42
  • $\begingroup$ Oh shoot, well this might be useful for someone else. :) $\endgroup$ – Olli Niemitalo Oct 28 '16 at 15:15
  • $\begingroup$ No, it was useful to me. Thank you. I've learned something! $\endgroup$ – Joris Weimar Oct 28 '16 at 19:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.