0
$\begingroup$

my questions might come across different SE forums. But here it is.

A little context : I'm currently building a midi sequencer, the architecture is this way (for now) I've got a ARM v7 + a PIC, the ARM v7 runs a linux distro and will do the Human Interface (getting interactions on buttons and hardware interfaces + react to them in order to fill my loops) and the PIC mcu, will only do the proper sequencing, ie : it will only write in rhythm what the arm will tell him. For now, the pic is only doing the real time interface with MIDI out using a crystal, at some point it might also handle some other hardware feature such as getting the inputs and outputs from the buttons and rotary but I will decide that later. And the linux board might only do audio processing (sampler part) + UI.

Now what I am interested in is the following : I'd like to add into my system a sampler, the idea is to have up to 16 samples and to play them into an audio output. I've already used some higher level approach to do that using external libraries as FMOD or even Alsa or Gstreamer, but what I want to do is to have full flexibility and possibility of optimization on my mixer. For instance later part will include effects and tuning, and bit crushing, and this kind of stuff. In the first version I'd like to add the amplitude of the signal, ie a value between 0 and 1 that would set the amplitude of the sample regarding the state of a rotary encoder.

So here is my question, let's say I have 16 wav file, 48kHz, 16bits. How can I do the mixing of these 16 samples ?

If I call k[i] the amplitude setting, and samplei the pcm data of the sample at the current moment then is the following correct :

mixer_out(t) = (k[0]*sample [0] (t)+...+k[15]*sample [15] (t)) / (k[0]+...+k[15])

I've seen every thing and it's contrary when it come to sample mixing. First can I simply add my pcm data in order to get the mix of the sample ? Then (if we don't take into account the clipping) how shall I afterward put my sample mixed into the mixer_out ? I think my naive idea will be far from a correct way of doing the mixing, but what I need is a very low level (at least at pcm level) approach of mixing.

So let's take N pcm samples (all at the same format), and 1 pcm audio out called mixer_out, and let's mix all the sample together adding a constant k[i] (i from 0 to N) for all the samples (k is a float between 0 and 1). I believe that it could be possible to take k from 0 to 1/N in order to simplify if that helps. Even though I do understand that this won't be optimal when I will have several sample with k=0.

Also take into consideration that my system will be run into an embedded processor in "real time" (or do be more precise, the mixer out will have to be calculated at least as fast as or faster than the audio is played) so it will not have unlimited processing power, but I'm ready to do compromise with the number of output in order to have a good audio quality out of my mixer. But this will be determined later, when I will have all the audio capacities of my audio processor.

You can give your answer in floating point or in integer, because I already know way about how to convert them regarding the need I would have.

I know that this question has been asked many times, but I don't feel like I had the satisfactory answer that I'm seeking. I need a general formula to my specific problem. If I'm finding good answer, I would be very happy to share my simple low level mixer code into a github and link it here.

Edit : For those that might fall here while searching for an audio mixer. The formula I gave is one of the correct one. To mix samples you'll just have to sum them. Easy as that. Then the clipping part is up to you, and what you want to do : most of the mixer let the user make the sound clip, it allows you more flexibility on the amplitude of each sample, ie : you want the kick to sound 2 times louder than the snare: go for it and find a way so that it won't clip.

$\endgroup$
6
  • $\begingroup$ The mixing is easily addressable, can you please clarify the role of the pic in this setup? What does the pic do exactly? $\endgroup$
    – A_A
    Apr 16 '18 at 12:33
  • 1
    $\begingroup$ I there is any chance you can afford, do all of your audio processing in floating point. Convert PCM to floating point as soon as possible and only convert back to PCM, if you have to provide and output. ARM 7 typically has good floating point support and also a Neon SIMD instruction set to parallelize if needed $\endgroup$
    – Hilmar
    Apr 16 '18 at 13:42
  • $\begingroup$ For now, the pic is only doing the interface with MIDI ins & outs using a crystal, at some point it might also handle some other hardware feature such as getting the inputs and outputs from the buttons and rotary but I will decide that later. And the linux board might only do audio processing (sampler part) + UI. $\endgroup$
    – YCN-
    Apr 16 '18 at 13:43
  • $\begingroup$ @Hilmar indeed I plan to use Neon instruction to optimize the mixing, but I'm mostly wondering about the theory of mixing. Let's say I convert my PCM into floating point and back into PCM before outputing, then would my formula get's usable ? $\endgroup$
    – YCN-
    Apr 16 '18 at 13:49
  • $\begingroup$ You're dealing with amplitudes, not magnitudes here. A simple mixer simply adds the 16 samples (at a particular time instance) together and sends them to output. To prevent clipping, you should divide the summed value by 16. Provided that constant k is in the 0-1 range this should work. $\endgroup$
    – dsp_user
    Apr 16 '18 at 14:02