2
$\begingroup$

I am having the most difficult time mixing simple unsigned 8-bit/8kHz linear PCM audio samples in JavaScript (AppsScript). I've tried all the basic maths listed here, employing basic number arrays.

  1. Sign, add, clip, unsign. Subtract 128 from each byte, add them, clip, then add 128.
  2. Produce the Average, each sample just a[x]+b[x]/2
  3. Viktor T. Toth's, maths linked.

Full example is below. This will ask you to authorize Google Drive to access the audio samples (no it does NOT touch anything of yours): https://script.google.com/macros/s/AKfycbzMOWV5Z5soG3uinrFc0WcscNXxsDq9elE9rzG0t703vsvDJZMS/exec

As you can see the 3 mixed results are downright horrible. I've always assumed the unsigned bytes were just offset by 128. Not the case? Do I need to fold half the byte to get a true linear representation? Does a basic number in JavaScript not translate appropriately into signed/unsigned representation? If that's the case the Java/AppsScript is somehow properly doing it with the two real samples.


Update Here is some specific code below, and the output of this is in the link above. Alas, it simply doesn't quite work (byte folding in JavaScript)?

   var vegaArray = DriveApp.getFileById('0B-e9EqGm0pWPQ3RUTXFyUERDVTA').getBlob().getBytes();
   var fdraArray = DriveApp.getFileById('0B-e9EqGm0pWPaUJQUmFRQWctNG8').getBlob().getBytes();

   for(var i=44;i<vegaArray.length;i++)
     vegaArray[i] = Math.round( ( 2 * (vegaArray[i] + fdraArray[i]) )
       - ( vegaArray[i] * fdraArray[i] / 128 )
       - 256
     ); // for
   // vegaArray[] is the mixed output
$\endgroup$
  • 2
    $\begingroup$ do you perform the addition phase with an 8-bit signed data type or a larger kind of data type such as an 32-bit signed integer or a float etc. Because using an 8 bit container, there will possibly be severe wrap-around distortions after addition, even before you can clip the saturated sums, moreover clipping is quite useless in that case as well... $\endgroup$ – Fat32 Dec 10 '16 at 0:41
  • $\begingroup$ @fat32, I believe the implicit numeric type in JavaScript is double, therefor the "byte" read is just [0.0,255.0] on the internal representation. If I output the array, I get values above and below the 8-bit range easily. $\endgroup$ – Jé Queue Dec 10 '16 at 1:05
  • 1
    $\begingroup$ I'm looking at a solution right now. the first thing I noticed is that both audio files are sized such that directly adding them will not clip, just FYI $\endgroup$ – portforwardpodcast Dec 10 '16 at 2:00
  • 1
    $\begingroup$ and average-mixing should never clip. $\endgroup$ – robert bristow-johnson Dec 10 '16 at 2:18
  • $\begingroup$ FYI, it appears as though AppsScript flops the byte. Effectively b>127?b-256:b $\endgroup$ – Jé Queue Dec 10 '16 at 18:57
1
$\begingroup$

Assuming that abyte[] and bbyte[] are two arrays with samples valued from 0-255, this is the solution:

combined = [];
for(i = 0; i < abyte.length; i++)
{   
    sample =  2*(abyte[i] + bbyte[i]) - ((abyte[i]*bbyte[i])/128) - 256
    combined[i] = Math.round(sample);
}

I have a full example with both wav files extracted into arrays (I did this using matlab). Also I verified that it plays back fine using matlab.

https://jsfiddle.net/4hsfo96g/

$\endgroup$
  • 1
    $\begingroup$ Well I can guarentee you mine works fine so start comparing my outputs to yours. Also you should edit your question to include code $\endgroup$ – portforwardpodcast Dec 10 '16 at 2:55
  • 1
    $\begingroup$ Maybe try a passthrough test where you call Math.round() on the samples from one array but that's it. Try and play that output, if it's all messed up you know you have a problem with how you are writing the results and not your DSP $\endgroup$ – portforwardpodcast Dec 10 '16 at 3:06
  • 1
    $\begingroup$ even though i've seen it in the Toth reference, can you explain how the multiplication of the two signals together is a functional part of "mixing" audio signals? is the generation of frequency components that exist in neither of the two inputs a function of "mixing"? $\endgroup$ – robert bristow-johnson Dec 10 '16 at 19:22
  • 1
    $\begingroup$ No it's not mixing In the traditional sense. It's just added. But the lay person term In audio is "mixing" $\endgroup$ – portforwardpodcast Dec 10 '16 at 22:10
  • 1
    $\begingroup$ no port, the Toth algorithm is not just adding. that's the point. despite that Viktor T. Toth may have some reputation as a software designer, his audio mixing algorithm is shit. if you use that algorithm, your output will sound bad. if you add without scaling, you will need to apply some kind of saturation (and there are soft-clipping algs out there) to prevent wrap-around overflow (which is the worst sounding), and if you add and scale by $\frac12$, then it is impossible that two clean signals going in will come out distorted. $\endgroup$ – robert bristow-johnson Dec 10 '16 at 22:16
0
$\begingroup$

Found the answer via some trial-error. Bytes are folded. AppsScript represents numerics (in byte example here) as 2s-complement, meaning the logical high 7 bits of this 8-bit integer [128,255] is [-128,-1] in ascending order. Therefor, averaging or adding as-is was pointless.

The step-wise code in AppsScript:

   // See code above from where vegaArray and fdraArray come
   for(var i=44;i<vegaArray.length;i++) {
     // flop the bytes atop
     vegaArray[i] = vegaArray[i]<0?vegaArray[i]+256:vegaArray[i];
     fdraArray[i] = fdraArray[i]<0?fdraArray[i]+256:fdraArray[i];

     // now do the unsigned 8-bit stuff (avg, add)
     vegaArray[i] += fdraArray[i];
     vegaArray[i] /= 2 ;

     // flop back the bytes abaft
     vegaArray[i] = vegaArray[i]<128?vegaArray[i]:vegaArray[i]-256;

   } // for
$\endgroup$
  • $\begingroup$ what is a "folded byte"? do you mean two's complement? i know what a folded spectrum is. $\endgroup$ – robert bristow-johnson Dec 12 '16 at 23:09
  • $\begingroup$ examining your code, i think you mean that the bytes are what we used to consistently call "Offset binary" as an alternative to two's complement. $\endgroup$ – robert bristow-johnson Dec 12 '16 at 23:11
  • $\begingroup$ also, "var" doesn't tell me anything when i look it up at places defining Java variable types. is it char or int? is it unsigned char or unsigned int? $\endgroup$ – robert bristow-johnson Dec 12 '16 at 23:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.