Convert a list of values and times into an audio file

Question

I have two pandas dataframe, one with time in seconds and one with the actual audio data (values from -1 to 1).

The audio data points are not equidistant, this why I need the list of time in seconds.

Here is a short example of the two arrays:

1. time in seconds:

0.000934
0.004197
0.005921
0.006978
0.007306
0.009449
0.012022
0.024245
0.030468
0.038149

2. audio data points:

0.446907
0.432984
0.410248
0.385579
0.363269
0.343606
0.328705
0.309281
0.285214
0.268380

How can convert them into an audio file?

Answer 1

I think the way to go here is oversampling, lowpass filtering and, optional, consecutive downsampling. This process will yield equidistant audiodata.

1. Oversampling

Choose a target oversampling period $T_{os}$ and correspondig oversampling frequency $f_{os}=1/T_{os}$ that is well below $T_{\text{min}}$, the shortest time distance occuring in your data. The higher you go, the smaller the error you make will become. My guess based on the short excerpt is 48kHz for $f_{os}$ will suffice, but feel free to go with 96 or even 192kHz. Create an array of zeros of length $l\cdot f_{os}$, where $l$ is the length of your audio data in seconds.

Then, divide your time values by $T_{os}$ and round the results to the nearest integer (ceil/floor). (This is, where the error is introduced.) Next, put your audio samples in your array of zeros using the integers in the time array as indices.

2. Lowpass filtering

Apply a steep lowpass filter with cutoff frequency $f_c=1/2T_{\text{min}}$. The result will be an interpolated version of the sparse signal you created by oversampling.

3. Downsampling (Optional)

Downsample your signal again, if you whish to do so. This may need another lowpass filtering, depending on your whished sample rate.

4. Write data to file

Write data to wav file or any other format of your choice. You will need to supply your sampling rate to the function.

Answer 2

An asynchronous resampler should do the trick.

Basically positioning a continuous time windowed sinc at the desired (uniform) output time instants, sampling it by a neighbourhood of input time instants, choosing sinc width (inverse of bandwidth) either as a function of the largest input inter sample spacing, or locally as a function of sample density.

Unless sample density variation is large or quality requirements are really high, you can probably get by with something simpler. For some applications, simple linear interpolation could be sufficient. For high quality audio applications featuring tonal components, that may result in severe audible artifacts, but I struggle to imagine high quality audio applications where sample time is significantly variable?

edit: A simple MATLAB code snippet that may do what you need would be this:

t1 = [0.000934,0.004197,0.005921,0.006978,0.007306,0.009449,0.012022,0.024245,0.030468,0.038149];
x1 = [0.446907,0.432984,0.410248,0.385579,0.363269,0.343606,0.328705,0.309281,0.285214,0.268380];
max_fs = 48e3;
fs = min(1/min(diff(t1)), max_fs);
[x2, t2] = resample(x1, t1, fs);

figure
plot(t1, x1, '*')
hold on
plot(t2, x2, '-')
hold off

Note that I did not bother to dig into the documentation to see nitty gritty details. I confirmed that there is some shape similarity between input points and output points and assume the implementation to be sensible.

Answer 3

Although I cannot directly provide a solution to your problem I think I can point you towards the "most well established" approach.

To the best of my knowledge, non-uniform sampling (both in time and space if this is of interest to you in some generic way) is related to uniform sampling via interpolation. I believe that the "simplest" and most straight forward approach is to go for a Lagrange interpolation scheme. On the other hand, both Max and Knut Inge have suggested pretty much a very similar approach (I am just making it explicit here).

I can't really comment on the relation between the two processes, resampling with possible downsampling as the final stage as Max suggested and direct interpolation of the original data to acquire the uniform-sampled version of it. For more information you could have a look at the Wikipedia page and (in my opinion even better) the "Nonuniform Sampling - Theory and Practice" book edited by Farokh Marvasti. I believe that chapters 3 (if you are unfamiliar with Lagrange interpolation), 4, 5 and 16 should be relevant to your question.

I am sorry I cannot provide a direct answer to your question and I do hope you'll manage to find a viable solution.