I am working on a small personal project where I want to control a matrix of roughly 100 actuators with an audio stream. The goal would be that the user can see/feel the music features in the movement of the actuators. I am still at a early stage, so I can't be very specific about what I need (for now)

What I had in mind was to extract a series of frequency band and map the power estimate to the intensity of one/many actuators. For instance, very low frequency could control the center of the matrix, the range from C3 to B6 would be mapped to arbitrarily selected actuators and very high frequency would be spread all across the matrix, so that sounds with high component such as synth sound would give a "buzz" effect. Well the strategy has to be elaborated more with some trial and errors I guess and a lot of human intepretation.

Nevertheless, one thing is sure, is that I will need to extract the power estimate of some frequency bands, and I want to go for the most CPU efficient technique possible as this will be run by an embedded device. Here's some technique I considered:

  1. Doing an STFT (or a moving DFT as this will be real time). I could multiply the STFT result with a mapping matrix of $N \times M$ (where $N=$ number of FFT bins and $M=$ number of actuators) to get a vector of intensity value that could be sent to the hardware controlling the actuator. The amount of calculation will of course depend on the amount of non-zero value in the matrix, which I suspect will be high.

  2. Wavelet transform : Probably a more efficient way of doing #1, trading precision at high frequency for lower cpu operations. Still, this would imply a downsmapling filtering as well.

  3. Doing a filter bank of bandpass filters. I did a quick estimate of the amount of operation needed. With an IIR bandpass filter with a 40dB attenuation at the neighbor demi-tone. I find roughly 16 multiplication ~10 addition per sample at low frequency and 30 multiplication, 20 additions at high freq. Which we can approximate ~1350 Multiplications and 720 additions if we consider 48 actuators and the average values of low and high freq requirements.

So, the question goes as follow : What technique should I mostly look for?

Even if there is still requirements to define, is there a technique that is to be avoided? Or maybe a similar application?

Right now, the wavelet approach seems to me like the most appealing approach.

Note that I may move some load on a FPGA if I get too short on CPU power.

  • $\begingroup$ why a "moving DCT" rather than a moving DFT? i think usually the DCT is obtained with post-processing of the result of the DFT and the DFT is most efficiently done with the FFT. how many samples does the DFT or DCT frame hop each time it's performed? sometimes, when there is a lot of overlap between frames (the frame hop is small), some technique exists to save redundant computation of intermediate values in the DFT. $\endgroup$ – robert bristow-johnson Sep 19 '18 at 4:38
  • $\begingroup$ I corrected my question, I meant a moving DFT. I thought of a hop of a single sample to be able to add on one side and substract on the other for each frequency bin I need. This technique is discussed by Richard Lyon in his book, I assumed this is the technique you are referring to $\endgroup$ – Pier-Yves Lessard Sep 19 '18 at 12:48
  • $\begingroup$ why does it appear that each "actuator" (whatever the heck that is) gets its own FFT? are there M channels of audio? $\endgroup$ – robert bristow-johnson Sep 20 '18 at 2:35
  • $\begingroup$ Hmm, I don't see how it appears like that, but each actuator can take an input from any bin of the same FFT result. If I have as much actuator than FFT bin, then the identity matrix would map each bin to an actuator. Since I will have less actuator than FFT bins, I<ll have to sum them up. I may want half of a bin; for instance [0 0 0.5 1 1 1 0.5 0 0] would merge 5 bins together with the edge at 50% coupling. Does that make sense? $\endgroup$ – Pier-Yves Lessard Sep 20 '18 at 3:00
  • $\begingroup$ so the $N \times M$ matrix is a matrix that maps the FFT bins to actuators. okay, i sorta get that. $\endgroup$ – robert bristow-johnson Sep 20 '18 at 3:21

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.