I am happy to accept suggestions either in R or Matlab, but the code I present below is R-only.
The audio file attached below is a short piece of conversation between two people. My goal is to distort their speech so that the emotional content will becomes unrecognisable. The difficulty is that I need some parametric space for this distortion lets say from 1 to 5, where 1 is 'highly recognisable emotion' and 5 is 'non-recognisable emotion'. There are three ways I thought I can use to achieve that with R.
The first approach was to decrease the overall intelligibility by introducing noise. This solution is presented below (thanks to @carl-witthoft for his suggestions). This will decrease both intelligibility and emotional content of the speech, but it's very 'dirty' approach - it's hard to make it right to get the parametric space, because the only aspect you can control there is an amplitude (volume) of noise.
require(seewave) require(tuneR) require(signal) h <- readWave("happy.wav") h <- cutw(h.norm,f=44100,from=0,to=2)#cut down to 2 sec n <- noisew(d=2,f=44100)#create 2-second white noise h.n <- h + n #combine audio wave with noise oscillo(h.n,f=44100)#visualize wave with noise(black) par(new=T) oscillo(h,f=44100,colwave=2)#visualize original wave(red)
The second approach would be to somehow adjust the noise, to distort the speech only in the specific frequency bands. I thought I could do it by extracting amplitude envelope from the original audio wave, generate noise from this envelope and then re-apply the noise to the audio wave. The code below shows how to do that. It does something different than noise itself, makes the sound cracking, but it goes back to the same point - that I am only able to change the amplitude of noise here.
n.env <- setenv(n, h,f=44100)#set envelope of noise 'n' h.n.env <- h + n.env #combine audio wave with 'envelope noise' par(mfrow=c(1,2)) spectro(h,f=44100,flim=c(0,10),scale=F)#spectrogram of normal wave (left) spectro(h.n.env,f=44100,flim=c(0,10),scale=F,flab="")#spectrogram of wave with 'envelope noise' (right)
The final approach might be the key to solve this, but it's quite tricky. I found this method in report paper published in Science by Shannon et al. (1996). They used quite tricky pattern of spectral reduction, to achieve something that probably sounds quite robotic. But at the same time, from the description, I assume they might have found the solution that could answer my problem. The important information is in the second paragraph in the text and note number 7 in References and Notes - the whole method is described there. My attempts to replicate it so far have been unsuccessful but below is the code I managed to find, together with my interpretation of how the procedure should be done. I think that almost all the puzzles are there, but I can't somehow get the whole picture yet.
###signal was passed through preemphasis filter to whiten the spectrum #low-pass below 1200Hz, -6 dB per octave h.f <- ffilter(h,to=1200)#low-pass filter up to 1200 Hz (but -6dB?) ###then signal was split into frequency bands (third-order elliptical IIR filters) #adjacent filters overlapped at the point at which the output from each filter #was 15dB down from the level in the pass-band #I have just a bunch of options I've found in 'signal' ellip()#generate an Elliptic or Cauer filter decimate()#downsample a signal by a factor, using an FIR or IIR filter FilterOfOrder()#IIR filter specifications, including order, frequency cutoff, type... cutspec()#This function can be used to cut a specific part of a frequency spectrum ###amplitude envelope was extracted from each band by half-wave rectification #and low-pass filtering ###low-pass filters (elliptical IIR filters) with cut-off frequencies of: #16, 50, 160 and 500 Hz (-6 dB per octave) were used to extract the envelope ###envelope signal was then used to modulate white noise, which was then #spectrally limited by the same bandpass filter used for the original signal
So how should the result sound? It should be something between hoarseness, a noisy cracking, but not so much robotic. It would be good if the dialogue would remain to some extend intelligible. I know - it's all a bit subjective, but don't worry about that - wild suggestions and loose interpretations are very welcome.
- Shannon, R. V., Zeng, F. G., Kamath, V., Wygonski, J., & Ekelid, M. (1995). Speech recognition with primarily temporal cues. Science, 270 (5234), 303. Download from http://www.cogsci.msu.edu/DSS/2007-2008/Shannon/temporal_cues.pdf