Timeline for Can Principal Component Analysis (PCA) Solve the Cocktail Party Problem?
Current License: CC BY-SA 4.0
29 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 12, 2021 at 15:26 | vote | accept | hotmeatballsoup | ||
| S Jan 8, 2021 at 19:03 | history | bounty ended | CommunityBot | ||
| S Jan 8, 2021 at 19:03 | history | notice removed | CommunityBot | ||
| Jan 7, 2021 at 17:05 | comment | added | Nathan Huchon | In 2019, when i and some friends made a state of the art on Blind Source Separation (BSS), we found that the most promising solution was brought by the Mitsubishi Electronics & Research laboratory (MERL). To be short, they used short time fourier transform to feed their neural network which used embeddings. Therefore, these trained embeddings were able to classify voices in a more "meaningfull" space. Their talks are easy to find on Youtube and their results are just astonishing. | |
| Jan 7, 2021 at 14:04 | answer | added | Laurent Duval | timeline score: 3 | |
| Jan 5, 2021 at 22:57 | history | edited | Royi | CC BY-SA 4.0 |
edited title; edited tags
|
| Jan 1, 2021 at 21:00 | history | tweeted | twitter.com/StackSignals/status/1345112587529285632 | ||
| Jan 1, 2021 at 15:08 | answer | added | Royi | timeline score: 6 | |
| Jan 1, 2021 at 14:07 | comment | added | hotmeatballsoup | Interesting point @Royi (+1) -- if not PCA/ICA, then what algorithm(s)/strategy(ies) would you use to solve this problem? | |
| Jan 1, 2021 at 2:00 | comment | added | Royi | Are you after some intuition why PCA / ICA work? I must say they don't work well all the time. There are cases they work. It depends on the assumption on the signals an their mixture. | |
| Dec 31, 2020 at 18:03 | comment | added | hotmeatballsoup | Thanks @Jazzmaniac (+1) that makes some very interesting points. But I'm not talking about the human brain here, I'm interesting in how software can tease apart the various sound sources from a sample containing a mixture of them. Any ideas how this can be accomplished in software? Meaning, you have 1+ microphones recording sound from the cocktail party, feeding that recorded sound into a device running software on it. What algorithms, techniques, etc. exist for the software to identify all the unique sound sources at the party, and how to analyze just one particular source? | |
| Dec 31, 2020 at 17:50 | comment | added | Jazzmaniac | If PCA or ICA were the solution to the cocktail party problem, you'd have to have at least as many ears as party guests. So, obviously not. Identifying and separating sound sources uses many cues, of which spatial cues are not even very important. Even a single channel without spatial information can be processed by the human brain and allow for identification of several simultaneous speakers and other sound sources . The greatest contribution is internal model-building in your brain. You know what sounds are there and know them individually. For speech, this is by far the paramount factor. | |
| S Dec 31, 2020 at 17:28 | history | bounty started | hotmeatballsoup | ||
| S Dec 31, 2020 at 17:28 | history | notice added | hotmeatballsoup | Authoritative reference needed | |
| Dec 31, 2020 at 2:01 | history | edited | hotmeatballsoup | CC BY-SA 4.0 |
deleted 4081 characters in body; edited title
|
| Dec 31, 2020 at 0:07 | comment | added | hotmeatballsoup | Hey @BenVoigt (+1) since you left your last comment, I've been encouraged to go down the road of spatial analysis and, in particular, beamforming. If blind source separation (I'm guessing something like principal component analysis?) was enough to identify all the different sound sources in a particular sample, of what use is your recommendation to also use spatial analyses as well? Would the advantage of doing that just be for honing in on a specific sound source and filtering the others out? Thanks for any help here. | |
| Dec 30, 2020 at 19:20 | history | edited | hotmeatballsoup | CC BY-SA 4.0 |
added 1501 characters in body
|
| Dec 30, 2020 at 17:37 | comment | added | hotmeatballsoup | Let us continue this discussion in chat. | |
| Dec 30, 2020 at 15:58 | comment | added | hotmeatballsoup | Thanks again @AnalogEE, I will look into delay-and-sum (+1). Last question (I promise!) going back to the microphones: when you say "The objective is to separate sources purely based on their location in physical space" what is actually going on under the hood here? Are the mics using timing differences in the signals to somehow triangulate where a certain sound is coming from? And then, once a sound is found to be coming from a particular physical location, perform filtering for that particular "sound object"? | |
| Dec 30, 2020 at 15:37 | comment | added | Keegs | No, usually the microphones are identical. The objective is to separate sources purely based on their location in physical space. Beamforming/steering are the same thing in this context and are a type of spatial filtering. Delay-and-Sum beamforming would be a good place to start. For lower power computing, you may be better off with a directional array instead of trying to beamform on the device. There are challenges with audible sound beamforming as it is extremely wide bandwidth compared to the constituent frequencies which makes the lobe width non-uniform. | |
| Dec 30, 2020 at 15:09 | comment | added | hotmeatballsoup | Interesting @AnalogEE (+1) - why would smart speakers use an array of microphones to perform this separation? Are the microphones all different (picking up on different frequencies)? Also, any specific spatial filtering algorithms you'd recommend here? You mention both "beamforming" and "beam steering"? | |
| Dec 30, 2020 at 3:18 | comment | added | Keegs | They seem somewhat related, but I can’t speak to what exactly was meant by diversity. There’s a reason consumer smart speakers use an array of microphones to better separate the user from ambient sounds and other people. Common solutions are to use directional microphones semi-independently, or to use omnidirectional microphones and perform spatial filtering with hard/firm/software beam steering techniques. | |
| Dec 30, 2020 at 2:46 | comment | added | hotmeatballsoup | Thanks @AnalogEE (+1) -- is your recommendation to perform "spatial filtering" or "beamforming" as you say, similar or even identical to the previous recommendation of "spatial diversity" as Ben Voigt suggested? | |
| Dec 29, 2020 at 20:29 | comment | added | Keegs | @hotmeatballsoup It’s not a matter of there being something in addition to fundamentals and harmonics, it’s about the inherent ambiguity of that model. If I see a signal with 1W of power in frequency F and 1/2W in frequency 2F, it’s impossible to know if there’s a single source emitting that energy (fundamental + harmonic) or two separate sources that happen to coincide (two fundamentals). You cannot determine purely through observation whether two elements of a signal are emitted by the same source without spatial filtering (beamforming). | |
| Dec 29, 2020 at 19:39 | comment | added | Hilmar | We went through this before: your assumption around "waves" and "fundamental + harmonics" is somewhere between "extremely simplistic" and "blatantly wrong" A dog bark simply doesn't have fundamentals and harmonics. Before you can let go of this concept, going any further doesn't make sense. | |
| Dec 29, 2020 at 19:02 | comment | added | hotmeatballsoup | Very interesting @AnalogEE (+1) -- (1) with respect to human vocalizations, what am I missing (besides fundamental and harmonics)? And (2) regardless of what I'm missing from my model/understanding, would the recommended blind source separation method be applicable for separating the constituent signals out by source? | |
| Dec 29, 2020 at 18:20 | comment | added | Keegs | Your first assumption is very aggressive in my estimation. Incredibly simple vibrating objects such as tubes and taught strings, in an ideal setting, exhibit this simple harmonic behavior. But the vocalizations of animals and artificially generated sounds are dramatically more complex. If you were to analyze human speech such that it was based on that model, you would be observing either such small segments of the signal or so many overlapped fundamentals and harmonics at once as to render the model useless. | |
| Dec 29, 2020 at 17:52 | comment | added | Ben Voigt | The technical term is "blind source separation", and spatial diversity is highly recommended. | |
| Dec 29, 2020 at 17:25 | history | asked | hotmeatballsoup | CC BY-SA 4.0 |