# Tag Info

The Cocktail Party Problem is a Blind Source Separation (BSS) problem. Given a linear mixture of signals: $$\boldsymbol{y} \left[ n \right] = A \boldsymbol{x} \left[ n \right]$$ We're trying to estimate the signal $\boldsymbol{x} \left[ n \right]$. The model can get even more complex with $A$ being time varying: $$\boldsymbol{y} \left[ n \right] = A \... 6 General Idea The general idea of Principal Component Analysis (PCA) is as following (Intuition over formalism): Given a set of points in space (Inner Product Space) find a set of vectors (Directions) which are uncorrelated which span the data in the most energy preserving manner. The tricky part is explaining "most energy preserving manner". So we're ... 5 For discrete data both are the same - Finding set of orthogonal directions which maximizes the Variance (Energy) of data along them. Sometimes those are called the natural axis of the. Since we're dealing with variance it is only natural both are calculated from the covariance matrix of data. You may encounter places where KL might be even defined on the ... 4 I'm not into details of this specific case but I can see some logic. A convolution layer can be reformulated as a Matrix Multiplication:$$ y = W x $$Let's say we trained on Data Set  {x}^{1}  which is big and general. Namely we expect the trained weights  {W}^{1}  to be good enough for almost any other data set. Let's assume we have another data ... 4 Imagine you have a set of 10,000 images (32 x 32) of faces. An intuitive way is to think they have a lot in common. One step farther would be that if you take one of the faces you could generate it from a Linear Combination of the other 9,999 images. Sounds reasonable, isn't it? Let's treat the images as a vector space with dimension  d = 32 \times 32 = ... 3 By projecting a vector x using PCA (on the PCs), you maximize the variance in the reduced space. Initially, the space is not optimal in terms of maximizing the variance. So: PCA projects vector 𝑥 to a space of 𝑝 dimensions where the difference between the initial vector and the projection has maximum energy. (initially the is no maximum variance ... 3 Provided you define it appropriately, the DFT is just an orthonormal transformation: the vectors that make up the DFT matrix are orthogonal to each other and are unit vectors. does that mean that whitening the original signal is equivalent to whitening its DFT? Yes. In fact, the DFT of a white noise sequence is... white noise. 3 I found an answer which is good-enough for me. As @Stanley Pawlukiewicz has pointed out in the comments, this is hard to do for a general case when there is little correlation between the images. I, however, want to work with real images of actual things in the real world. This means there will be a lot of low frequency components (that's why jpeg ... 2 If what you want is PC compression, a more useful form of PCA would be achieved via Singular Value Decomposition (which is, in many cases, more accurate and faster than an eigendecomposition). Assuming that X is a "tall" matrix with dimensions [n, p], n>p; Then: [U, S, V] = svd(X); eig_vals = diag(S.^2); %recall that eigenvalues of cov(X) are the ... 2 One option is called Multilinear principal component analysis: Multilinear principal component analysis (MPCA) is a multilinear extension of principal component analysis (PCA). MPCA is employed in the analysis of n-way arrays, i.e. a cube or hyper-cube of numbers, also informally referred to as a "data tensor". And a RGB image is a sort of cube. ... 1 Speech Source Separation (SSS) or Audio Source Separation (ASS) can be seen as a specialized version of source separation. I mention these expressions under which one can find additional works. One acceptation of the "Cocktail Party Problem" is the task of hearing/recovering one specific sound of interest in a complex environment (one-source ... 1 I've just recently reviewed a paper that used the t-svd as a multidimensional extension to the PCA. They've explicitly tested this on RGB images and claimed to achieve good results. Might be another candidate solution to look at. 1 It does appear that the (re)ranking code is using the wrong dataset, i.e. the Oxford model with the Paris images. This question was raised in the following github issue: wrong dataset name #6. However, the explanation given by the authors is that this is a convention in the literature and they do this to be able to measure their results against prior art. ... 1 We are applying something similar like so: A CNN is trained on a particular image dataset. PCA (or some other transform) is performed on the feature vectors to obtain the main axes of variation. The images are inspected to see what the variations actually correspond to. Features are calculated using the same network on a much much larger dataset. The ... 1 The first approach assumes that you already have identified local features, special points on the face. This identification task is not always straightforward to perform: imagine a face with sunglasses. So it requires quite sophisticated preprocessing. The second one uses more global features, by produce a set of optimal "fake images" (eigenfaces) built ... 1 What lies at the heart of pattern recognition and pattern classification is the selection of the correct features that is used in decisions. And the most important properties of correct features are 1-ease of measurement , 2-power of discrimination. So not every feature is the same. And finding the ones which yields the best performance requires research, ... 1 PCA dimensionality reduction seeks for a linear transformation, mapping the data to a lower dimensional space. The linearity comes from the mapping, which is simply a rotation (and translation if you take into account the de-meaning). In the end, it is nothing but a projection of the data. So if your low dimensional embedding is not reconstructing the data ... 1 The piece I was missing was the distribution of the initial phase values \varphi_1 and \varphi_2. It is standard to assume that these are uniformly distributed [^]. This leads to:$$ \mathbf{R_x} = \mathbf{E[xx^H]} = \mathbf{S \Lambda S^H}  where $\mathbf{\Lambda} = \begin{bmatrix} A_1^2 & 0 \\ 0 & A_2^2 \end{bmatrix}$ and \$\mathbf{S} = \begin{...