I am trying to implement compressive sensing for grayscale 2D images, then reconstructing them using a multi-layer perceptron(MLP). It seems to perform well no matter how many layers I add or remove, even without activation functions which is basically a linear combination of the inputs. there must me something wrong with my implementation. this is what I'm doing:
Encoding
- Load image (PIL)
- normalize image by dividing pixel values by 255
- partition image into $B \times B$ blocks(patches) to get a matrix of size $P \times B \times B$ where $P$ is the number of blocks.
- Flatten the blocks to form a matrix of size $P \times B^2$ let's call it $X$
- Take M measurements from each block by getting the dot product of $\Phi$, an $M \times B^2$ Gaussian matrix, and $X^T$, the matrix containing the flattened, normalized blocks. The result is the measurement matrix $Y$ of size $P \times M$
Decoding
From what i understand, this is a sparse approximation problem. meaning my images(or blocks) are supposed to be sparse in some basis $\Psi$ in order to get an acceptable construction. I assumed the basis to be 2D-DCT. Therefore for each block, i try to reconstruct $S_i$ which is a $k$-sparse vector representing the flattened block in frequency domain. So my optimization problem becomes as follows:
$$\min|S_i|_1 s.t. |\Phi\Psi^{-1} S_i-Y_i|_2 < \mu$$
$\mu$ being a small quantity. Now this applies to per-image optimization methods. What i am trying to do is an MLP which learns a universal function $f(Y_i)$ for reconstruction. So after training, my decoding process is as follows: $$\hat{S_i} = f(Y_i)$$ following the acquisition of $\hat{S_i}$ i apply $\Psi^{-1}$ to it in order to get $\hat{X_i}$. finally, $\hat{X_i} * 255$ gives me the flattened block pixel values. reshaping each $\hat{X_i}$ into a $B \times B$ matrix then arranging them gives me the full image $X$.
Training
My dataset consists of $50,000$, $256 \times 256$ images. I just use the first batch (1000 images) since adding more data does not affect my model at all somehow. I do the aforementioned encoding process to every image then concatenate and shuffle all the blocks (i.e instead of images, i now have a matrix of flattened shuffled blocks from multiple images). My model inputs are the measurements taken from these images, of size $1000*P \times M$. To get my training labels, i apply 2D-DCT to each unflattened block before taking the measurements (right after encoding step 3 above), then flatten the sparse labels $1000*P \times B^2$, let's call the matrix $S$. Loss is calculated by taking $MSE(\hat{S}, S)$. Number of layers/nodes-per-layer do not seem to affect the model accuracy at all.
My questions
- A Linear(no activations) MLP with a single hidden layer performs the same or sometimes better than a Multi-layer model with ReLU activations. What might be the cause of this?
- In the literature, to guarantee construction $X$ has to be sparse in some domain. But i tried removing all transformations from my model and just worked in spatial domain and the model performs exactly the same. shouldn't it struggle to reconstruct the signal?
Output example
A model with 1 hidden layer having $B^2$ nodes takes the input of shape $M$. then the intermediate output is given to the output layer having $B^2$ nodes. in short, $M \rightarrow B^2 \rightarrow B^2$, no activations. training $MSE$ is $0.002$. $10$ of the blocks are reserved for validation, validation loss is also $0.002$. $B = 8$, M = $ 0.25 * B^2$. Now trying the trained model on an unseen images like "lena.bmp" $(512 \times 512)$ gives the following results compared to OMP:
N.B. The same $\Phi$ used in training is used in testing.
Any help is appreciated.
