# Can a linear reconstruction in compressive sensing perform well?

I am trying to implement compressive sensing for grayscale 2D images, then reconstructing them using a multi-layer perceptron(MLP). It seems to perform well no matter how many layers I add or remove, even without activation functions which is basically a linear combination of the inputs. there must me something wrong with my implementation. this is what I'm doing:

## Encoding

1. Load image (PIL)
2. normalize image by dividing pixel values by 255
3. partition image into $$B \times B$$ blocks(patches) to get a matrix of size $$P \times B \times B$$ where $$P$$ is the number of blocks.
4. Flatten the blocks to form a matrix of size $$P \times B^2$$ let's call it $$X$$
5. Take M measurements from each block by getting the dot product of $$\Phi$$, an $$M \times B^2$$ Gaussian matrix, and $$X^T$$, the matrix containing the flattened, normalized blocks. The result is the measurement matrix $$Y$$ of size $$P \times M$$

## Decoding

From what i understand, this is a sparse approximation problem. meaning my images(or blocks) are supposed to be sparse in some basis $$\Psi$$ in order to get an acceptable construction. I assumed the basis to be 2D-DCT. Therefore for each block, i try to reconstruct $$S_i$$ which is a $$k$$-sparse vector representing the flattened block in frequency domain. So my optimization problem becomes as follows:
$$\min|S_i|_1 s.t. |\Phi\Psi^{-1} S_i-Y_i|_2 < \mu$$ $$\mu$$ being a small quantity. Now this applies to per-image optimization methods. What i am trying to do is an MLP which learns a universal function $$f(Y_i)$$ for reconstruction. So after training, my decoding process is as follows: $$\hat{S_i} = f(Y_i)$$ following the acquisition of $$\hat{S_i}$$ i apply $$\Psi^{-1}$$ to it in order to get $$\hat{X_i}$$. finally, $$\hat{X_i} * 255$$ gives me the flattened block pixel values. reshaping each $$\hat{X_i}$$ into a $$B \times B$$ matrix then arranging them gives me the full image $$X$$.

## Training

My dataset consists of $$50,000$$, $$256 \times 256$$ images. I just use the first batch (1000 images) since adding more data does not affect my model at all somehow. I do the aforementioned encoding process to every image then concatenate and shuffle all the blocks (i.e instead of images, i now have a matrix of flattened shuffled blocks from multiple images). My model inputs are the measurements taken from these images, of size $$1000*P \times M$$. To get my training labels, i apply 2D-DCT to each unflattened block before taking the measurements (right after encoding step 3 above), then flatten the sparse labels $$1000*P \times B^2$$, let's call the matrix $$S$$. Loss is calculated by taking $$MSE(\hat{S}, S)$$. Number of layers/nodes-per-layer do not seem to affect the model accuracy at all.

## My questions

1. A Linear(no activations) MLP with a single hidden layer performs the same or sometimes better than a Multi-layer model with ReLU activations. What might be the cause of this?
2. In the literature, to guarantee construction $$X$$ has to be sparse in some domain. But i tried removing all transformations from my model and just worked in spatial domain and the model performs exactly the same. shouldn't it struggle to reconstruct the signal?

## Output example

A model with 1 hidden layer having $$B^2$$ nodes takes the input of shape $$M$$. then the intermediate output is given to the output layer having $$B^2$$ nodes. in short, $$M \rightarrow B^2 \rightarrow B^2$$, no activations. training $$MSE$$ is $$0.002$$. $$10$$ of the blocks are reserved for validation, validation loss is also $$0.002$$. $$B = 8$$, M = $$0.25 * B^2$$. Now trying the trained model on an unseen images like "lena.bmp" $$(512 \times 512)$$ gives the following results compared to OMP:

N.B. The same $$\Phi$$ used in training is used in testing.

Any help is appreciated.