I was reading about Digital Cosine Transform (DCT Type-I) and it's use in JPEG. I have the following doubt. My teacher offered an explanation which wasn't very convincing. What property of Digital Cosine Transform makes it attractive for compression of images? Why Digital Sine Tranform is not used? More specifically, what is the intuition behind the usage of DCTs?
A rectangular window of data can have a discontinuity between the front and back edge. An FFT of that window represents that circular edge discontinuity with energy in a bunch of high frequency bins. Take that segment of signal and mirror it around one edge. Note that a circular or periodic extension of this no longer has a discontinuity across the edge. The FFT of that doubled-by-mirroring signal might thus have less information energy in the high bins. Thus there may be more of these high bins that can be "thrown away" by a compression algorithm without major damage (information loss).
A DCT is an FFT of the doubled-by-mirroring product. Since the mirrored intermediate vector is symmetric, some efficiency can be gotten by throwing away any computations only required for the sine components since they will be zero anyway.
A DST would be the FFT of a doubled-by-mirroring-the-inverse signal, which would make the discontinuities worse, which would spread even more information/energy into the bins you would really like to throw away, thus making things far worse for simply but lossy compression.
Unlike DFT, DCT outputs real (non-complex) coefficients. This allows to have smaller outputs (no phase should be stored). Furthermore, it corresponds to a special type of boundary conditions in the DFT that is easily handled by implementations: symmetric signals. This makes 2 arguments in favor of the DCT.
Like DFT, DCT produces outputs with few significant coefficients, hence its use in compression and more generally in sparse algorithms. This just corresponds to the fact that not all the possible spatial frequencies exist in an image, but only a subset (for example, there are usually many horizontal and vertical lines).