# What math should I study to really understand signal processing?

I am reading an elementary book on signal processing - "Signals and Systems".

It never struck until recently, the math involving signal processing seemingly has a lot more depth than the book presents. I always suspected some elementary linear algebra and mathematical analysis. But now, I am certain it entails functional analysis which is too opaque for me. Purportedly, the book shied away from gifting any clue.

I am afraid this question could become obscure very soon. I would love to receive as much words as is courtesy of the answerer. Please validate my understanding below.

Systems (operations thereof) are believed to be function spaces. A function of this space could represent system action, say T(). Considering the LTI nature, these function spaces are linear, will have some "characteristics" (eigen). T() is a linear map. Due to causal/real nature, T() is a real hermitian map. Signals are modelled as vectors. Harmonics (comp exp) are eigenfunctions of these spaces, and thus eigenvectors of T(). Also, a set of eigenvalues (collectively called spectrum) are associated to each system action. Thus, spectral analysis exploits eigen properties of LTI function spaces. Signal-energy represents the norm functional of this system space. A transfer function is a spectral realization of T(). Earlier methods of system analysis pertaining operational calculus have been superseded by spectral analysis (for LTIs) and state space analysis. In OC, LTI functions along with convolution and addition form an algebraic group.

Typing it out helped. I still can't know if its correct. I would be grateful to receive feedback on it.

Relevant Math.SE post.

• A book which principally expanded the horizons for me " linear system theory " - Zadeh
– user67550
Commented Jan 21 at 18:00
• If you want to give this question long legs, I suggest you figure out how to change the title to something more positive -- ultimately something like "What math should I study to really understand signal processing?" I found real analysis to be very, very helpful. Had I had time I would have studied complex analysis as well. Commented Jan 21 at 19:19
• @TimWescott Just master the residue theorem and you're good to go lol Commented Jan 21 at 20:31
• Calculus, differential equations, complex variables, probability (and random variables and random processes), numerical methods, vectors and matrix theory, and maybe if you're really into it, take a grad school course in functional analysis (which includes metric spaces and Hilbert spaces). Commented Jan 23 at 20:22
• @robertbristow-johnson Thank you. would you suggest Rudin series of books? Could you please suggest some resources on Probability and Stochastics? I have found Cinlar's book. But it requires measure theory to start with. Another book is Goodman's.
– user67550
Commented Jan 24 at 3:20

## 2 Answers

The entire series of books titled Signal Processing and authored by Oppenheim, Willsky, plus some random grad student(s), is designed to give a student who has the typical mathematical knowledge of a 2nd-year EE student a tool kit to do practical work. It does not, and can not, give a full mathematical grounding of why everything works.

In order to accomplish what it does it has to present a lot of the math as axiomatic, or it gives hand-wavy demonstrations (not proofs, by any means) of the underpinning math.

There's two things you can do with this:

1. Accept that it works, and use the tools. Generations of engineers have founded highly successful careers on the Oppenheim, Willsky, and SRGS books. You can just accept the lack of underpinning and take what they say on trust, and base a career on that.
2. Study the applicable math, to the extent you can. This is what I've done, although the last "official" math class I took related to this was real analysis; the rest I've picked up in bits and bobs from graduate-level classes (signal processing and state-space linear systems), and from books.

Really, if you're blocked because you don't feel that you can start until you know everything, you'll never start at all. Just dive in, start doing work, and start learning all the practical bits that don't show up in the hard math at all.

• I must concede with this answer. It indicates the most pragmatic approach. Thanks for slapping some sense into me. The economy, market, my own finances. I do need a job. However, I just cannot let go of this "wish". I dug deeper than I was required to. Now, I can "see" the math and physics behind. I can understand these devices and circuits as "ideas". This comment makes me seem so pretentious. I hate to say this, but I am smitten by this indescribable desire to keep going. The experience has been too illuminating for me. Hedonistic. I can't accept this answer, although I concede.
– user67550
Commented Jan 22 at 4:00
• " Really, if you're blocked because you don't feel that you can start until you know everything, you'll never start at all. Just dive in, start doing work. " - Exactly, what I need. Thanks.
– user67550
Commented Jan 22 at 4:02
• There's no reason you can't stop studying this stuff after you enter the working world. It's not as easy, but you can still do it -- particularly in this age of Internet learning. Commented Jan 22 at 4:42
• @KartikPandey if you're really struck by learning the maths and physics behind, maybe you should study maths or physics instead :) Commented Jan 22 at 14:45

I had the first edition of this book for a course in the second semester of my senior year of my electrical engineering bachelor's program, i.e. my final semester.

The course was called Introduction to Communication Theory and Systems, and its prerequisites were as follows:

• two semester courses of Signals and Linear Systems (I and II)
• Introduction to Applied Probability
• Electrical Circuits
• differential and integral calculus (two semesters)
• ordinary differential equations
• "a knowledge of complex variable theory"