I myself recently graduated from Applied Mathematics and began PhD in signal processing. I do Stochastic Geometry modeling of wireless networks in particular, which is quite mathematical subject. It involves measure theory, probability theory, Fourier Analysis etc. etc.
The area of Signal Processing is very broad indeed. It of course depends if you want to do more theoretical or practical stuff, but at least Fourier Analysis is one subject you should get into. It is widely used everywhere, especially in signal processing and image processing etc. Complex analysis raises naturally in Fourier analysis. Circuit Theory is crucial at least if you want to do more "practical" stuff, and get into implementation details. New areas involves far out subjects like Topological Signal Processing. Of course you should understand basic concepts relating to signal propagation and interference. (Check e.g. signal to interference ratio, SINR, and Rayleight fading.) Understanding of Maxwell Equations can be useful as well as they crucially are involved in understanding of antenna patterns at least.
This seems to be a great (and free) book on DSP: https://www.analog.com/en/education/education-library/scientist_engineers_guide.html. I got some good insights at least to Fourier- and Laplace Transform from the book.
As you can tell I am relatively green in this subject, but I hope I gave you at least some insights to the subject.e
EDIT: Added some stuff.