Why do we not seem to care about the nonlinear phase response in certain applications?
Very likely because these certain applications don't care about phase!
Is it possible to correct the phase distortion if we are filtering offline (i.e. not in realtime)?
With another filter that, combined with the distorting filter, has the desired phase properties, yes. The question whether such a filter exists is actually a complicated one and depends on the distorting filter!
Given that we have a model of our filter, can we take the outputted phase distorted signal, and then individually shift the frequencies within the signal?
This is where the complications come from: If you pick "single" frequencies only, you could only represent perfectly periodic, infinite signals (limit of bandwidth-time product in the Fourier transform). So you'll need some kind of system that is "continuous enough".
It also seems like for application where we can't allow phase distortion, we often implement FIR filters. What are some applications where FIR filters would be absolutely necessary? Essentially anything being filtered in realtime?
No; real time isn't the point here.
So, there's many applications where you want a linear system.
Which practically reduces your choices to IIR and FIR filters.
However, the question whether to choose either of these is often one of design possibilities (I personally find it relatively easy to design a FIR with the window methods, and relatively hard to design good IIRs, but that's me, who's grown up in a FIR-friendly environment); it's often also a question of how stable an IIR design can stay under limited bit width assumptions, or how long a FIR becomes.
Overall, the choice of digital filters is a along topic, that actually fills shelves in libraries. It's impossible to just say "we use that", without knowing a lot of the requirements set for a filter.
Having said that, in many cases you just really need something that is easily adaptable. FIRs lend themselves to that excellently, because you'd usually "just" have to exchange some tap values in software, and depending on the actual implementation of the FIR, you might even have something like a continuous response during that exchange.
Note that all questions regarding which type of filter you use typically involve tradeoffs – your filter response isn't perfect, but you can't implement infinite length FIRs anyway. Your latency is low, but you paid for that with a suboptimal suppression in the stop band. Your decimating filter has adjustable decimation, but in exchange its CIC response is rather round.
A lot of these tradeoffs involve the actual thing you need to implement your filter on. You really shouldn't assume that an experienced DSP hardware designer would just take your Matlab FIR and put it in his FPGA/ASIC design. First of all, there's typically no floating point in those designs, so he'd first need to check dynamic range, convert to fixed point, estimate the error incurred by that, adjust for that, and secondly, things that might be hard in matlab might be easier in hardware; changing the path that samples travel based on the sign bit at Megasamples per second? Will totally thrash CPU performance if you constantly exchange taps in CPU caches, but for FPGA designs, that doesn't matter, since the hardware needs to be there, anyway. Using an IIR in software? What's the point, linear access to memory is preferable and caches are fast, so if the filter's significantly long, it'll probably be fastest to use a FFT-based FIR implementation. In hardware, that'd be a horrible waste of space if the same can be done with an IIR of half the length.
Coming back to the question of phase distortion and filter choices: We know it's very simple to test whether a FIR filter has linear phase: iff its taps are symmetric (hermitian in the complex case), then it has linear phase. That is a strong tool, also for designing linear phase systems (don't confuse with linear systems). Approximating linear phase with an IIR can get pretty complicated / impossible, and getting linear phase with a nonlinear filter – I'm a communications engineer; non-linear filters are pretty much frequency mixers, so we, most of the time, avoid those, if we don't intend to drop phase information anyway.
