The best choice of filter depends on your specific application requirements. There are two basic choices: FIR and IIR. IIR will be much more efficient however, it will results in phase distortions. The phase distortions are completely inaudible (unless it's a bizarre outlier case) but clearly measurable. So it depends whether you can tolerate this our not.
In either case you need to decide how close you need to get to the new Nyquist frequency and how much aliasing noise you can tolerate. A typical example would be that you want the passband to extend to 90% of the new Nyquist frequency and that you would like your aliasing products to be below -80dB. Based on these specifications you can then design the appropriate filter. Other considerations include how much pass band ripple you can accept and if you have any constraints on maximum group delay and/or latency.
Here is an example: Let's say you want to downsample from 44.1 kHz to 32 kHz and the new Nyquist frequency is 16kHz. Going to 90% Nyquist (14400 Hz), with 0.1dB pass band ripple and 80 dB of attenuation at 16 kHz could be done with an elliptical filter of 9th order.
As nibot has pointed out, zeroing FFT bins is a poor choice for a low pass filter since the resulting low pass has very big side lobes and aliasing rejection will be quite poor. It would also require a proper implementation of an overlap-add or overlap-save algorithm to deal with a continuous signal.