No I would not suggest using symbollic math at all...
Matlab internally uses 64-bit IEEE binary64 (CPU hardware supported) numerical data format for all arithmetic operations including FFT function. Even at 64-bits there is a limit of precision and accumulation of errors.
You can consider the followings to increase (if possible) the precison of your numerical format while losing computational efficiency.
Fundamentally, you can try using a software emulated FPU library with, say, 256 bit, or 512-bit number formats. This will be much slower than the hardware supported 64-bit computations, but still much faster than the symbolic math packages. How to handle this using Matlab is another issue.
Instead of an emulated FPU librray, may be you can consider fixed point integer based implementation which can be much faster than an emulated FPU library. Harder to realize, but it can save a lot. Matlab has some integer support and fixed-point blockset, does that help ?
Furthermore, you can also consider using new features of CPU's such as SSE4, AVX1, AVX2 which are added into intel/amd architectures for vector-parallel operations. May be they would help accelerate wider bit operations than 64-bits (I'm not sure on this.). And even if this is possible how you would integrate this into Matlab is also not easy... (C-Mex files might be needed)