Don't quantize the numerator and denominator coefficients of the total transfer function. Use the coefficients of the second-order sections - sos
in Matlab - and quantize those. You will usually implement an IIR filter using second-order sections, especially in a fixed-point implementation.
The first thing to do is to scale the coefficients of the second-order sections (SOS). Note that you can scale each section individually. Scaling means multiplying the numerator coefficients of each SOS such that you minimize the chance of overflow in that section. The scaling depends on the chosen structure of the SOS, and on the filter coefficients. A detailed discussion on scaling can be found in the textbook Digital Filter Design by Parks and Burrus. If you're interested in the design and implementation of digital filter you should definitely read that book.
After having scaled the coefficients of each SOS, you quantize them by simply rounding them to the closest number on an equidistant grid consisting of $2^N$ numbers, where $N$ is the number of bits.
Example: The coefficients of one of the SOS are given by
b = [1, 1.621783996, 1]
a = [1, -4.030702997e-01, 2.332661953e-01]
You decide to scale the coefficients of that section by a factor of $\frac12$, which of course only affects the numerator coefficients:
bs = [0.5, 8.10891998e-01, 0.5]
You use $N=10$ bits to quantize the coefficients. Since the coefficients are in the range $[-1,1)$, we can multiply them by $2^{N-1}=2^9$, round the result to the nearest integer, and then multiply by $2^{-9}$ to scale back to the original range. In Matlab/Octave this is easily done as follows:
b = [1, 1.621783996, 1];
a = [1, -4.030702997e-01, 2.332661953e-01];
scale = 0.5;
b = b * scale;
N = 10;
range = 2^(N-1);
ac = round( a(2:3) * range ) / range;
bc = round( b * range ) / range;
The quantized coefficients are given by
ac = [-4.0234375e-01, 2.32421875e-01]
and
bc = [0.5, 8.10546875e-01, 0.5]
Or, in $10$ bit binary two's complement form
ac = [1100110010, 0001110111]
bc = [0100000000, 0110011111, 0100000000]