If you're still new to digital filter design, I would not recommend you to dive into methods for model order reduction. First of all, they are not filter design methods, but they represent a second step to simplify an already existing model/system. Second, these methods are more applicable to control design, where you might have obtained some overly complex model by discretization and/or linearization. The problem of an inappropriate (too high) system order will generally not occur in the design of digital filters, and if it does, you simply design another filter with the same specifications but with a smaller filter order.
It is impossible to explain even just the basics of IIR filter design in one answer, but as a simple guideline, if the desired filter characteristic is one of the four standard characteristics (low pass, high pass, band pass, band stop), and if the phase response is irrelevant, then you're probably best off with a design based on the transformation of an analog prototype filter (Butterworth, Chebyshev, etc.). For non-standard frequency responses, including prescribed phase responses, you will need some non-linear optimization techniques. In some cases it might also be worthwhile to consider FIR filters instead of IIR filters, simply because they are much easier to design, and because they're always guaranteed to be stable.
For a brief overview of digital filter design basics take a look at this answer. This thesis is probably too specialized for you at this point, but you can find some important references in its bibliography.
I highly recommend these two books:
Digital Filter Design, Parks and Burrus
Introduction to Signal Processing, Orfanidis