I have a DAC which is assumed to be nonlinear, such that it produces unwanted harmonic distortions at integer multiples of the input frequencies. (**EDIT**: Any other nonlinear distortions, such as intermodulation products, are assumed to be negligible).

If the harmonics fall outside the bandwidth of the analog bandpass filter, then they are assumed to be eliminated:

<img src="https://i.sstatic.net/RZEvj.png" width="500" />

However, if a harmonic falls within the bandpass filter’s passband, then the unwanted distortion remains. In an attempt to avoid this from happening, I can adjust the sample rate of the DAC. (This changes the Nyquist frequency, which changes the frequency of the harmonic after it is aliased into the Nyquist zone of interest).

**Example**: Assume the following:

- Baseband signal bandwidth = 1 GHz.
- Center frequency = 5.5 GHz.
- Bandpass filter passband is 5 GHz - 6 GHz (exactly covering the signal).
- 2nd and 3rd order harmonics are nonzero, but all higher order harmonics are negligible.
- The DAC supports up to 10 GS/s.

If we try sampling at **8 GS/s**, then the 2nd order harmonics (HD2) wrap into the signal band:

<img src="https://i.sstatic.net/Nta3Q.png" width="500" />

If we try sampling at **10 GS/s**, then the 3rd order harmonics (HD3) wrap into the signal band:

<img src="https://i.sstatic.net/xWVuC.png" width="500" />

However, **9 GS/s** appears to be a perfect choice in this case. Neither HD2 nor HD3 are wrapped into the signal band:

<img src="https://i.sstatic.net/Us9wH.png" width="500" />

My problem is that I found this result by brute force. It's not obvious to me how to analyze these harmonics more generally because I don't know how to characterize the aliasing in a convenient way. Could anyone offer any suggestions?

My goal would be to answer more general questions such as (for some maximum sampling rate):

1. *If the signal band is from A MHz to B MHz and the filter passband is from X MHz to Y MHz, then how many harmonics (HD2, then HD3, then HD4, etc) can be avoided (and how)?*
2. *If I want to avoid the first N harmonics (given a signal band from A MHz to B MHz), then what is the widest possible filter passband?*