How to test I/Q modulation with only one branch (I or Q)?

Question

Given is a direct-conversion I/Q up- and downconverter system. Receiver and transmitter share the same (10MHz) reference and hence the LO frequency is identical and there is an (unknown) phase difference between them. The transmitted RF signal is given as:

$$ x_{\rm rf} = x_i \cos(\omega_c t) + x_q \sin(\omega_c t) $$

Now I look at only one channel (e.g. the I channel) at the receiver (that means that any I/Q imbalance of the receiver is irrelevant). At the I branch, this signal is downconverted with an arbitrary phase shift phi and filtered with H:

$$ y_i = H( x_{\rm rf} \cos(\omega_c t + \phi) ) = \cdots \\ \approx H( \cos(\phi) x_i/2 - \sin(\phi) x_q/2 ) \\ = \frac{\cos(\phi)}{2}H(x_i + x_q \tan \phi) \\ = G(x_i + \alpha x_q) \\ = G(x_i) + \alpha G( x_q) \\ = G_1(x_i) + G_2(x_q) $$

The filter H is contains all filtering effects (mainly anti-aliasing filter).

I want to find a linear relationship between the time-domain samples $y_i[n]$, $x_i[n]$ and $x_q[n]$. A straight forward way is to bring this equation in a vector-/matrix form, model $G_1$ and $G_2$ as FIR filters with $Q$ taps and solve it via Least Squares:

$$ \mathbf{y}_i = \mathbf{X}_i \mathbf{g}_1 + \mathbf{X}_q \mathbf{g}_2 $$

This works nicely and I get the result as expected (in my case, the error between $\mathbf{y}_i$ and the received samples is -45dB). The caveat is I get a set of $2Q$ (real-valued) coefficients but according my derivation, $G_1$ should identical to $G_2$ up to a constant factor. If I would do the same for the $Q$ channel I get again $2Q$ coefficients and a total of $4Q$ real-valued coefficients if I were to model the I/Q relationship that way.

Clearly I can model everything directly in complex domain (with $Q$ complex-valued or $2Q$ real-valued coefficients) but then I cannot look at only the I channel without effect from the Q channel.

Hence a more proper way is to look at this relationship:

$$ \mathbf{y}_i = \mathbf{X}_i \mathbf{g}_1 + \mathbf{X}_q \alpha \mathbf{g}_1 $$

However, this is not a least squares problem any more. For that reason, I sweep $\alpha$ within a reasonable range, compute the LS solution and take the $\alpha$ with the minimum error (this seems to be convex). But if I do this, the error -35 dB (as opposed to -45dB above).

I really do not understand why this approach would not work. Why? And what is the proper way to model this?

Background info: I am testing my own direct conversion receiver. If I transmit data in only one channel (I or Q) and look at only one output channel (e.g. I) I get the performance that I expect based on linearity-, noise- and phase noise measurements of the receiver. If I look at I/Q at the same time (and model the the effect of the receiver as complex-valued FIR filter) then the error is more than 5 dB worse. For debugging I would like to avoid any interaction (such as I/Q imbalance) between I and Q and only look at the I channel output. But as soon as I transmit a full complex-valued input and look only at the I output my result is again at least 5 dB worse than predicted by noise, linearity, phase noise.

As another step, I use an Arbitrary Waveform Generator (Rohde & Schwarz SMW200A) used as transmitter and a VSA (Rohde & Schwarz FSW) as a receiver where I see the same issues. Both instruments are calibrated and the transmitter has superior image rejection. For these reasons I assume I made a conceptional or algebraic mistake in my derivations/assumptions.

Answer 1

The FSW is superheterodyne.

This means that it will not have a DC offset or I/Q imbalance, but the IF path will appear as an asymmetrical lowpass (because it is really a shifted bandpass), which is then compensated for in the calibration.

The first question: have you run the calibration properly?

• leave both machines running for half an hour in a stable environment.
• disable the generator output (OUTP OFF)
• calibrate both (*CAL)

Lots of equipment lenders don't understand that the calibration sequence is used to compensate temperature, run the calibration once (usually on a cold machine) to make the error message go away and tell customers to never touch that button. This gives a fairly bad filter to compensate the IF, and since that filter is applied before downconverting the IF to zero, this will also appear as a complex FIR filter to you.

Second, test your setup with a simple test:

• generate a sine wave (*RST, FREQ 2 GHZ, OUTP ON)
• look at the vector display in IQ Analyzer (*RST, INST IQ, LAY:REPL '1',VECT)

You should see a single stationary dot there, with a position that corresponds to an arbitrary phase angle that is the sum of the phase between the LOs (they are certainly not the same frequency!) and the digital downconversion inside the FSW, which together give a constant, but only as long as all the oscillators keep running at their fixed frequency.

Once you have reached that state, you should be able to repeat your measurement with more complex signals.

Answer 2

I also attempt to answer my own question here based on my findings in Contradiction between complex baseband and real-valued baseband.

There is indeed a flaw in the model that I described and this is that the model is only valid if the only "nonideality" between TX and RX is the LO phase shift.

However, this breaks as soon as there is any filter in the RF path (which is always the case). In this case, the i output is not a linear combination of the I/Q input any more but a linear combination of filtered versions, each filtered with a different filter.

Hence, the proper model is the first suggested one:

$$ \mathbf{y}_i = \begin{bmatrix} \mathbf{X}_i & \mathbf{X}_q \end{bmatrix} \begin{bmatrix} \mathbf{g}_1\\\mathbf{g}_2 \end{bmatrix} $$

while the second one does not have enough degrees of freedom to describe the act of the RF filter on the complex baseband signals.