This is a problem from Multirate Systems and Filter Banks by P. P. Vaidyanathan. In Section 4.6.2.E, there is a discussion on what he calls Euclidean Complementary Functions. Specifically, he discusses one of the results of Euclid's theorem:
... if $H_0(z)$ and $H_1(z)$ are relatively prime, there exists polynomials $F_0(z)$ and $F_1(z)$ such that
$$H_0(z)F_0(z) + H_1(z)F_1(z) = c, c \neq 0.$$
In the text, he uses $H_k(z)$ and $F_k(z)$ to denote the analysis and synthesis filters, respectively. He uses $E_k(z)$ and $R_k(z)$ to denote the Type I and Type II polyphase components of the analysis and synthesis filter banks, so I'll do the same here.
If you take the polyphase decomposition of $H_0(z)$, we find that $E_0(z) = 1 + 3z^{-1} + z^{-2}$ and $E_1(z) = 2 + 2z^{-1}$. Because $E_0(z)$ and $E_1(z)$ are relatively prime, we can find an $R_0(z)$ and an $R_1(z)$ to fulfill
$$E_0(z)R_0(z) + E_1(z)R_1(z) = c, c \neq 0.$$
We can recognize that $c$ is the inevitable delay for $\hat{x}(n)$, so we should probably make $c = 1$. To cancel out the $z^{-1}$ terms so that $c = 1$, the order of $R_0(z)$ needs to be one less than that of $R_1(z)$, by inspection. So, we can represent $R_0(z) = \alpha$ and $R_1(z) = \beta + \gamma z^{-1}$, then solve for the coefficients. If we multiply everything out, we'll see that we want
$$\alpha + 2\beta = 1,\\
3\alpha + 2\beta + 2\gamma = 0, \text{and}\\
\alpha + 2\gamma = 0,$$
which is just solving
$$\begin{bmatrix}
1 & 2 & 0 \\
3 & 2 & 2 \\
1 & 0 & 2
\end{bmatrix}
\begin{bmatrix}
\alpha \\
\beta \\
\gamma
\end{bmatrix}
=
\begin{bmatrix}
1 \\
0 \\
0
\end{bmatrix}
$$
to get $\alpha = -1$, $\beta = 1$, and $\gamma = 0.5$. Now, we can solve the problem by composing $F_k(z)$ from the Type II polyphase components: $F_0(z) = z^{-1} R_0(z^2) + R_1(z^2)$ and $F_1(z) = z^{-1} R_0(z^2) - R_1(z^2)$.
There is actually also a nonzero scaling factor of 2 that comes from the DFT filter bank representation, because $\mathbf{W}^*\mathbf{W} = M\mathbf{I}$, where $M = 2$ because we have two channels.
