I stumbled upon an equation system as shown in the image below while designing an orthonormal wavelet filter. I tried solving the equation (5.63,5.64,5.65 => 5.66) but no avail. I'm not sure if there's any property or parameterization method to solve this equation?
The equation below is taken from "Introduction to wavelets and wavelets transform A Primer" page 66