From Simon Haykin's Adaptive Filter Theory: consider the characteristic equation is $1+𝑎_1𝑧^{−1}+𝑎_2𝑧^{−2}=0$, then for the roots to be inside the unit circle (i.e. in the unit disk), the coefficients of the quadratic must satisfy the following conditions: $-1\le a_1+a_2$, $-1\le a_2-a_1$, and $-1\le a_2\le 1$.
I could figure this out some of the way. For complex roots it follows that the coefficients lie in the region defined by the intersection of $4a_2>a_1^2$ and $a_2<1$. For real roots, I could prove that $|a_2|<1$, and obviously $4a_2\le a_1^2$. But I cannot figure out how the other two line boundaries of the region can be derived. Can someone please explain how?
An approach for the case with complex roots is shown here: https://math.stackexchange.com/questions/1839654/show-the-roots-of-the-quadratic-equation-z2-bz-c-0-lie-in-or-on-the-unit
I've also asked this question here: https://math.stackexchange.com/questions/1839654/show-the-roots-of-the-quadratic-equation-z2-bz-c-0-lie-in-or-on-the-unit, although there is a typo in that posting in the third condition specifying the boundary of the region of the coefficients, and I don't have enough reputation there to fix it.