(Ideal) square waves are often drawn in a misleading way, because the vertical lines don't actually represent a signal value. The square wave actually jumps instantanously between two values, creating a discontinuity.
In other words, a function that is defined like this:
$$f(t) = \begin{cases} 0\,\,\,t<0\\1\,\,\,t>=0\end{cases}$$ is discontinuous. Even if you draw a vertical line from 0 to 1 at $t=0$, the vertical line does not actually represent the function: $f(t)$ is never 0.5, or 0.3, or anything besides 0 and 1. Mathematically, the function has a different value when $t$ approaches 0 from the left ($t$ increasing from $-\infty$ to 0) than when approached from the right ($t$ decreasing from $\infty$ to 0).
The triangular wave does not suffer from this problem (but note that its derivative is discontinuous). Mathematically, whether you approach the peak from the left or from the right, you reach the same value.