I read that SVM is a supervised learning method, it is also a non-probabilistic binary linear classifier. I understand why it is binary because it classifies our training pattern to two classes $w_1$ and $w_2$ which are labeled as $y_i=+ 1$ and $-1$. And I understand why it is linear because it classifies when the classes are linearly separable. But I don't understand the non-probabilistic part, could someone clarify?
It can be said that, it is not based on a generative model which models the generation process of the data and itself a probabilistic model. SVM is based on a minimization problem, which seeks for a hyperplane that optimally separates data points of two clusters given the labels. There is nothing probabilistic in this formulation. You just formulate an optimization problem and solve it via Lagrange multipliers.
I heard (but not studied) that, there are probabilistic SVMs, maybe if you study them, you can see the difference.
Note that, at the fundamentals, authors talk about probabilistic notions, such as risk minimization, however at the macro scale, there is no probabilistic notion.