Hyperplane

Equation of a Hyperplane

Similarly, to represent a hyperplane in d-dimensions, we need 2 things:

1. \(\vec{w}\) = direction of the hyperplane = vector perpendicular to the hyperplane
2. \(w_0\) = distance from the origin

\[ \pi_d = w_1x_1 + w_2x_2 + \dots + w_dx_d + w_0 = 0\\[10pt] \text{ representing 'w' and 'x' as vectors: } \\[10pt] \mathbf{w} = \begin{bmatrix} w_1 \\ w_2 \\ \vdots \\ w_d \end{bmatrix}_{\text{d×1}}, \mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_d \end{bmatrix}_{\text{d×1}} \\[10pt] \pi_d = \mathbf{w}^\top \mathbf{x} + w_0 = 0 \\[10pt] => \mathbf{w} \cdot \mathbf{x} + w_0 = 0 \\[10pt] \text{ dividing both sides by the Euclidean norm of w: } \Vert \mathbf{w} \Vert_2 \\[10pt] \frac{\mathbf{w} \cdot \mathbf{x}}{\Vert \mathbf{w} \Vert} + \frac{w_0}{\Vert \mathbf{w} \Vert} = 0 \\[10pt] \text { since the vector divided by its magnitude is a unit vector, so: } \frac{\mathbf{w}}{\Vert \mathbf{w} \Vert} = \mathbf{\widehat{w}} \\[10pt] \mathbf{\widehat{w}} = \frac{w_1x_1 + w_2x_2 + \dots + w_dx_d}{\sqrt{w_1^2 + w_2^2 + \dots + w_d^2}} \\[10pt] \mathbf{\widehat{w}} \cdot \mathbf{x} + \frac{w_0}{\Vert \mathbf{w} \Vert} = 0 \\[10pt] \mathbf{\widehat{w}} \text{ : is the direction of the hyperplane } \\[10pt] \frac{w_0}{\Vert \mathbf{w} \Vert} \text{ : is the distance from the origin } \]

Note: There can be only 2 directions of the hyperplane, i.e, direction of a unit vector perpendicular to the hyperplane:

1. Towards the origin
2. Away from the origin

Distance from Origin

If a point ‘x’ is on the hyperplane, then it satisfies the below equation:

\[ \pi_d = \mathbf{w}^\top \mathbf{x} + w_0 = 0 \\ => {\Vert \mathbf{w} \Vert}{\Vert \mathbf{x} \Vert} cos{\theta} + w_0 = 0 \\[10pt] \because d = \text{ distance from the origin } = {\Vert \mathbf{x} \Vert} cos{\theta} \\ => {\Vert \mathbf{w} \Vert}.d = -w_0 \\[10pt] => d = \frac{-w_0}{{\Vert \mathbf{w} \Vert}} \\[10pt] \therefore distance(0, \pi_d) = \frac{-w_0}{\Vert \mathbf{w} \Vert} \]

Key Points:

1. By convention, the direction of the hyperplane is given by a unit vector perpendicular to the hyperplane , i.e, \({\Vert \mathbf{w} \Vert} = 1\), since the direction is only important.
2. \(w_0\) gives the signed perpendicular distance from the origin.
   \(w_0 = 0\) => Hyperplane passes through the origin.
   \(w_0 < 0\) => Hyperplane is in the same direction of unit vector \(\mathbf{\widehat{w}}\) w.r.t the origin.
   \(w_0 > 0\) => Hyperplane is in the opposite direction of unit vector \(\mathbf{\widehat{w}}\) w.r.t the origin.

Half Spaces

A hyperplane divides a space into 2 distinct parts called half-spaces.
e.g.: A 2D hyperplane divided a 3D space into 2 distinct parts.
Similar example in real world: A wall divides a room into 2 distinct spaces.

Positive Half-Space:
A half space that is in the same direction as the unit vector w.r.t the origin.

Negative Half-Space:
A half space that is in the opposite direction as the unit vector w.r.t the origin.

\[ \text { If point 'x' is on the hyperplane, then:} \\[10pt] \pi_d = \mathbf{w}^\top \mathbf{x} + w_0 = 0 \\[10pt] \mathbf{w}^\top \mathbf{x_1} + w_0 > 0 ~or~ \mathbf{w}^\top \mathbf{x_1} + w_0 < 0 \quad ? \\[10pt] \text { Distance of } x_1 < x_1\prime \text{ as } x_1 \text{ is between the origin and the hyperplane and } x_1\prime \text { lies on the hyperplane } \\[10pt] => \tag{1}\Vert \mathbf{x_1} \Vert < \Vert \mathbf{x_1\prime} \Vert \]

\[ \mathbf{w}^\top \mathbf{x_1\prime} + w_0 = 0 \\[10pt] => \tag{2} \Vert \mathbf{w} \Vert \Vert \mathbf{x_1\prime} \Vert cos{\theta} + w_0 = 0 \]

from equations (1) & (2), we can say that:

\[ \Vert \mathbf{w} \Vert \Vert \mathbf{x_1} \Vert cos{\theta} + w_0 < \Vert \mathbf{w} \Vert \Vert \mathbf{x_1\prime} \Vert cos{\theta} + w_0 \]

Everything is same on both the sides except \(\Vert \mathbf{x_1}\Vert\) and \(\Vert \mathbf{x_1\prime\Vert}\), so:

\[ \mathbf{w}^\top \mathbf{x_1} + w_0 < 0 \]

i.e, negative half-space, opposite to the direction of unit vector or towards the origin.
Similarly,

\[ \mathbf{w}^\top \mathbf{x_2} + w_0 > 0 \]

i.e, positive half-space, same as the direction of unit vector or away from the origin.

Equation of distance of any point \(x\prime\) from the hyperplane:

\[ d_{\pi_d} = \frac{\mathbf{w}^\top \mathbf{x\prime} + w_0}{\Vert \mathbf{w}\Vert} = 0 \]