It is widely known that if you were to calculate the maximizer of the dual SVM program (denote as $\alpha^*$), then the primal minimizer of the hard-margin SVM program,

\begin{aligned}&{\underset {{w} ,\;b }{\operatorname {minimize} }}&&\|{w} \|_{2}^{2}\\&{\text{s.t.}}&&y_{i}({w} ^{\top }{x} _{i}-b)\geq 1\quad \forall i\in \{1,\dots ,N\}\end{aligned}

can be found through the KKT conditions, in which case,

$$w^*= \sum_{i = 1}^N \alpha^*_i y_i x_i$$ In particular, those $x_i$ corresponding to non-zero $\alpha^*_i$ are the famous support vectors.

However, a technical detail that people seem to ignore is that this only holds if strong duality also holds.

I struggled with verifying strong duality, so I searched widely online. Essentially, online reference argue that Slater's condition guarantees strong duality always hold for hard-margin SVM.

Slater condition in this case can be written as either:

1. There exists a pair $(w,b)$ that satisfies $y_{i}({w} ^{\top }{x} _{i}-b)> 1$ for all $i = 1, \ldots, N$, or
2. There exists a pair $(w,b)$ that satisfies $y_{i}({w} ^{\top }{x} _{i}-b)\geq 1$, for all $i = 1, \ldots, N$ The latter is because the constraint functions are affine.

If Slater holds, then strong duality holds and we can make the claim $$w^*= \sum_{i = 1}^N \alpha^* y_i x_i.$$ If it doesn't, then this relationship does not hold.

I do not think Slater always holds despite many answers in the contrary online (virtually all of them). I suspect they are making a mistake. I think you can only assume Slater holds, but cannot claim that it always holds.

My reasoning: hard-margin SVM can easily have no solution. On any non-linearly separable dataset, hard-margin SVM does not have a solution. So how then can we write $w^*= \sum_{i = 1}^N \alpha^* y_i x_i? $

An alternative argument I've is that if dataset is linearly separable, then Slater holds for hard-margin SVM. I think that's also a mistake, as the Slater's condition is much more stringent than the linear separability condition, which simply requires RHS to be greater than $0$, i.e., $y_{i}({w} ^{\top }{x} _{i}-b)> 0.$

But then again, strong duality can hold without Slater's. So I am confused as whether strong duality holds for all hard-margin SVM problems. I hope someone can help clarifying this.

References:

https://www.cs.cmu.edu/~yifengt/courses/machine-learning/slides/lecture2-kernel-machine-v1.0.pdf ("it’s easy to see Slater’s condition satisfied")

https://www.gatsby.ucl.ac.uk/~gretton/coursefiles/Slides5A.pdf ("Slater's condition trivially hold")

Among many others.