# Lagrange Multiplier

A technique for constrained optimisation.

# 1 Technique

See Wikipedia: Lagrange multiplier.

# 2 Examples

Notation: using capitals for vectors, lower case for scalars, and subscripts for vector components.

# 2.1 Nearest Point On Perpendicular Bisector

Find the distance dd from a given point XX to the nearest point YY on the perpendicular bisector of two other given points AA and BB.

(Practical use: drawing Voronoi tesselation using distance estimate, when AA and BB are the two nearest centers to XX.)

Let the function to be minimized be f(Y)=d2=(xiyi)2f(Y) = d^2 = \sum (x_i - y_i)^2 then (gradf)i=2(xiyi).(\operatorname{grad} f)_i = -2 (x_i - y_i).

Let the midpoint of A,BA,B be M=(A+B)/2M = (A + B)/2 and the normal of the bisector be N=(AB)/2N = (A - B)/2. Then let the constraint on YY be g(Y)=(YM)N=(yimi)nig(Y) = (Y - M) \cdot N = \sum (y_i - m_i) n_i with (gradg)i=ni.(\operatorname{grad} g)_i = n_i.

By the method of Lagrange multiplier, solve simultaneously gradf=λgradgg=0\operatorname{grad} f = \lambda \operatorname{grad} g \quad\quad g = 0 that is 2(xiyi)=λni(yimi)ni=0.-2(x_i - y_i) = \lambda n_i\quad\quad\sum(y_i-m_i)n_i = 0.

In matrix form using 4D with μ=λ/2\mu = \lambda / 2, that gives

(1n11n21n31n4n1n2n3n40)(y1y2y3y4μ)=(x1x2x3x4NM)\begin{pmatrix} 1 & & & & -n_1 \\ & 1 & & & -n_2 \\ & & 1 & & -n_3 \\ & & & 1 & -n_4 \\ n_1 & n_2 & n_3 & n_4 & 0 \end{pmatrix} \begin{pmatrix} y_1 \\ y_2 \\ y_3 \\ y_4 \\ \mu \end{pmatrix} = \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ N \cdot M \end{pmatrix}

Subtracting from the bottom row, n1n_1 times the top row, n2n_2 times the second row, and so on (Gaussian elimination by elementary row operations), gives the equivalent triangular system:

(1n11n21n31n4NN)(y1y2y3y4μ)=(x1x2x3x4N(MX))\begin{pmatrix} 1 & & & & -n_1 \\ & 1 & & & -n_2 \\ & & 1 & & -n_3 \\ & & & 1 & -n_4 \\ & & & & N \cdot N \\ \end{pmatrix} \begin{pmatrix} y_1 \\ y_2 \\ y_3 \\ y_4 \\ \mu \end{pmatrix} = \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ N \cdot (M - X) \end{pmatrix}

Solving by back substitution, μ=N(MX)NNY=X+μN\mu = \frac{N \cdot (M - X)}{N \cdot N}\quad\quad Y = X + \mu N

Therefore d2=|YX|2=|μN|2=μ2(NN)=(N(MX))2NNd^2 = |Y - X|^2 = |\mu N|^2 = \mu^2 (N \cdot N) = \frac{(N \cdot (M - X))^2}{N \cdot N}

Checking the result, geometric considerations indicate the line XYXY should be perpendicular to the perpendicular bisector, so vector projection can be used: Y=X+(MX)NNNNY = X + \frac{(M - X) \cdot N}{N \cdot N} N This is the same answer, which is good. The dance with Lagrange multipliers may be more useful in other circumstances.