黑塞矩陣

海森矩陣（德語：Hesse-Matrix；英語：Template:Lang 或 Template:Lang），又譯作黑塞矩阵、海塞（赛）矩陣或海瑟矩陣等，是一個由多變量實值函數的所有二階偏導數組成的方陣，由德國數學家奧托·黑塞引入並以其命名。

定義

假設有一實值函數$f(x_1,x_2,\dots ,x_n)$，如果 $f$的所有二階偏導數都存在並在定義域內連續，那麼函數$f$的黑塞矩陣為

$$𝐇=\left[\begin{array}{cccc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_1\partial x_n}\\ \\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}& \cdots & \frac{{\partial }^{2}f}{\partial x_2\partial x_n}\\ \\ ⋮& ⋮& \ddots & ⋮\\ \\ \frac{{\partial }^{2}f}{\partial x_n\partial x_1}& \frac{{\partial }^{2}f}{\partial x_n\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_n^2}\end{array}\right]$$

或使用下標記號表示為

$${𝐇}_{ij}=\frac{{\partial }^{2}f}{\partial x_i\partial x_j}$$

顯然黑塞矩陣 $𝐇$是一個$n\times n$方陣。黑塞矩陣的行列式被稱爲黑塞式（英語：Template:Lang），而需注意的是英語環境下使用Hessian一詞時可能指上述矩陣也可能指上述矩陣的行列式。

性質

由高等數學知識可知，若一元函數$f(x)$在$x=x_0$點的某個鄰域內具有任意階導數，則函數$f(x)$在$x=x_0$點處的泰勒展開式為

$$f(x)=f(x_0)+f^{\prime }(x_0)\mathrm{\Delta }x+\frac{f^{″}(x_0)}{2!}\mathrm{\Delta }{x}^2+\cdots$$

其中，$\mathrm{\Delta }x=x-x_0$。

同理，二元函數$f(x_1,x_2)$在$x_0(x_{10},x_{20})$點處的泰勒展開式為

$$f(x_1,x_2)=f(x_{10},x_{20})+f_{x_1}(x_{10},x_{20})\mathrm{\Delta }{x}_1+f_{x_2}(x_{10},x_{20})\mathrm{\Delta }{x}_2+\frac{1}{2}[f_{x_1{x}_1}(x_{10},x_{20})\mathrm{\Delta }{x}_1^2+2f_{x_1{x}_2}(x_{10},x_{20})\mathrm{\Delta }{x}_1\mathrm{\Delta }{x}_2+f_{x_2{x}_2}(x_{10},x_{20})\mathrm{\Delta }{x}_2^2]+\cdots$$

其中，$\mathrm{\Delta }{x}_1=x_1-x_{10}$，$\mathrm{\Delta }{x}_2=x_2-x_{20}$，$f_{x_1}=\frac{\partial f}{\partial x_1}$，$f_{x_2}=\frac{\partial f}{\partial x_2}$，$f_{x_1{x}_1}=\frac{{\partial }^{2}f}{\partial x_1^2}$，$f_{x_2{x}_2}=\frac{{\partial }^{2}f}{\partial x_2^2}$，$f_{x_1{x}_2}=\frac{{\partial }^{2}f}{\partial x_1\partial x_2}=\frac{{\partial }^{2}f}{\partial x_2\partial x_1}$。

將上述展開式寫成矩陣形式，則有

$$f(x)=f(x_0)+\mathrm{\nabla }f(x_0{)}^{\mathrm{T}}\mathrm{\Delta }x+\frac{1}{2}\mathrm{\Delta }{x}^{\mathrm{T}}G(x_0)\mathrm{\Delta }x+\cdots$$

其中，$\mathrm{\Delta }x=\left[\begin{array}{c}\mathrm{\Delta }{x}_1\\ \\ \mathrm{\Delta }{x}_2\end{array}\right]$，$\mathrm{\Delta }{x}^{\mathrm{T}}=\left[\begin{array}{cc}\mathrm{\Delta }{x}_1& \mathrm{\Delta }{x}_2\end{array}\right]$是$\mathrm{\Delta }x$的轉置，$\mathrm{\nabla }f(x_0)=\left[\begin{array}{c}\frac{\partial f}{\partial x_1}\\ \\ \frac{\partial f}{\partial x_2}\end{array}\right]$是函數$f(x_1,x_2)$在$x_0(x_{10},x_{20})$的梯度，矩陣

$$G(x_0)={\left[\begin{array}{cc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}\\ \\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}\end{array}\right]}_{x_0}$$

即函數$f(x_1,x_2)$在$x_0(x_{10},x_{20})$點處的$2\times 2$黑塞矩阵。它是由函数$f(x_1,x_2)$在$x_0(x_{10},x_{20})$点处的所有二階偏導數所組成的方陣。

由函數的二次連續性，有

$$\frac{{\partial }^{2}f}{\partial x_1\partial x_2}=\frac{{\partial }^{2}f}{\partial x_2\partial x_1}$$

所以，黑塞矩陣$G(x_0)$为對稱矩陣。

將二元函數的泰勒展開式推廣到多元函數，函數$f(x_1,x_2,\cdots ,x_n)$在$x_0(x_1,x_2,\cdots ,x_n)$點處的泰勒展開式為

$$f(x)=f(x_0)+\mathrm{\nabla }f(x_0{)}^{\mathrm{T}}\mathrm{\Delta }x+\frac{1}{2}\mathrm{\Delta }{x}^{\mathrm{T}}G(x_0)\mathrm{\Delta }x+\cdots$$

其中，$$\mathrm{\nabla }f(x_0)={\left[\begin{array}{cccc}\frac{\partial f}{\partial x_1}& \frac{\partial f}{\partial x_2}& \cdots & \frac{\partial f}{\partial x_n}\end{array}\right]}_{x_0}^T$$ 為函數$f(x)$在$x_0(x_1,x_2,\cdots ,x_n)$點的梯度，

$$G(x_0)={\left[\begin{array}{cccc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_1\partial x_n}\\ \\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}& \cdots & \frac{{\partial }^{2}f}{\partial x_2\partial x_n}\\ \\ ⋮& ⋮& \ddots & ⋮\\ \\ \frac{{\partial }^{2}f}{\partial x_n\partial x_1}& \frac{{\partial }^{2}f}{\partial x_n\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_n^2}\end{array}\right]}_{x_0}$$

為函數$f(x)$在$x_0(x_1,x_2,\cdots ,x_n)$點的$n\times n$黑塞矩陣。若函數有$n$次連續性，則函數的$n\times n$黑塞矩陣是對稱矩陣。

說明：在優化設計領域中，黑塞矩陣常用$G$表示，且梯度有時用$g$表示。

函數$f$的黑塞矩陣和雅可比矩陣有如下關係：

$$\mathrm{H}(f)=\mathrm{J}(\mathrm{\nabla }f{)}^T$$

即函數$f$的黑塞矩陣等於其梯度的雅可比矩陣。

應用

函數的極值條件

對於一元函数$f(x)$，在給定區間內某$x=x_0$點處可導，並在$x=x_0$點處取得極值，其必要條件是

$$f^{\prime }(x_0)=0$$

即函數$f(x)$的極值必定在駐點處取得，或者說可導函數$f(x)$的極值點必定是駐點；但反過來，函數的駐點不一定是極值點。檢驗駐點是否為極值點，可以採用二階導數的正負號來判斷。根據函數$f(x)$在$x=x_0$點處的泰勒展開式，考慮到上述極值必要條件，有

$$f(x)=f(x_0)+\frac{f^{″}(x_0)}{2!}\mathrm{\Delta }{x}^2+\cdots$$

若$f(x)$在$x=x_0$點處取得極小值，則要求在$x=x_0$某一鄰域內一切點$x$都必須滿足

$$f(x)-f(x_0)>0$$

即要求

$$\frac{f^{″}(x_0)}{2!}\mathrm{\Delta }{x}^2>0$$

亦即要求

$$f^{″}(x_0)>0$$

$f(x)$在$x=x_0$點處取得極大值的討論與之類似。於是有極值充分條件：

設一元函数$f(x)$在$x=x_0$點處具有二階導數，且$f^{\prime }(x_0)=0$，$f^{″}(x_0)\ne 0$，則

1. 當$f^{″}(x_0)>0$時，函數$f(x)$在$x=x_0$處取得極小值；
2. 當$f^{″}(x_0)<0$時，函數$f(x)$在$x=x_0$處取得極大值。

而當$f^{″}(x_0)=0$時，無法直接判斷，還需要逐次檢驗其更高階導數的正負號。由此有一个規律：若其開始不為零的導數階數為偶數，則駐點是極值點；若為奇數，則為拐點，而不是極值點。

對於二元函数$f(x_1,x_2)$，在給定區域內某$x_0(x_{10},x_{20})$點處可導，並在$x_0(x_{10},x_{20})$點處取得極值，其必要條件是

$$f_{x_1}(x_0)=f_{x_2}(x_0)=0$$

即

$$\mathrm{\nabla }f(x_0)=0$$

同樣，這只是必要條件，要進一步判斷$x_0(x_{10},x_{20})$是否為極值點需要找到取得極值的充分條件。根據函數$f(x_1,x_2)$在$x_0(x_{10},x_{20})$點處的泰勒展開式，考慮到上述極值必要條件，有

$$f(x_1,x_2)=f(x_{10},x_{20})+\frac{1}{2}[f_{x_1{x}_1}(x_0)\mathrm{\Delta }{x}_1^2+2f_{x_1{x}_2}(x_0)\mathrm{\Delta }{x}_1\mathrm{\Delta }{x}_2+f_{x_2{x}_2}(x_0)\mathrm{\Delta }{x}_2^2]+\cdots$$

設$A=f_{x_1{x}_1}(x_0)$，$B=f_{x_1{x}_2}(x_0)$，$C=f_{x_2{x}_2}(x_0)$，則

$$f(x_1,x_2)=f(x_{10},x_{20})+\frac{1}{2}[A\mathrm{\Delta }{x}_1^2+2B\mathrm{\Delta }{x}_1\mathrm{\Delta }{x}_2+C\mathrm{\Delta }{x}_2^2]+\cdots$$

或

$$f(x_1,x_2)=f(x_{10},x_{20})+\frac{1}{2A}[(A\mathrm{\Delta }{x}_1+B\mathrm{\Delta }{x}_2{)}^2+(AC-B^2)\mathrm{\Delta }{x}_2^2]+\cdots$$

若$f(x_1,x_2)$在$x_0(x_{10},x_{20})$點處取得極小值，則要求在$x_0(x_{10},x_{20})$某一鄰域內一切點$x$都必須滿足

$$f(x_1,x_2)-f(x_{10},x_{20})>0$$

即要求

$$\frac{1}{2A}[(A\mathrm{\Delta }{x}_1+B\mathrm{\Delta }{x}_2{)}^2+(AC-B^2)\mathrm{\Delta }{x}_2^2]>0$$

亦即要求$A>0$，$AC-B^2>0$

即
$${\frac{{\partial }^{2}f}{\partial x_1^2}|}_{x_0}>0$$

$${\left[\begin{array}{c}\frac{{\partial }^{2}f}{\partial x_1^2}\frac{{\partial }^{2}f}{\partial x_2^2}-(\frac{{\partial }^{2}f}{\partial x_1\partial x_2}{)}^2\end{array}\right]}_{x_0}>0$$

此條件反映了$f(x_1,x_2)$在$x_0(x_{10},x_{20})$點處的黑塞矩陣$G(x_0)$的各階主子式都大於零，即對於

$$G(x_0)={\left[\begin{array}{cc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}\\ \\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}\end{array}\right]}_{x_0}$$

要求
$${\frac{{\partial }^{2}f}{\partial x_1^2}|}_{x_0}>0$$

$$|G(x_0)|={\left|\begin{array}{cc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}\\ \\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}\end{array}\right|}_{x_0}>0$$

$f((x_1,x_2)$在$x_0(x_{10},x_{20})$點處取得極大值的討論與之類似。於是有極值充分條件：

設二元函数$f(x_1,x_2)$在$x_0(x_{10},x_{20})$點的鄰域內連續且具有一階和二階連續偏導數，又有$f_{x_1}(x_0)=f_{x_2}(x_0)=0$，同時令$A=f_{x_1{x}_1}(x_0)$，$B=f_{x_1{x}_2}(x_0)$，$C=f_{x_2{x}_2}(x_0)$，則

1. 當$A>0$，$AC-B^2>0$時，函數$f(x_1,x_2)$在$x_0(x_{10},x_{20})$處取得極小值；
2. 當$A<0$，$AC-B^2>0$時，函數$f(x_1,x_2)$在$x_0(x_{10},x_{20})$處取得極大值。

此外可以判斷，當$AC-B^2<0$時，函數$f(x_1,x_2)$在$x_0(x_{10},x_{20})$點處沒有極值，此點稱爲鞍點。而當$AC-B^2=0$時，無法直接判斷，對此，補充一個規律：當$AC-B^2=0$時，如果有$A\equiv 0$，那麼函數$f(x_1,x_2)$在$x_0(x_{10},x_{20})$有極值，且當$C>0$有極小值，當$C<0$有極大值。

由線性代數的知識可知，若矩陣$G(x_0)$滿足
$${\frac{{\partial }^{2}f}{\partial x_1^2}|}_{x_0}>0$$

$${\left|\begin{array}{cc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}\\ \\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}\end{array}\right|}_{x_0}>0$$

則矩陣$G(x_0)$是正定矩陣，或者說矩陣$G(x_0)$正定。

若矩陣$G(x_0)$滿足
$${\frac{{\partial }^{2}f}{\partial x_1^2}|}_{x_0}<0$$

$${\left|\begin{array}{cc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}\\ \\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}\end{array}\right|}_{x_0}>0$$

則矩陣$G(x_0)$是負定矩陣，或者說矩陣$G(x_0)$負定。

於是，二元函數$f(x_1,x_2)$在$x_0(x_{10},x_{20})$點處取得極值的條件表述為：二元函數$f(x_1,x_2)$在$x_0(x_{10},x_{20})$點處的黑塞矩陣正定，則取得極小值；在$x_0(x_{10},x_{20})$點處的黑塞矩陣負定，則取得極大值。

對於多元函數$f(x_1,x_2,\cdots ,x_n)$，若在$x_0(x_1,x_2,\cdots ,x_n)$點處取得極值，則極值存在的必要條件為

$$\mathrm{\nabla }f(x_0)={\left[\begin{array}{cccc}\frac{\partial f}{\partial x_1}& \frac{\partial f}{\partial x_2}& \cdots & \frac{\partial f}{\partial x_n}\end{array}\right]}_{x_0}^T=0$$

取得極小值的充分條件為

$$G(x_0)={\left[\begin{array}{cccc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_1\partial x_n}\\ \\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}& \cdots & \frac{{\partial }^{2}f}{\partial x_2\partial x_n}\\ \\ ⋮& ⋮& \ddots & ⋮\\ \\ \frac{{\partial }^{2}f}{\partial x_n\partial x_1}& \frac{{\partial }^{2}f}{\partial x_n\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_n^2}\end{array}\right]}_{x_0}$$

正定，即要求$G(x_0)$的各階主子式都大於零，即
$${\frac{{\partial }^{2}f}{\partial x_1^2}|}_{x_0}>0$$

$${\left|\begin{array}{cc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}\\ \\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}\end{array}\right|}_{x_0}>0$$

$$⋮$$

$$|G(x_0)|>0$$
取得極大值的充分條件為

$$G(x_0)={\left[\begin{array}{cccc}\frac{{\partial }^{2}f}{\partial x_1^2}& \frac{{\partial }^{2}f}{\partial x_1\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_1\partial x_n}\\ \\ \frac{{\partial }^{2}f}{\partial x_2\partial x_1}& \frac{{\partial }^{2}f}{\partial x_2^2}& \cdots & \frac{{\partial }^{2}f}{\partial x_2\partial x_n}\\ \\ ⋮& ⋮& \ddots & ⋮\\ \\ \frac{{\partial }^{2}f}{\partial x_n\partial x_1}& \frac{{\partial }^{2}f}{\partial x_n\partial x_2}& \cdots & \frac{{\partial }^{2}f}{\partial x_n^2}\end{array}\right]}_{x_0}$$

負定。

拓展閱讀

• 雅可比矩陣
• 梯度