MATLAB,Python,Scilab,Julia比較 第4章 その78【誤差逆伝播法⑤】

• 誤差逆伝播法の全体像の確認(済)
• 出力層の重みとバイアスを求める誤差からの連鎖律(済)
• 隠れ層の重みとバイアスを求める誤差からの連鎖律(済)
• 上記をプログラミングするための最適化

\(
\displaystyle\frac{\partial E}{\partial W_2}={\color{red}\frac{\partial E}{\partial A_2}\frac{\partial A_2}{\partial Z_2}}\frac{\partial Z_2}{\partial W_2}
\)

\(
\displaystyle\frac{\partial E}{\partial W_2}={\color{red}\frac{\partial E}{\partial A_2}\frac{\partial A_2}{\partial Z_2}}1
\)

\(
\displaystyle\frac{\partial E}{\partial W_2}={\color{red}\frac{\partial E}{\partial A_2}\frac{\partial A_2}{\partial Z_2}}{\color{blue}\frac{\partial Z_2}{\partial A_1}\frac{\partial A_1}{\partial Z_1}}\frac{\partial Z_1}{\partial W_1}
\)

\(
\displaystyle\frac{\partial E}{\partial W_2}={\color{red}\frac{\partial E}{\partial A_2}\frac{\partial A_2}{\partial Z_2}}{\color{blue}\frac{\partial Z_2}{\partial A_1}\frac{\partial A_1}{\partial Z_1}}1
\)

\(
\displaystyle\Delta_2={\color{red}\frac{\partial E}{\partial A_2}\frac{\partial A_2}{\partial Z_2}}
\)

\(
\displaystyle\Delta_1=\Delta_2{\color{blue}\frac{\partial Z_2}{\partial A_1}\frac{\partial A_1}{\partial Z_1}}
\)

\(
\displaystyle\frac{\partial E}{\partial W_2}=\Delta_2\frac{\partial Z_2}{\partial W_2}
\)

\(
\displaystyle\frac{\partial E}{\partial W_1}=\Delta_1\frac{\partial Z_1}{\partial W_1}
\)

\(
\displaystyle\frac{\partial E}{\partial b_2}=\Delta_2 1
\)

\(
\displaystyle\frac{\partial E}{\partial b_1}=\Delta_1 1
\)