Optimization in DL#

SGD#

AdaGrad#

RMSProp#

Adam#