|March 19th, 2017, 03:12 AM||#1|
Joined: Mar 2017
I've not touched calculus post my maths degree many years ago.
I need to calculate a gradient, to use in back-propagation in a neural network, but it's making my head hurt!
I have the formula
δ(t) = tanh[ <w, f(t) > + b + uδ(t−1) ]
and need to calculate
dδ(t)/dθ = ∂δ(t)/∂θ + ∂δ(t)/∂δ(t−1) * dδ(t-1)/dθ
The derivative of tanh(x) is 1 - tanh^2(x), so I assume the partial derivative,
∂δ(t)/dδ(t−1) = 1 - tanh^2[ <w, f(t) > + b + uδ(t−1) ]
I'm less sure about the first element, given the θ decomposition into its constituent elements.
Last edited by skipjack; March 19th, 2017 at 04:00 PM.
|Thread||Thread Starter||Forum||Replies||Last Post|
|Bending stresses||greeny93||Physics||0||December 11th, 2016 10:54 AM|
|hydro-bending setup||sarah208||Trigonometry||3||January 7th, 2015 12:38 AM|
|another mind twister||mathkid||Calculus||9||November 27th, 2012 08:58 PM|
|never mind, sorry||ElMarsh||Real Analysis||1||October 24th, 2009 06:32 PM|
|never mind - i got it||tomotomo||Algebra||1||July 30th, 2008 05:37 AM|