My Math Forum Derivative Mind-bending!

 Calculus Calculus Math Forum

 March 19th, 2017, 03:12 AM #1 Newbie   Joined: Mar 2017 From: uk Posts: 1 Thanks: 0 Derivative Mind-bending! Hi all, I've not touched calculus post my maths degree many years ago. I need to calculate a gradient, to use in back-propagation in a neural network, but it's making my head hurt! I have the formula Code: δ(t) = tanh[ + b + uδ(t−1) ] where t, t-1 represent time steps and need to calculate Code: dδ(t)/dθ = ∂δ(t)/∂θ + ∂δ(t)/∂δ(t−1) * dδ(t-1)/dθ where θ comprises the parameter set [w, b, u]. b and u are scalars, but w is a vector of multiple elements, e.g. w1, w2 etc.. The derivative of tanh(x) is 1 - tanh^2(x), so I assume the partial derivative, Code: ∂δ(t)/dδ(t−1) = 1 - tanh^2[ + b + uδ(t−1) ] The final element (dδ(t-1)/dθ) comes from recursively calculating the gradient, so I'm happy with that. I'm less sure about the first element, given the θ decomposition into its constituent elements. Any ideas?! Last edited by skipjack; March 19th, 2017 at 04:00 PM.

 Thread Tools Display Modes Linear Mode

 Similar Threads Thread Thread Starter Forum Replies Last Post greeny93 Physics 0 December 11th, 2016 10:54 AM sarah208 Trigonometry 3 January 7th, 2015 12:38 AM mathkid Calculus 9 November 27th, 2012 08:58 PM ElMarsh Real Analysis 1 October 24th, 2009 06:32 PM tomotomo Algebra 1 July 30th, 2008 05:37 AM

 Contact - Home - Forums - Top