My Math Forum multi-variable functions, Gateaux/Frechet differentiable

 Real Analysis Real Analysis Math Forum

 September 26th, 2013, 12:43 PM #1 Newbie   Joined: Sep 2013 Posts: 8 Thanks: 0 multi-variable functions, Gateaux/Frechet differentiable Regarding directional derivatives, I understand that a function is Gateaux differentiable if it is directionally differentiable in every direction. The text then states the conditions for a frechet differentiable function as: f(.) is frechet differentiable if there exists an f(subx)(x¯) element of R^(1xn), called the frechet derivative, such that: lim x -> x¯ of |f(x) - f(x¯) - f(subx)(x¯)(x - x¯)| / ||x - x¯|| = 0, Now I just don't know how to interpret this, or how I would find the frechet derivative. Then Im asked things like this: 1. Calculate f(subx)(x) and f(subxx)(x) for the following f(x): (i) f(x) = (aTx)(bTx), with a,b elements of R^n I know how to find the directional derivative here for some arbitrary direction but the only place Ive seen the notation f(subx)(x) is in the definition of frechet differentiability. Basically I just don't know what Im being asked here. If anyone could lead me in the right direction Id really appreciate it, thanks
 September 26th, 2013, 12:45 PM #2 Newbie   Joined: Sep 2013 Posts: 8 Thanks: 0 Re: multi-variable functions, Gateaux/Frechet differentiable just to clarify, by aT I mean a transposed..
 September 26th, 2013, 01:56 PM #3 Senior Member   Joined: Dec 2012 Posts: 372 Thanks: 2 Re: multi-variable functions, Gateaux/Frechet differentiable I know typing in Latex is a challenge, but you should try to learn. Whenever a function is Frechet differentiable, then it's also Gateaux differentiable and the two coincide. The converse isn't true and I doubt that you would be tested with a function that is not Frechet differentiable. Given E, F linear spaces and an open subset $U \subseteq E$, let $f : U \rightarrow F$ be Frechet differentiable. For a point $u \in U$, the Frechet derivative at $u$ is often denoted by $df(u)$ and to be very precise, it is a linear map from the tangent space of E at $u$ to the tangent space of F at $f(u)$. However, the tangent space of E (denoted $E_u$) at any of its points is also E. We are often interested in applying the Frechet derivative $df(u)$ to vectors in $E_u$. This is always characterized by $[df(u)]v= \underset{t \rightarrow 0}{lim} \dfrac{f(u + tv) - f(u)}{t}$, which gives the directional derivative of $f$ at $u$ in the direction of $v$. Sometimes in analysis, we make reference to the normalized directional derivative given by $[df(u)]v= \underset{t \rightarrow 0}{lim} \dfrac{f(u + tv) - f(u)}{t||v||}$ which allows for directional derivatives along different scales of a fixed vector to coincide. In the question you posed, the domain and codomain are not specified. I would like you to do so before I make an attempt to solve.
 September 26th, 2013, 04:38 PM #4 Senior Member   Joined: Dec 2012 Posts: 372 Thanks: 2 Re: multi-variable functions, Gateaux/Frechet differentiable Now, I see what you mean. You are specifically in cases of real valued functions defined on $\mathbb{R}^n$. I will first compute the Frechet derivative and then show that it satisfies the differentiability criterion. $f(x)= (a^Tx)(b^Tx) \ \Rightarrow \ df(x) = (b^Tx)a^T + (a^Tx)b^T$ Hence, $\dfrac{f(y) - f(x) - L_y(y - x)}{||y - x||} \ \ (L_y := df(y))\\ = \dfrac{a^Ty b^Ty - a^Tx b^Tx - [(b^Ty)a^T + (a^Ty)b^T](y - x)}{||y - x||} \\= \dfrac{-a^Ty b^Ty - a^Tx b^Tx + a^Tx b^Ty + a^Ty b^Tx }{||y - x||} \\ =\dfrac{a^Ty b^T(x-y) + a^Tx b^T(y-x) }{||y - x||} \\=\dfrac{[a^Tx b^T - a^Ty b^T](y-x) }{||y - x||} \\=\dfrac{[a^T(x - y)b^T ](y-x) }{||y - x||}$. Finally, $\dfrac{|f(y) - f(x) - L_y(y - x)|}{||y - x||} \leq \dfrac{||a^T(x - y)b^T ||.||y-x|| }{||y - x||}= ||a^T(x - y)b^T ||$ which clearly approaches 0 as $x \rightarrow y.$
 September 27th, 2013, 08:55 AM #5 Newbie   Joined: Sep 2013 Posts: 8 Thanks: 0 Re: multi-variable functions, Gateaux/Frechet differentiable Thanks for your response. Im still unclear about a couple things. Can you expand on how you calculated the derivative of that example problem. So far the only method for finding a derivative in $R^n$ (how do you do the actual symbol for the reals in latex?) thats been presented is the directional derivative formula you mentioned in your first post. So when I use it to find $f_x(x)$ in the direction $v$ for $f(x)=(a^Tx)(b^Tx)$ I get $(a^Tx)(b^Tv)+(a^Tv)(b^Tx)$ and then just doing the same for $f_{xx}(x)$ I get $(a^Tv)(b^Tv) + (a^Tv)(b^Tv)$ The problem doesn't mention a directional derivative or state a direction, but I don't see how a derivative in a higher dimension than 2 would not need a direction for it to make sense
 September 27th, 2013, 12:23 PM #6 Senior Member   Joined: Dec 2012 Posts: 372 Thanks: 2 Re: multi-variable functions, Gateaux/Frechet differentiable Yes your computation is on point, you got $[df(x)]v= (a^Tx)(b^Tv) + (a^Tv)(b^Tx) = (a^Tx)(b^Tv) + (b^Tx)(a^Tv).$ We may commute in the last step since $(b^Tx)$ and $(a^Tv)$ are scalars. Observe that we may extract the linear map $df(x)= (a^Tx)b^T + (b^Tx)a^T$ given that this is a linear operator. Why have you computed $f_{xx}(x)$; is this the second derivative? The reason the derivative is applied to a given direction is that you cannot always extract the linear map $df(x)$ as we did in this problem. For an example, try differentiating the function$f : \mathcal{M}_{n \times n}(\mathbb{R}) \ \rightarrow \ \mathcal{M}_{n \times n}(\mathbb{R}) \ ; \ A \mapsto f(A)= A^2$. There are some rules from single variables which extrapolate to differentiation of multivariate functions, such as the product rule and the chain rule, but as expected, multivariable differentiation is more intricate than single variable differentiation. Look for more examples to give you a general overview.
 September 27th, 2013, 02:15 PM #7 Newbie   Joined: Sep 2013 Posts: 8 Thanks: 0 Re: multi-variable functions, Gateaux/Frechet differentiable Thank you for clarifying, that makes perfect sense. Im finding the notation especially confusing in this course. The problem as stated was: Calculate $f_x(x)$ and $f_{xx}(x)$ for the following $f(x)$ At first I took that to mean the second derivative, but pretty sure I was wrong. When asked on a quiz to find the directional derivative at $x$ in the direction $u$ he used the notation $f'(\bar x;u)$. The first time the notation $f_x$ appears is in the frechet definition. I think now he was talking about partial derivatives and a hessian matrix but Im having a hard time following his notes and he doesnt follow the textbook here is the link if you care to take a look: http://math.cos.ucf.edu/~jyong/Optimization.pdf multi variable differentiation starts on 38 and he starts using $f_{x_i}$ to refer to partial derivatives on pg 40 which leads to defining $f_{xx}$ as a matrix of partials but I don't know how I would calculate that for the function I gave..
 September 27th, 2013, 03:45 PM #8 Senior Member   Joined: Dec 2012 Posts: 372 Thanks: 2 Re: multi-variable functions, Gateaux/Frechet differentiable Thanks for the book, Duncan. Perhaps, you could make reference to it in future posts as I have saved it on my computer. The Hessian matrix, which you denote $f_{xx}$ is indeed the second derivative of a real-valued multivariate function. For your problem, I have computed the Hessian to be $[b_ia_j + a_ib_j]_{ij}.$ This is obtained by differentiating the gradient function; $g : \mathbb{R}^n \rightarrow \mathbb{R}^n \ ; \ (x_1 , x_2 , \cdots , x_n) \ \mapsto \ (g^1 , g^2 , \cdots , g^n), \$ where $g^j= \sum _{1 \leq i \leq n}b_ix_ia_j + \sum _{1 \leq i \leq n}a_ix_ib_j$

multiple differentiation frechet

Click on a term to search for related topics.
 Thread Tools Display Modes Linear Mode

 Similar Threads Thread Thread Starter Forum Replies Last Post irunktm Algebra 6 September 7th, 2012 12:42 PM Pollaj145161 Algebra 4 March 21st, 2012 10:10 PM Chasej Calculus 7 September 16th, 2011 12:18 AM person1200 Calculus 1 September 12th, 2010 07:17 PM Th3Locksmith Calculus 4 March 17th, 2009 10:24 AM

 Contact - Home - Forums - Cryptocurrency Forum - Top