Mathematics

Questions tagged [scalar-fields]

A scalar field is a function of the type $X\to \Bbb R$, where $X$ may be an open set in $\mathbb R^n$ or more generally a smooth manifold.

388 questions
186
votes
15 answers

Why is gradient the direction of steepest ascent?

$$f(x_1,x_2,\dots, x_n):\mathbb{R}^n \to \mathbb{R}$$ The definition of the gradient is $$ \frac{\partial f}{\partial x_1}\hat{e}_1 +\ \cdots +\frac{\partial f}{\partial x_n}\hat{e}_n$$ which is a vector. Reading this definition makes me consider…
Jing
  • 2,567
116
votes
5 answers

Difference between gradient and Jacobian

Could anyone explain in simple words (and maybe with an example) what the difference between the gradient and the Jacobian is? The gradient is a vector with the partial derivatives, right?
Math_reald
  • 1,417
  • 2
  • 12
  • 12
85
votes
4 answers

Gradient of 2-norm squared

Could someone please provide a proof for why the gradient of the squared $2$ -norm of $x$ is equal to $2x$ ? $$\nabla\|x\|_2^2 = 2x$$
user167133
  • 973
84
votes
6 answers

How to take the gradient of the quadratic form?

It's stated that the gradient of: $$\frac{1}{2}x^TAx - b^Tx +c$$ is $$\frac{1}{2}A^Tx + \frac{1}{2}Ax - b$$ How do you grind out this equation? Or specifically, how do you get from $x^TAx$ to $A^Tx + Ax$?
victor
  • 963
43
votes
2 answers

Taking a derivative with respect to a matrix

I'm studying about EM-algorithm and on one point in my reference the author is taking a derivative of a function with respect to a matrix. Could someone explain how does one take the derivative of a function with respect to a matrix...I don't…
jjepsuomi
  • 9,049
38
votes
4 answers

How to calculate the gradient of log det matrix inverse?

How to calculate the gradient with respect to $X$ of: $$ \log \mathrm{det}\, X^{-1} $$ here $X$ is a positive definite matrix, and det is the determinant of a matrix. How to calculate this? Or what's the result? Thanks!
pluskid
  • 1,169
33
votes
2 answers

The gradient as a row versus column vector

Kaplan's Advanced Calculus defines the gradient of a function $f : \mathbb{R^n} \to \mathbb{R}$ as the $1 \times n$ row vector whose entries respectively contain the $n$ partial derivatives of $f$ . By this definition then, the gradient is just the…
ItsNotObvious
  • 11,361
33
votes
4 answers

Geometric interpretation of $\frac {\partial^2} {\partial x \partial y} f(x,y)$

Is there any geometric interpretation for the following second partial derivative? $$f_{xy} = \frac {\partial^2 f} {\partial x \partial y}$$ In particular, I'm trying to understand the determinant from second partial derivative test for determining…
Lie Ryan
  • 1,241
30
votes
5 answers

Gradient of squared Frobenius norm of a matrix

In linear regression, the loss function is expressed as $$ W \mapsto \frac1N \left\| X W - Y \right\|_{\text{F}}^2 $$ where the matrices $X$ and $Y$ are given. Taking the gradient yields $$ W \mapsto \frac 2N \, X^T( X W - Y ) $$ Why is this so?
wrek
  • 499
28
votes
6 answers

Differentiate $f(x)=x^TAx$

Calculate the differential of the function $f: \Bbb R^n \to \Bbb R$ given by $$f(x) = x^T A x$$ with $A$ symmetric. Also, differentiate this function with respect to $x^T$ . How exactly does this work in the case of vectors and matrices? Could…
dreamer
  • 3,479
25
votes
4 answers

Gradient of linear scalar field $X \mapsto \operatorname{tr}(AXB)$

Could someone explain the following? $$ \nabla_X \operatorname{tr}(AXB) = BA $$ I understand that $$ {\rm d} \operatorname{tr}(AXB) = \operatorname{tr}(BA \; {\rm d} X) $$ but I don't quite understand how to move ${\rm d} X$ out of the trace.
Chris D
  • 411
21
votes
3 answers

Relation between the Hessian matrix and curvature

According to Hessian matrix , It describes the local curvature of a function. AFAIK, for one-variable function $f(x)$ , its local curvature is $$\kappa = \frac{|f''|}{(1 + f'^2)^{3/2}},$$ and its Hessian matrix is $$\mathcal{Hess}(f) = [f''],$$…
avocado
  • 1,279
19
votes
3 answers

What is the derivative of $\log \det X$ when $X$ is symmetric?

According to Appendix A.4.1 of Boyd & Vandenberghe's Convex Optimization , the gradient of $f(X):=\log \det X$ is $$\nabla f(X) = X^{-1}$$ The domain of the $f$ here is the set of symmetric matrices $\mathbf S^n$ . However, according to the book…
evangelos
  • 469
15
votes
3 answers

Does strictly convex imply invertible gradient?

If $f:\mathbb R^n \to \mathbb R$ is strictly convex and continuously differentiable, does this imply that $\nabla f$ is a one-to-one mapping? To be precise, can we say that $x, y \in \mathbb R^n$ and $\nabla f(x) = \nabla f(y)$ implies $x = y$ ?
Garrett
  • 1,671
14
votes
2 answers

Why does the Hessian work?

I am working through Leonard Susskind's The Theoretical Minimum (on physics, but it also includes some maths). In particular, there is an interlude for which he discusses partial differentiation. He discusses a surface, a function of two variable.…
lagrange103
  • 711
1
2 3
…
25 26