I've been given the following theorem for the derivative of the determinant of a matrix:
"Let $A\in \mathbb{R}^{n\times n}$ be a square matrix. Then the Fréchet derivative of det$: \mathbb{R}^{n\times n}\to \mathbb{R}$ at $A$ is given by:
d det(A)B = tr(adj(A)B)."
What I don't understand is what $B$ actually is. Where does it come from? Why did the theorem not follow up with "where $B$ is..."
Could anyone please explain this to me?
Thanks in advance!
The Frechet derivative $dF(x)$ of a function $F$ defined on a Banach space $X$ is defined as a (continous) linear functional on $X$, i.e. a mapping $dF(x):X\to\mathbb{R}$, or, more precisely $dF(x)\in X^*$ ($dF(x)$ is an element in the dual space of $X$).
In your case you have $\det:\mathbb{R}^{n\times n}\to \mathbb{R}$, i.e. $X = \mathbb{R}^{n\times n}$. The dual space of $X$ is equal to $X$, and hence $d\det(A)$ is a linear mapping from $\mathbb{R}^{n\times n}$ to $\mathbb{R}$ and it is precisely what you've written: $d\det(A)B = \mathrm{tr}(A^TB)$.
Another way to put it: $B$ is the direction in which you take the derivative, i.e. the limit of $$ \frac{\det(A + tB) - \det(A)}{t} $$ for $t\to 0$.
Note that your function $$ \det: \mathbb{R}^{n \times n } \rightarrow \mathbb{R} $$ Thus its frechet derivative $D\det : \mathbb{R}^{n \times n } \rightarrow \mathit{L} (\mathbb{R}^{n \times n }, \mathbb{R} ) $, where $\mathit{L} (\mathbb{R}^{n \times n }, \mathbb{R} ) $ is the vector space of all linear maps(here it is forms) from $\mathbb{R}^{n \times n } $ to $ \mathbb{R}$. Hence, for any $ A \in \mathbb{R}^{n \times n } $, we have $ D\det(A) \in \mathit{L} (\mathbb{R}^{n \times n }, \mathbb{R} ) $, i.e. $ D\det(A) $ is a linear map $$ D\det(A) : \mathbb{R}^{n \times n } \rightarrow \mathbb{R}$$ Thus for any $B \in \mathbb{R}^{n \times n } $, $ D\det(A) B $ is well defined and is a real number ($\in \mathbb{R}$). Hope this clarify the idea for you, and any question is welcomed.
The derivative may be defined on functions mapping one normed vector space to another. The idea is always that the derivative of a function $f$ at a point $x$ in its domain provides a linear approximation to the behavior of the function. That is, the derivative $f'(x)$ is the linear operator in the approximation formula \[ f(x+h) = f(x) + f'(x)h + E(x;h), \] where $E$ is an error function of order $||h||^2$.
In one-dimensional calculus, $f$ maps the real line into itself, so $f'(x)$ may be thought of as a number, the slope of a tangent line. It could also be regarded as a linear operator mapping the real line to itself, the operator of multiplication by the number.
In vector calculus, $f$ might map $\mathbb{R}^m$ to $\mathbb{R}^n$. A linear operator mapping $\mathbb{R}^m$ to $\mathbb{R}^n$ may be represented by multiplication by an $n \times m$ matrix. (This is where $h$ might be considered an $m$-vector, a "direction".)
In your case, $f(A) = \det A$, a function that maps the vector space of real $n \times n$ matrices into $\mathbb{R}$. Therefore, its derivative is a linear operator mapping the space of $n \times n$ matrices into $\mathbb{R}$.
The answer to your question is: the $B$ in the formula is a generic real $n \times n$ matrix. (The approximation is good when $||B||$ is small enough.)
That raises the question of the representation of the derivative of the determinant. It can't simply be represented as square matrix multiplication. With the elements of $B$ given by $(b^\nu_\mu)$, and elements of the linear transformation $f'(x)$ given by $(t^\mu_\nu)$, something like $f'(x)B = \sum_{\mu,\nu} t_\mu^\nu b^\mu_\nu$ does the job.
I should point out that, in some restricted cases, there are simpler representations of the derivative of the determinant. For example, in the space of $n\times n$ conformal matrices, that is, matrices $A$ for which $A^T A = \alpha^2 I$, where $\alpha$ is a real number.