Let $H$ be a Hilbert space and let $A$ be a linear operator on $H$. See here for the very basic definitions. We also recall the following definitions.
If $A$ is injective, we define the linear operator $A^{-1}$ on $H$ as follows. If $x, y\in H$ we say $x\in {\rm dom} \, A^{-1}$ and $A^{-1}x = y$ if and only if $x\in {\rm ran} \, A$, $y\in {\rm dom} \, A$ and $Ay = x$. Note that $A$ does not need to be surjective in order to define the inverse $A^{-1}$ of $A$.
If $A$ is densely defined, we define the linear operator $A^{*}$ on $H$ as follows. If $y, z\in H$ we say $y\in {\rm dom} \, A^{*}$ and $A^{*}y = z$ if and only if we have $(Ax, y)_{H} = (x, z)_{H}$ for all $x\in {\rm dom} \, A$. We further say $A$ is self-adjoint if $A^{*} = A$.
We consider $H\times H$ as a Hilbert space equipped with the inner product
\begin{equation}
((x_{1}, y_{1}), (x_{2}, y_{2}))_{H\times H} = (x_{1}, x_{2})_{H} + (y_{1}, y_{2})_{H} .
\end{equation}
Recall that if $A$ is an injective linear operator on a Hilbert space $H$, it is not sufficient to show that
\begin{equation}
(A^{-1}x, y)_{H} = (x, A^{-1}y)_{H}
\end{equation}
for all $x,y\in {\rm dom} \, A^{-1}$ in order to conclude that $A^{-1}$ is self-adjoint. A couple of issues preventing this are described below.
The linear operator $A^{-1}$ may not be densely defined so that $(A^{-1})^{*}$ does not even exist as a linear operator.
Even if $A^{-1}$ is densely defined, it may be that ${\rm dom} \, A^{-1} \neq {\rm dom} (A^{-1})^{*}$ so that $A^{-1}$ and $(A^{-1})^{*}$ are not equal as linear operators. See here for examples of linear operators defined on suitable Hilbert spaces that are densely defined and symmetric but not self-adjoint.
To establish the result from the question, we first prove a lemma which provides a characterisation of self-adjoint operators.
Lemma. Let $A$ be a densely defined linear operator on a Hilbert space $H$. Define $J\colon H\times H\to H\times H$ by $J(x,y) = (-y,x)$. Then the following hold.
$({\rm i})$ We have ${\rm graph} \, A^{*} = (J({\rm graph} \, A))^{\perp}$.
$({\rm ii})$ We have that $A$ is self-adjoint if and only if ${\rm graph} \, A = (J({\rm graph} \, A))^{\perp}$.
Proof of $({\rm i})$. Let $y, z \in H$. Each of the statements 1, 2, 3, 4 and 5 below is equivalent to the one following it.
We have $(y, z) \in {\rm graph} \, A^{*}$.
We have $(Ax, y)_{H} = (x, z)_{H}$ for all $x\in {\rm dom} \, A$.
We have $(-Ax, y)_{H} + (x, z)_{H} = 0$ for all $x\in {\rm dom} \, A$.
We have $((-Ax, x), (y, z))_{H\times H} = 0$ for all $x\in {\rm dom} \, A$.
We have $(J(x, Ax), (y, z))_{H\times H} = 0$ for all $x\in {\rm dom} \, A$.
We have $(y, z) \in (J({\rm graph} \, A))^{\perp}$.
The equivalence of statements 1 and 6 gives us $({\rm i})$.
Proof of $({\rm ii})$. As $A$ is self-adjoint if and only if ${\rm graph} \, A = {\rm graph} \, A^{*}$, the result follows from $({\rm i})$.
We now prove the main result.
Theorem. Let $A$ be an injective self-adjoint linear operator on a Hilbert space $H$. Then $A^{-1}$ is self-adjoint.
Proof. First let $x, y \in H$. Each of the statements 1, 2 and 3 below is equivalent to the one following it.
We have $(x, y) \in {\rm graph} \, A^{-1}$.
We have $(y, x) \in {\rm graph} \, A$.
We have $(y, -x) \in {\rm graph} (-A)$.
We have $(x,y) = J(y, -x) \in J({\rm graph} (-A))$.
From the equivalence of statements $1$ and $4$ we see that
\begin{equation}
{\rm graph} \, A^{-1} = J({\rm graph} (-A)) . \tag{1}
\end{equation}
Then by applying $J$ to both sides as well as using $J^{2} = -{\rm id}_{H\times H}$ and that ${\rm graph}(-A)$ is a vector subspace, we get
\begin{equation}
J({\rm graph} \, A^{-1}) = {\rm graph}(-A) . \tag{2}
\end{equation}
Now as $A$ is self-adjoint we also have that $-A$ is self-adjoint, so by $({\rm ii})$ in the lemma we have ${\rm graph} (-A) = (J({\rm graph} (-A)))^{\perp}$. But combining this with $(1)$ and $(2)$ implies
\begin{equation}
{\rm graph} \, A^{-1} = J({\rm graph}(-A)) = ({\rm graph} (-A))^{\perp} = (J({\rm graph} \, A^{-1}) )^{\perp} . \tag{3}
\end{equation}
Finally, by applying $({\rm ii})$ of the lemma to $(3)$ we conclude that $A^{-1}$ is self-adjoint.
Here is some additional information.
The proof above can be found in Theorem 10.44 of Functional Analysis by van Neerven. An online copy for personal use is available here.
Here is a related result. Let $H_{1}$ and $H_{2}$ be Hilbert spaces over the same field. Let $A$ be a densely defined linear operator from $H_{1}$ into $H_{2}$ that is injective and has dense range. Then $A^{*}$ is injective, $A^{-1}$ is densely defined and $(A^{*})^{-1} = (A^{-1})^{*}$. A proof can be found in Theorem 1.8(iii) from Unbounded Self-Adjoint Operators on Hilbert Space by Schmüdgen. There are also versions for multivalued linear operators. A reference for this is Section C.2 of The Functional Calculus for Sectorial Operators by Haase.