It is always true. We first provide a proof that does not use any functional calculus and then provide a proof of a more general result using functional calculus.
This first proof does not use functional calculus. A proof can be found in Proposition 3.18 of Unbounded Self-adjoint Operators on Hilbert Space by Schmüdgen. We quote the proposition below from the book by Schmüdgen.
Proposition 3.18. Let $T$ be a densely defined closed linear operator from a Hilbert space $\mathcal{H}_{1}$ into a Hilbert space $\mathcal{H}_{2}$. Then:
$({\rm i})$ $I + T^{*}T$ is a bijective mapping of $\mathcal{H}_{1}$. Its inverse $C := (I + T^{*}T)^{-1}$ is a bounded self-adjoint operator on $\mathcal{H}_{1}$ such that $0 \leq C \leq I$.
$({\rm ii})$ $T^{*}T$ is a positive self-adjoint operator on $\mathcal{H}_{1}$, and $\mathcal{D}(T^{*}T)$ is a core for $T$.
Note that the book assumes all Hilbert spaces are over the complex numbers, but all the arguments for this proposition carry straight over to the real case. If we apply Proposition 3.18 to a self-adjoint operator $T$ on a Hilbert space $H$, then we deduce that $T^{2}$ is a self-adjoint operator on $H$.
We now prove the relevant parts of Proposition 3.18 for our uses. We first show $({\rm i})$. The first task is to show $I + T^{*}T \colon \mathcal{D}(T^{*}T) \to \mathcal{H}_{1}$ is surjective. We then show that
\begin{equation}
\|x\|_{\mathcal{H}_{1}} \leq \|(I + T^{*}T)x\|_{\mathcal{H}_{1}} \tag{1}
\end{equation}
for all $x\in \mathcal{D}(T^{*}T)$ and deduce the remaining results from there.
We show that $I + T^{*}T$ is surjective. Define $V\colon \mathcal{H}_{1} \times \mathcal{H}_{2} \to \mathcal{H}_{2} \times \mathcal{H}_{1}$ by $V(x,y) := (-y,x)$. By this answer we have $\mathcal{G}(T^{*}) = V(\mathcal{G}(T))^{\perp}$. Since $T$ is closed and $V$ is a unitary isomorphism we have that $V(\mathcal{G}(T))$ is closed in $\mathcal{H}_{1} \times \mathcal{H}_{2}$. Hence it follows that
\begin{equation}
\mathcal{H}_{2} \oplus \mathcal{H}_{1} = V(\mathcal{G}(T)) \oplus \mathcal{G}(T^{*}) . \tag{2}
\end{equation}
Let $u\in \mathcal{H}_{1}$. By $(2)$ there are $x\in \mathcal{D}(T)$ and $y\in \mathcal{D}(T^{*})$ such that
\begin{equation}
(0,u) = V(x, Tx) + (y, T^{*}y) = (y - Tx, T^{*}y + x ) . \tag{3}
\end{equation}
From $(3)$ we deduce $y = Tx$ and $u = x + T^{*}y$. This implies $u = (I + T^{*}T)x$. Hence $I + T^{*}T$ is surjective.
We now show $(1)$. Let $x\in \mathcal{D}(T^{*}T)$. We have
\begin{equation}
\begin{aligned}
\|(I + T^{*}T)x \|^{2} &= (x + T^{*}Tx, x + T^{*}Tx )_{H} \\
&= \|x\|^{2} + \|T^{*}Tx\|^{2} + 2 \|Tx\|^{2} \\
&\geq \|x^{2}\| .
\end{aligned} \tag{4}
\end{equation}
Hence we deduce $(1)$. From what we have shown so far, we have that $I + T^{*}T$ is a linear bijection from $\mathcal{D}(T^{*}T)$ onto $\mathcal{H}_{1}$. Hence $C := (I + T^{*}T)^{-1}$ is well-defined and is bounded with $\|C\| \leq 1$ by $(1)$. All that remains is to show that $C$ is a positive operator with $0 \leq C \leq I$, since positive operators are automatically self-adjoint. Let $u\in \mathcal{H}_{1}$. Since $I + T^{*}T$ is surjective there is $x\in \mathcal{D}(T^{*}T)$ such that $(I + T^{*}T)x = u$. Then $x = Cu$ and by $(4)$ we have
\begin{align*}
(Cu, u)_{\mathcal{H}_{1}} &= (x, x + T^{*}Tx )_{\mathcal{H}_{1}} \\
&= \|x\|^{2} + \|Tx\|^{2} \\
&\leq \|(I + T^{*}T)x\|^{2} = \|u\|^{2} .
\end{align*}
It follows that $C$ is self-adjoint, positive and satisfies $0 \leq C \leq I$. This proves $({\rm i})$.
For $({\rm ii})$ we only need that $T^{*}T$ is self-adjoint for our purposes, which follows from this result.
Here is a second proof. We base it off an exposition that does not use spectral measures and instead takes a point of view focusing on the measurable functional calculus from the following paper by Haase.
[1] Haase, M., The Functional Calculus Approach to the Spectral Theorem, 2020. (Link)
One advantage of this approach is that we can directly focus on the Borel functional calculus rather than also having to think about spectral measures or multiplication operator representations. This approach also has a simple method for obtaining the general Borel functional calculus from the bounded version.
We first define the notion of a measurable functional calculus. Let $H$ be a real or complex Hilbert space, let $(\Omega , \mathcal{A})$ be a measurable space, let $\mathcal{M}(\Omega , \mathcal{A})$ denote the set of measurable functions $f\colon \Omega \to \mathbb{K}$ and let $\mathcal{C}(H)$ denote the set of closed linear operators on $H$. We recall that a linear operator $A$ on $H$ is closed if ${\rm dom}(A)$ is a vector subspace of $H$ and the graph $\{(x, Ax) : x\in {\rm dom}(A) \}$ is a closed subspace of $H\times H$. A measurable functional calculus is a pair $(\Phi , H)$ where $\Phi \colon \mathcal{M}(\Omega , \mathcal{A}) \to \mathcal{C}(H)$ is a map satisfying the following properties for all $f,g\in \mathcal{M}(\Omega , \mathcal{A})$ and $\lambda \in \mathbb{K}$.
We have $\Phi (1_{\Omega}) = I_{H}$.
We have $\Phi (f) + \Phi (g) \subseteq \Phi (f+g)$ and $\lambda \Phi (f) \subseteq \Phi (\lambda f)$.
We have $\Phi (f) \Phi (g) \subseteq \Phi (fg)$ and
\begin{equation}
{\rm dom}(\Phi (f) \Phi (g) ) = {\rm dom}(\Phi (g)) \cap {\rm dom}(\Phi (fg)) .
\end{equation}
If $f$ is bounded then $\Phi (f) \in \mathcal{L}(H)$ and $\Phi (f)^{*} = \Phi (\overline{f})$.
If $(f_{n})_{n\in\mathbb{N}}$ is a bounded sequence of measurable functions such that $(f_{n}(x))_{n\in\mathbb{N}}$ converges to $f(x)$ for every $x\in\Omega$, then $f$ is bounded and
\begin{equation}
\lim_{n\to\infty} (\Phi (f_{n})x, y)_{H} = (\Phi (f)x, y)_{H}
\end{equation}
for all $x,y\in H$.
It turns out that many standard properties of the functional calculus associated with the spectral measure construction can be deduced from just those five conditions above. Here we will only need the following results.
Lemma 1. Let $\Phi \colon \mathcal{M}(\Omega , \mathcal{A}) \to \mathcal{C}(H)$ be a measurable functional calculus. Let $p \in \mathbb{K}(\mathbf{x})$ be a nonconstant polynomial taken as a function. Then $\Phi (p\circ f) = p(\Phi (f))$. In particular, we have ${\rm dom}(\Phi (p\circ f)) = {\rm dom}(p(\Phi (f)))$.
Proof. See Corollary 2.3.c) of [1].
So far we have not said anything about how the measurable functional calculus relates to self-adjoint operators on $H$. This is what we will do next. We say a measurable functional calculus $(\Phi , H)$ on $(\Omega , \mathcal{A})$ is a Borel functional calculus if $\Omega$ is a topological space and $\mathcal{A}$ is the Borel $\sigma$-algebra on $\Omega$. We have the following result that relates the measurable functional calculus definition above to self-adjoint operators.
Theorem 1. Let $A$ be a self-adjoint operator on a real or complex Hilbert space. Then there is a unique Borel functional calculus $(\Phi , H)$ on $\mathbb{K}$ such that $\Phi ({\rm id}) = A$.
Proof. See Theorem 7.6 in [1].
The last result we need is the following condition which provides a natural condition for linear operators arising from the measurable functional calculus to be self-adjoint.
Lemma 2. Let $A$ be a self-adjoint operator on a Hilbert space $H$. Let $(\Phi , H)$ be the unique Borel functional calculus on $\mathbb{K}$ such that $\Phi ({\rm id}) = A$ from Theorem 1. Let $f\colon \mathbb{K} \to \mathbb{K}$ be a Borel measurable function such that $f(x) \in \mathbb{R}$ for all $x\in\mathbb{R}$. Then $\Phi (f)$ is self-adjoint.
Proof. This follows from combining Theorem 4.1.d) and Corollary 4.4 of [1].
We now have everything we need to address your original question. In fact, we can show the following more general result while we are at it.
Theorem 2. Let $A$ be a self-adjoint operator on a real or complex Hilbert space $H$. Let $p \in \mathbb{K}(\mathbf{x})$ be a nonconstant polynomial with real coefficients taken a function. That is, there are $k\in\mathbb{N}$ and $c_{0}, \ldots , c_{k} \in \mathbb{R}$ with $c_{k} \neq 0$ such that $p(z) = \sum_{j=0}^{k} c_{j}z^{j}$ for all $z\in\mathbb{K}$. Then the operator $p(A)$ defined by
\begin{equation}
p(A) = \sum_{j=0}^{k} c_{j} A^{j}
\end{equation}
with ${\rm dom}(p(A)) = {\rm dom}(A^{k})$ is self-adjoint.
Proof. By Theorem 1 there is a Borel functional calculus $(\Phi , H)$ on $\mathbb{K}$ such that $\Phi ({\rm id}) = A$. By Lemma 1 we have
\begin{equation}
\Phi (p) = \Phi (p\circ {\rm id}) = p(\Phi ({\rm id})) = p(A)
\end{equation}
with equality of domains. Furthermore, as $p$ has real coefficients we have that $\Phi (p)$ is self-adjoint by Lemma 2. Therefore $p(A)$ is self-adjoint.
