All of these are algebraic integers:
$$\frac1{1+\cos\tfrac\tau4}=\frac1{1+0}=1$$ $$\frac1{1+\cos\tfrac\tau8}=\frac1{1+\tfrac12\sqrt2}=2-\sqrt2$$ $$\frac1{1+\cos\tfrac\tau{12}}=\frac1{1+\tfrac12\sqrt3}=4-2\sqrt3$$ $$\frac1{1+\cos\tfrac\tau{16}}=\frac1{1+\tfrac12\sqrt{2+\sqrt2}}=(2+\sqrt2)\Big(2-\sqrt{2+\sqrt2}\Big)$$ $$\frac1{1+\cos\tfrac\tau{20}}=\frac1{1+\tfrac12\sqrt{2+\varphi}}=2(1+\varphi)\Big(2-\sqrt{2+\varphi}\Big)$$ $$\bigg(\varphi=\frac{1+\sqrt5}2\bigg)$$ $$\frac1{1+\cos\tfrac\tau{24}}=\frac1{1+\tfrac12\sqrt{2+\sqrt3}}=2(2+\sqrt3)\Big(2-\sqrt{2+\sqrt3}\Big)$$
And I checked a few more values of $1/(1+\cos\tfrac\tau{4n})$ using WolframAlpha to calculate the minimal polynomials. Does this hold for all $n$?
I know that $2\cos\tfrac\tau n$ is always an algebraic integer, being a sum of roots of unity. Also, here's a similar question: For what integer values of n is $\tan (\pi /n)$ an algebraic integer?
It would be sufficient to prove that $1/(1-\cos\tfrac\tau{2n})$ is an algebraic integer, because
$$\frac1{1+\cos\tfrac\tau{4n}}=\frac{1-\cos\tfrac\tau{4n}}{\sin^2\tfrac\tau{4n}}=\frac{2-2\cos\tfrac\tau{4n}}{1-\cos\tfrac\tau{2n}}.$$
Hence the title. (Note $\pi=\tau/2$. See https://tauday.com/tau-manifesto )