Solution $-$ Victor Wang
Let $r=35$, $I$, $\omega$, $u$, $O$, and $\Omega$ be the inradius, incenter (given), insphere, circumradius, circumcenter, and circumsphere of $SABC$, respectively. Let $s=SI=125$; $Q$ be the reflection of $S$ over $O$; $F$ be the foot from $S$ to plane $ABC$; $h=SF$ be the length of the $S$-altitude; $I_A$, $I_B$, $I_C$ be the feet from $I$ to $SBC$, $SCA$, $SAB$, respectively; $O_A$, $O_B$, $O_C$, $O_S$ be the circumcenters of triangles $SAB$, $SBC$, $SCA$, $ABC$, respectively; and $a=108$ be the common circumradius of triangles $SAB$, $SBC$, $SCA$. For convenience, define $v = SI_A = SI_B = SI_C = \sqrt{s^2-r^2} = 120$.
First, we note that $OO_A^2 = OO_B^2 = OO_C^2 = u^2-a^2$ by the Pythagorean theorem, so $O$ is equidistant from the three planes $SAB$, $SBC$, $SCA$.
Taking the cross section formed by $\triangle{OO_BO_C}$, we see that $O$ lies on one of the two planes bisecting the angle formed by planes $SAB$ and $SAC$: either the interior one or the exterior one (these correspond naturally to the two-dimensional interior and exterior angle bisectors). Now assume for the sake of contradiction that $O$ lies on the exterior bisector (which is perpendicular to the interior one), and WLOG suppose plane $SAB$ separates $O$ and $C$. Since $O$ is the center of $\Omega=(SABC)$, we then have $a = R_{SAC} < R_{SAB} = a$, a contradiction. Thus $O$ must lie on the interior angle bisector $P_A$ of $\angle{(SAB,SAC)}$.
If we similarly define $P_B,P_C$, then $S,O,I\in P_A\cap P_B\cap P_C$ (as $I$ lies completely inside $PABC$). But the intersection of three distinct planes can only form a point, a line, or the empty set, so $S,O,I$ must be collinear. Let $\epsilon$ equal $+1$ if $O,S$ are on the same side of plane $ABC$ and $-1$ otherwise. Clearly $O$ cannot be higher than $S$ (relative to plane $ABC$), since the line $SI = SO$ intersects the interior of $\triangle{ABC}$. In particular, $S$ and $Q$ must lie on opposite sides of $ABC$. (*) Using similar triangles and the Pythagorean theorem, we get
$$ \begin{align*} \frac{h-\epsilon OO_S}{h-r} = \frac{SO}{SI} = \frac{SO_A}{SI_A} \implies \frac{h-\epsilon\sqrt{u^2-R^2}}{h-r} = \frac{u}{s} = \frac{a}{v}.\tag{1} \end{align*}$$It immediately follows that $u = as/v = 225/2$.
We will now compute $h$ using inversion (this is natural because several circles and spheres in our diagram pass through $S$, and when we invert about $X$, the circumcenter of $\triangle{XYZ}$ is simply mapped to the reflection of $X$ over $Y'Z'$). Invert about $S$ with radius $\alpha$, and let $J_A,J_B,J_C$ be the midpoints of $I'_A,I'_B,I'_C$, respectively. Observe that $SQ$ is a diameter of $\Omega$, so $\angle{SQ'A'}=\angle{SQ'B'}=\angle{SQ'C'} = 90^\circ$, whence $Q'$ is the foot from $S$ to $A'B'C'$ (the spheres with diameters $SA'$, $SB'$, $SC'$ intersect at $S$ and its reflection over the plane formed by their centers). But $J_A$ is the foot from $S$ to $B'C'$, so $Q'P$ is minimized over $P\in B'C'$ at $P=J_A$ (by the Pythagorean theorem) and $J_A$ is the foot from $Q'$ to $B'C'$. On the other hand, $SJ_A=SJ_B=SJ_C = \frac{\alpha^2}{2SI_A}$ implies $Q'J_A = Q'J_B = Q'J_C$, whence $\triangle{A'B'C'}$ has incircle $(J_AJ_BJ_C)$ and inradius
$$\begin{align*} t = Q'J_A = \sqrt{\left(\frac{\alpha^2}{2SI_A}\right)^2 - \left(\frac{\alpha^2}{2SO}\right)^2} = \frac{\alpha^2}{2}\sqrt{\frac{1}{a^2}-\frac{1}{u^2}} = \frac{\alpha^2}{2}\frac{r}{as}.\tag{2} \end{align*}$$(Indeed, according to (*), $Q'$ cannot be an excenter of $\triangle{A'B'C'}$, since $Q'$ lies on the same side of planes $SB'C'=SBC,SC'A'=SCA,SA'B'=SAB$ as $A',B',C'$, respectively.)
It remains to analyze $F'$. $\angle{SA'F'} = \angle{SB'F'} = \angle{SC'F'} = 90^\circ$, so $A',B',C'$ lie on the sphere $\Gamma$ with diameter $SF'$. Let $K$ be the midpoint of $SF'$ and $L$ be the foot from $K$ to $A'B'C'$; then $K$ is the center of $\Gamma$ and $L$ is the circumcenter of $\triangle{A'B'C'}$. If $x=SK$ denotes the radius of $\Gamma$, then $2x = SF' = \alpha^2/SF = \alpha^2/h$.
Fortunately, $Q'$, the foot of $S$, lies in the interior of $\triangle{A'B'C'}$, so $K$ cannot be higher than $S$ (relative to plane $A'B'C'$); hence $T$ lies on ray $\overrightarrow{SQ'}$. Since the (acute) angle $\theta$ between lines $SF'=SF$ and $SQ'=SQ$ is fixed under inversion, we get
$$ \frac{Q'L}{x} = \frac{TK}{SK} = \sin\theta = \frac{\sqrt{SI^2 - (h-r)^2}}{SI} = \frac{\sqrt{s^2-(h-r)^2}}{s} $$and
$$ KL = TQ' = \lvert SQ' - ST \rvert = \left\lvert \frac{\alpha^2}{2u} - SK\cos\theta \right\rvert = \left\lvert \frac{xh}{u} - x\frac{h-r}{SI} \right\rvert = x\left\lvert \frac{h}{u} - \frac{h-r}{s}\right\rvert. $$Plugging the previous two equations and (2) into Euler's formula (for $\triangle{A'B'C'}$) yields
$$\begin{align*}
x^2 - x^2\frac{(h-r)^2}{s^2} = x^2\frac{s^2-(h-r)^2}{s^2} = Q'L^2
&= R_{A'B'C'}^2 - 2tR_{A'B'C'} \\
&= (x^2 - KL^2) - \alpha^2\frac{r}{as}\sqrt{x^2 - KL^2} \\
&= x^2 - x^2\left(\frac{h}{u} - \frac{h-r}{s}\right)^2 - 2xh\frac{r}{as}x\sqrt{1 - \left(\frac{h}{u} - \frac{h-r}{s}\right)^2}.
\end{align*}$$Adding $-x^2$ to both sides, substituting $as=uv$, and multiplying both sides by $-x^{-2}u^2s^2$, we get
$$\begin{align*}
u^2(h-r)^2 &= (sh - u(h-r))^2 + \frac{2hrs}{v}\sqrt{u^2s^2 - (sh - u(h-r))^2} \\
-s^2h^2 + 2shu(h-r) &= \frac{2hrs}{v}\sqrt{u^2s^2 - (sh - u(h-r))^2} \\
v(-sh+2u(h-r))& = 2r\sqrt{u^2s^2 - (sh - u(h-r))^2} \\
(s^2-r^2)[h^2(2u-s)^2 - 4h(2u-s)ur + 4u^2r^2] &= 4r^2 [-h^2(s-u)^2 - 2h(s-u)ur - u^2(r^2-s^2)] \\
v^2[h(2u-s)^2 - 4(2u-s)ur] &= 4r^2 [-h(s-u)^2 - 2(s-u)ur],
\end{align*}$$whence
$$\begin{align*}
h = 4ur \frac{v^2(2u-s) - 2r^2(s-u)}{v^2(2u-s)^2 + 4r^2(s-u)^2}
&= 4\frac{225}{2}35\frac{120^2(225-125) - (35)^2(2\cdot125-225)}{120^2(225-125)^2 + (35)^2(2\cdot125-225)^2} \\
&= 70\cdot15^2\frac{120^2\cdot10^2-35^2\cdot5^2}{120^2\cdot10^4+35^2\cdot5^4} \\
&= 70\cdot3^2\frac{24^2\cdot2^2-7^2}{24^2\cdot2^4+7^2} \\
&= \frac{284130}{1853}.
\end{align*}$$We can find $\epsilon$ now (actually, $h>u$, so we trivially have $\epsilon=+1$), but this is unnecessary: plugging $h$ into (1) yields
$$\begin{align*}
R^2
&= u^2 - \left(h-\frac{a}{v}(h-r)\right)^2 \\
&= \frac{225^2}{2^2} - \left(\frac{284130}{1853}\left(1-\frac{108}{120}\right)+\frac{108\cdot35}{120}\right)^2 \\
&= \frac{9^2\cdot25^2}{2^2} - \left(\frac{28413}{1853}+\frac{9\cdot7}{2}\right)^2 \\
&= \frac{9^2\cdot25^2}{2^2} - \left(\frac{28413}{1853}+\frac{9\cdot7}{2}\right)^2 \\
&= \left(9^2 - \frac{28413}{1853}\right)\left(9\cdot16 + \frac{28413}{1853}\right) \\
&= \frac{121680\cdot295245}{1853^2} \\
&= \frac{35925411600}{3433609}.
\end{align*}$$Thus the answer is $35925411600+3433609 = \boxed{35928845209}$.
I posted this as a:
- bump
- example of a solution
- demo of what was already on AoPS
