6

Many sources claim that $f(x)=x+1$ is not backwards stable. That is, it does not give an exact solution to a slightly perturbed (or "nearby") problem.

e.g. https://www.cs.usask.ca/~spiteri/CMPT898/notes/numericalStability.pdf on page 24.

Now, when I work this out myself, I think I'm able to show that $\exists\,\,\, \epsilon $ s.t. the computed problem with errors of $\epsilon_1, \epsilon_2,$ and $\epsilon_3 $ is equal to a slightly perturbed problem. Since there is an error for rounding on each input, and then an error for the addition, the computed solution is $(x(1+\epsilon_1)+1(1+\epsilon_2))(1+\epsilon_3)$, and the exact solution slightly perturbed is $(x+1)(1+\epsilon)$. So i am showingthese are equal for some epsilon on the order of machine precision:

$(x(1+\epsilon_1)+1(1+\epsilon_2))(1+\epsilon_3)=(x+1)(1+\epsilon)$

multiplying these out we get:

$x+\epsilon_1x+1+\epsilon_2+\epsilon_2x+\epsilon_3\epsilon_1x+\epsilon_3+\epsilon_3\epsilon_2=x+1+\epsilon x+\epsilon$

subtract $x$ and $1$ from both sides

$\epsilon_1x+\epsilon_2+\epsilon_2x+\epsilon_3\epsilon_1x+\epsilon_3+\epsilon_3\epsilon_2=\epsilon x+\epsilon$

Now, it seems clear to me that we cna always find an epsilon on the right hand side that will complete the equation.

Most sources cite $x=0$ as the value that breaks this condition, but if I set $x=0$ I am still able to see an $\epsilon$ that mkaes this work.

What am i missing here?

makansij
  • 1,653
  • In case you're wondering if the error for addition is typical, sources such as the book by Trefethen and Bau definitely seem to show that there is some error in the addition/subtraction of two numbers. https://i.sstatic.net/Dd0Ln.png – makansij Nov 12 '17 at 05:22
  • How coincidental... – J. M. ain't a mathematician Nov 12 '17 at 05:24
  • 1
    "the exactly solution slightly perturbed is $(x+1)(1+\epsilon)$". In the analysis of backward stability, that should be $x(1+\epsilon) +1$. With this corrected, you see that on the left-hand side there is something proportional to $\epsilon$ but independent of $x$, while on the right-hand side the $\epsilon$ is multiplied with $x$. Hence when $x$ is sufficiently close to $0$, there will be no solution. – George C Sep 14 '20 at 14:55

2 Answers2

5

The fundamental problem is that the domain is not clearly stated. We have two functions which are relevant in this context. Let $\mathcal F$ denote our set of floating point numbers. Then the relevant functions are $$ f : \mathcal F \times \mathcal F \rightarrow \mathbb R, \quad f(x,y) = x +y$$ and $$ g : \mathcal F \to \mathbb R, \quad g(x) = 1 + x.$$ This two function are closely related, yet decidedly different as evidenced by their different domains.

Let $\hat{f}$ and $\hat{g}$ denote the computed value of $f$ and $g$. In the absence of floating point exceptions, we have $$ \hat{f} = (x+y)(1+\delta), \quad |\delta| \leq u,$$ where $u$ is the unit roundoff. This can also be expressed as $$\hat{f} = f(\bar{x},\bar{y}), \quad \bar{x} = x(1+\delta), \quad \bar{y} = y(1+\delta).$$ We conclude that $f$ is backward stable. In contrast, we have $$ \hat{g} = (1 + x)(1+ \nu), \quad |\nu| \leq u.$$ It is clear, that $$ \hat{g} = 1 + \nu + x(1 + \nu) = 1 + z, \quad z = \nu + x(1+\nu) = x\left( 1 + \left[\frac{\nu}{x} + \nu \right] \right) .$$ Now while $z$ is a good approximation of $x$ in the absolute sense, the relative error is large, when $x$ is tiny. This is the point were we discover that backwards stability has been lost.

Carl Christian
  • 13,776
  • 1
    Thanks @CarlChristian. However I do not understand your example all the way through. For $\hat{g}$, why do you not have an error around 1 and $x$? You seem to ONLY have an error around the entire problem. It seems to me like it should be $\hat{g}=(1(1+\nu_1)+x(1+\nu_2))(1+\nu_3)$ This is what I mentioned in the title of my question "error propagated by addition". Is there any reason you only have the error around the entire $(1+x)$? – makansij Nov 12 '17 at 22:46
  • Also, @CarlChristian, I think you are mixing conditioning of the problem, with stability of the algorithm. Yes, the problem can be ill-conditioned in some cases, but it is possible for an ill-conditioned problem to be solved with a backwards stable algorithm. So, I did not think "relative error" would be mentioned in your answer here. If you look in pdf that I linked to (https://www.cs.usask.ca/~spiteri/CMPT898/notes/numericalStability.pdf) you can see the definition on page 16 for backward stability - i uses relative error for $x$ but not for $f(x)$. – makansij Nov 12 '17 at 23:02
  • 2
    @guimption Backward error analysis assumes, that function arguments are exact, i.e. belong to the set $F$, hence are not perturbed by the algorithm. – Pawel Kowal Nov 13 '17 at 02:02
  • 1
    @guimption: There is not and should not be any error term associated with the number $1$ because $1$ is a floating point number. I use the words "relative error" correctly inside a sentence which deals exclusively with input values and not with output values as you are implying. – Carl Christian Nov 13 '17 at 08:40
  • ah, that is a key point, thanks!!! – makansij Nov 13 '17 at 17:04
2

In this case $\tilde{f}(x) = \mathrm{fl}(x) \oplus 1$. Now there is a bound $c>0$ such that if $0<\mathrm{fl}(x)< c$ then $\mathrm{fl}(x)\oplus 1=1$. So in this case $\tilde{f}(x) = f(\tilde{x})$ implies $\tilde{x}=0$. However, $0$ is too far from $\mathrm{fl}(x)$ to satisfy the backward stability requirement.

WimC
  • 33,614
  • 2
  • 52
  • 99
  • I just do not understand your answer. Do you mind rephrasing it or something? How does $\tilde{f}(x)=f(\tilde{x})$ imply that $\tilde{x}=0$? You seem to be on the right track, but I don't see it all the way through. – makansij Nov 12 '17 at 22:13
  • are you saying that there is no roundoff error in the $1$ in $\tilde{f}(x) = \mathrm{fl}(x) \oplus 1$ ? If not - why not? – makansij Nov 12 '17 at 22:49
  • yes, $1$ is represented exactly ($\mathrm{fl}(1)=1$). – WimC Nov 13 '17 at 18:34