8

This question introduces a dynamical system inspired by entropy where each step

$$(p_1, \dots, p_n) \mapsto (\frac1H p_1 \ln p_1, \dots, \frac1H p_n \ln p_n)$$ where $H = \sum_i p_i \ln p_i$ and conjectures that this always limits to the uniform distribution on $n$ outcomes.

and my answer to it shows that this follows from a particular inequality. Namely, that no point $(x,y)$ in the region $$e^{-x} + e^{-y} \le 1 \tag{1}$$ satisfies $$\frac{\ln x - \ln y}{x - y} \ge 1.99 \tag{2}$$ Where substituting $1.99$ for any other number $<2$ works too (but it might not be true if we set our number too small).

I can prove that this is true (and thus prove the original conjecture), but more than half of the proof is taken up showing the above inequalities, in a very inelegant way, especially considering how suggestively the curves fit together (I strongly encourage you to look at the graph). I would like to know if there is a better way.


My proof

First, observe that, by concavity of $\ln$, $(2)$ is strictly decreasing in $x$ and $y$, and $(1)$ is also strictly decreasing in $x$ and $y$. Hence if there is any point of overlap, in particular we can find a point of overlap on the boundary of $(1)$ by decreasing $x$ and/or $y$ until we reach the boundary.

$(1)$ can be rearranged to $$y \ge -\ln(1 - e^{-x})$$ $$y \ge e^{-x} + \frac12 e^{-2x} + \frac13 e^{-3x} + \dots$$ so on the boundary we have $$e^{-x} < y < e^{-x} + e^{-2x} + e^{-3x} + \dots = \frac{e^{-x}}{1 - e^{-x}} = \alpha$$ So taking wlog $x > y$, in $(2)$ the numerator is $$\ln x - \ln y < \ln x - \ln(e^{-x}) = \ln x + x$$ and the denominator is $$x - y > x - \alpha$$ bounding $(2)$ as $$\frac{\ln x - \ln y}{x - y} < \frac{\ln x + x}{x - \alpha} \tag{3}$$ The RHS of which is plotted in the graph I linked, and, treating $\alpha$ as exogenous for now, for $\alpha = \frac12$ it can be seen to have a global maximum of $2$ at $x=1$*. Clearly the whole expression is strictly increasing in $\alpha$, so if we pick any $\alpha^* < \frac12$ there will be some constant $c < 2$ that $(3)$ is upper bounded by. Hence on the boundary of $(1)$, and when $x$ is large enough that $\alpha$ (which remember is actually a function of $x$) is $\le \alpha^*$, we have that $(2)$ is bounded above by $c$. $\alpha = \frac12$ when $x = \ln 3$ so picking $x \ge x^* = 1.1 > \ln 3$ will work.

By symmetry, if $y \ge x^*$, $(2)$ is bounded above by $c$. So the only region we still need to check is the area away from the tails, where the curves are a finite distance apart, so we can do this mechanically:

If we can find any curve going from $x = x^*$ to $y = x^*$ that sandwiches between $(1)$ and $(2)$, intersecting neither, that shows the two regions cannot intersect. We can do this by building a staircase of points that we can just computationally verify to be in neither region. Because both expressions are decreasing in $x$ and $y$, if two points are both inside the gap, then the vertical or horizontal line between them must also be in the gap, so we only need to verify the computation for finitely many points. An example staircase is plotted on the graph I linked, showing it is possible, and completing the proof.

*proof: $$\frac{\ln x + x}{x - \frac12} \le 2$$ rearranges to $$\ln x \le x - 1$$ for $x > \frac12$, which is true by concavity of $\ln$ and tangency at $(1,0)$, meaning for all $x > \frac12$ the expression is bounded above by $2$.

Amir
  • 11,213
Zoe Allen
  • 8,022
  • Interesting question. Just for clarification, you've written "Where $1.99$ can be substituted out for any other number $<2$ if we like" which (obviously) isn't true - it seems in your original answer you found $1.443$ to be the lowest that number could be - but are you asking just to show the maximum value is less than $2$? – Chris Lewis Jul 29 '24 at 09:22
  • @ChrisLewis I meant that proving it for anything $<2$ would be sufficient, not that it would be possible. I will edit my question. – Zoe Allen Jul 29 '24 at 10:40

1 Answers1

3

Alternative way.

Fact 1. If $x > y > 0$ and $\mathrm{e}^{-x} + \mathrm{e}^{-y} \le 1$, then $$\frac{\ln x - \ln y}{x - y} \le \frac{1}{\ln 2} \approx 1.442695041.$$

Proof of Fact 1.

It suffices to prove that $$f(x) := \frac{1}{\ln 2} (x - y) - \ln x + \ln y \ge 0.$$

We have $f'(x) = \frac{1}{\ln 2} - \frac{1}{x}$. Thus, $f'(x) \ge 0$ on $x \ge \ln 2$.

We split into two cases.

Case 1. $y \ge \ln 2$

Using $x > y \ge \ln 2$, we have $f(x) \ge f(y) = 0$.

Case 2. $0 < y < \ln 2$

Using $\mathrm{e}^{-x} + \mathrm{e}^{-y} \le 1$ and $0 < y < \ln 2$, we have $$x \ge -\ln(1 - \mathrm{e}^{-y}) \ge \ln 2.$$

It suffices to prove that $f(-\ln(1 - \mathrm{e}^{-y})) \ge 0$, or $$g(y) := \frac{-\ln(1 - \mathrm{e}^{-y}) - y}{\ln 2} - \ln(-\ln(1 - \mathrm{e}^{-y})) + \ln y \ge 0.$$

Since $g(\ln 2) = 0$, it suffices to prove that $g'(y) \le 0$, or $$h(y) := -\ln(1 - \mathrm{e}^{-y}) - \left(\frac{1}{(1-\mathrm{e}^{-y})\ln 2} - \frac{1}{y}\right)^{-1}\frac{\mathrm{e}^{-y}}{1 - \mathrm{e}^{-y}} \ge 0.$$

Since $h(\ln 2) = 0$, it suffices to prove that $h'(y) \le 0$, or $$- \frac{(a^2 - ay - 2a + y + a(2 + y - a)\mathrm{e}^{-y})y\mathrm{e}^{-y}}{(1 - \mathrm{e}^{-y})(a\mathrm{e}^{-y} - a + y)^2} \le 0$$ where $a := \ln 2$. It suffices to prove that $$F(y) := a^2 - ay - 2a + y + a(2 + y - a)\mathrm{e}^{-y} \ge 0.$$ Note that $F''(y) = -\mathrm{e}^{-y}(\ln 2 - y) \ln 2 \le 0$. Also, $F(0) = F(\ln 2) = 0$. Thus, $F(y) \ge 0$ on $[0, \ln 2]$.

We are done.

River Li
  • 51,793
  • Hi River! Could you determine how the upper bound (which is $1/\log2$ for $a=1$) changes when $\mathrm{e}^{-x} + \mathrm{e}^{-y} \le a$ with $a\in (0,2)$ is given? – Amir Aug 07 '24 at 08:11
  • 1
    @Amir It seems that if $0 < a < 1$, then the upper bound is $\frac{1}{\ln 2 - \ln a}$. – River Li Aug 07 '24 at 09:48
  • 1
    @Amir It looks fine. You may wait some users to check it rigorously. – River Li Aug 07 '24 at 12:04
  • There is some connection to the logarithmic mean since the inequality can be written as $$\ln \frac{2}{e^{-x}+e^{-y}} \le \frac{x-y}{\ln x - \ln y} , .$$ This does not hold for arbitrary $x, y > 0$ (a counterexample is $(x, y)=(0.2,1)$), but it may hold for all $x, y$ with $e^{-x}+e^{-y} \le 1$. The left-hand side has also the properties of a “mean” but I have never seen it before. – Martin R Aug 07 '24 at 18:00
  • @MartinR Perhaps it is the quasi-arithmetic mean. – River Li Aug 07 '24 at 23:29
  • @RiverLi: You are right, that is https://en.wikipedia.org/wiki/Quasi-arithmetic_mean for $f(x) = e^{-x}$. I had thought that the interpretation as inequality between means might help, but was not successful so far. I had also thought that the above inequality might hold in general (which would mean that your upper bound $1/(\ln 2 - \ln a)$ holds not only for $0 < a < 1$ but for $0 < a < 2$) until I found the counterexample. – Martin R Aug 08 '24 at 07:10
  • @MartinR Yes, for $a \in (1, 2)$, it is unbounded. – River Li Aug 08 '24 at 07:15