Why is every $p$-norm convex?

Question

I know that $p$-norm of $x\in\Bbb{R}^n$ is defined as, for all $p\ge1$,$$\Vert{x}\Vert_p=\left(\sum_{i=1}^{n} \vert{x_i}\vert^p\right)^{1/p}.$$

The textbook refers to "Every norm is convex" for an example of convex functions.

I failed to prove $f(x)=\Vert{x}\Vert_p$ for all $p\ge1$, then tried to find the proof on the internet but I cannot find it.

Can someone let me understand why $p$-norm is convex for all $p\ge1$.

Answer 1

The Definition of a norm is:

Be V a Vectorspace, $\|\cdot\|: V \rightarrow \mathbb{R} $ is a norm $:\iff $

1. $\forall v \in V: \|v\|\ge0$ and $\|v\| =0 \iff v=0$ (positive/definite)
2. $\forall v\in V, \lambda\in \mathbb{R}: |\lambda|\|v\| =\|\lambda v\|$ (absolutely scaleable)
3. $\forall v,w\in V : \|v+w\| \le \|v\|+\|w\|$ (Triangle inequality)

The Definition of convex is:

$f:V\rightarrow\mathbb{R}$ is convex $:\iff$ $\forall v,w \in V, \lambda \in [0,1]: f(\lambda v+(1-\lambda )w)\le \lambda f(v) +(1-\lambda)f(w)$

So using the Triangle inequality and the fact that the norm is absolutely scalable, you can see that every Norm is convex: $$\|\lambda v+(1-\lambda )w\|\le\|\lambda v\|+\|(1-\lambda)w\| = \lambda\|v\|+(1- \lambda)\|w\|$$

So by definition every norm is convex. What is left to show is, that the p-norm is in fact a norm. The first two requirements are pretty easy to show, the third is hard. That is why it has its own name: the Minkowski Inequality which is a result of the Hölder inequality and shows that the triangle inequality holds for every p-norm (if p>1) and thus that it is a norm.

---

EDIT: Since this seems to be somewhat popular, I thought I would add a sketch of the proof of the Minkowski inequality.

1. You show Young's Inequality: $xy\le \frac{x^p}{p}+\frac{y^q}{q}\quad \forall q,p>1 \text{ with } \frac{1}{p}+\frac{1}{q}=1,\ \forall x,y\ge 0$.

You can do that by looking at the function $f(x)=\frac{x^p}{p}+\frac{y^q}{q}-xy$ find the critical point, show it is a minimum and is greater zero (derivatives).

2. You show the Hölder Inequality: $\|fg\|_1 \le \|f\|_p\|g\|_q \quad \forall q,p>1 \text{ with } \frac{1}{p}+\frac{1}{q}=1$

You can do that by setting $x=\frac{|f|}{||f||_p}$ and $y=\frac{|g|}{||g||_q}$ and plug them into young's inequality. You get \begin{align} &&\frac{|fg|}{\|f\|_p\|g\|_q}&\le \frac{|f|^p}{p\|f\|_p^p} + \frac{|g|^q}{q\|g\|_q^q} \\ \Rightarrow &&\int \frac{|fg|}{\|f\|_p\|g\|_q} d\mu &\le \int \frac{|f|^p}{p\|f\|_p^p}d\mu + \int \frac{|g|^q}{q\|g\|_q^q}d\mu \\ \Rightarrow &&\frac{\|fg\|_1}{\|f\|_p\|g\|_q}&\le \frac{1}{p}+\frac{1}{q}=1 \end{align} It works just the same for sequences or $\mathbb{R}^n$, you just use young's inequality for every index and then sum over it instead of using the integral.

3. And last the Minkowski Inequality: $\|x+y\|_p\le\|x\|_p+\|y\|_p \quad \forall p>1$

Set $q=\frac{p}{p-1}$ thus $q(p-1)=p$ and $\frac{1}{p}+\frac{1}{q}=1$. Then: \begin{align} \|x+y\|_p^p&=\int |x+y|^pd\mu\le\int |x+y|^{p-1}|x|d\mu+ \int |x+y|^{p-1}|y|d\mu \\ &\le \left(\int|x+y|^{q(p-1)}d\mu\right)^{1/q}\left(\int|x|^pd\mu\right)^{1/p} + \left(\int|x+y|^{q(p-1)}d\mu\right)^{1/q}\left(\int|y|^pd\mu\right)^{1/p} \\ &=\left(\int|x+y|^{p}d\mu\right)^{\frac{1}{p}\frac{p}{q}}(\|x\|_p+\|y\|_p) =\|x+y\|_p^{p/q}(\|x\|_p+\|y\|_p) \end{align}

If you realize that $p-\frac{p}{q}=p(1-\frac{1}{q})=1$ you are done.

Answer 2

Ref: Thomas Ahle’s answer here: Link.

Consider ${ \mathbb{R} ^n . }$ Let ${ p \geq 1 . }$

Is

$${ \lVert x \rVert _p := \left( \sum _{i=1} ^{n} \vert x _i \vert ^p \right) ^{\frac{1}{p}} }$$

a norm on ${ \mathbb{R} ^n }$?

Note that it suffices to show

$${ \text{To show: } \quad \left( \sum _{i=1} ^{n} (\vert x _i \vert + \vert y _i \vert ) ^p \right) ^{\frac{1}{p}} \leq \left( \sum _{i=1} ^{n} \vert x _i \vert ^p \right) ^{\frac{1}{p}} + \left( \sum _{i=1} ^{n} \vert y _i \vert ^p \right) ^{\frac{1}{p}} . }$$

Note that we can take limits of inequalities. Hence it suffices to show

$${ \text{To show: } \quad \left( \sum _{i=1} ^{n} ( x _i + y _i ) ^p \right) ^{\frac{1}{p}} \leq \left( \sum _{i=1} ^{n} x _i ^p \right) ^{\frac{1}{p}} + \left( \sum _{i=1} ^{n} y _i ^p \right) ^{\frac{1}{p}} }$$

for all ${ x, y \in (0, \infty) ^n . }$

Note that it suffices to show

$${ \text{To show: } \quad F(x) = \left(\sum x _i ^p \right) ^{\frac{1}{p}} \text{ is convex on } (0, \infty) ^n . }$$

Let ${ x \in (0, \infty) ^n . }$ Note that the Hessian of ${ F }$ is given by

$${ {\begin{aligned} &\, \nabla ^2 F(x) _{i, i} \\ = &\, \frac{\partial}{\partial x _i} \frac{1}{p} \left(\sum x _{\nu} ^ p \right) ^{\frac{1}{p} - 1 } p x _i ^{p - 1} \\ = &\, \left( \frac{1}{p} - 1 \right) \left( \sum x _{\nu} ^p \right) ^{\frac{1}{p} - 2} p x _i ^{p-1} x _i ^{p - 1} + \left(\sum x _{\nu} ^ p \right) ^{\frac{1}{p} - 1 } (p-1) x _i ^{p - 2} \\ = &\, \left( \frac{1}{p} - 1 \right) \left( \sum x _{\nu} ^p \right) ^{\frac{1}{p} - 2} x _i ^{p - 2} \left( p x _i ^{p } - p \left( \sum x _{\nu} ^p \right) \right) \\ = &\, (p-1) \left( \sum x _{\nu} ^p \right) ^{\frac{1}{p} - 2} x _i ^{p - 2} \left( \sum _{\nu \neq i} x _{\nu} ^p \right) \end{aligned}} }$$

and

$${ {\begin{aligned} &\, \nabla ^2 F(x) _{i, j} \\ = &\, \frac{\partial}{\partial x _j} \left(\sum x _{\nu} ^ p \right) ^{\frac{1}{p} - 1 } x _i ^{p - 1} \\ = &\, \left( \frac{1}{p} - 1 \right) \left( \sum x _{\nu} ^p \right) ^{\frac{1}{p} - 2} p x _j ^{p - 1} x _i ^{p - 1} \\ = &\, - (p-1) \left( \sum x _{\nu} ^p \right) ^{\frac{1}{p} - 2} x _i ^{p - 1} x _j ^{p - 1} \end{aligned}} }$$

for ${ i \neq j . }$

Hence

$${ \nabla ^2 F(x) = (p-1) F(x) ^{1 - 2p} A(x) }$$

where

$${ A(x) _{i, j} = {\begin{cases} x _i ^{p - 2} \left( \sum _{\nu \neq i} x _{\nu} ^p \right) \quad &\, \text{ for } i = j \\ - x _i ^{p - 1} x _j ^{p - 1} \quad &\, \text{ for } i \neq j . \end{cases}} }$$

Note that it suffices to show ${ \nabla ^2 F(x) }$ positive semidefinite.

Hence it suffices to show ${ A(x) }$ is positive semidefinite.

Let ${ v \in \mathbb{R} ^n . }$ Note that

$${ {\begin{aligned} &\, v ^T A(x) v \\ = &\, \sum _{i, j} A(x) _{i, j} v _i v _j \\ = &\, \sum _{i=1} ^{n} x _i ^{p - 2} \left( \sum _{\nu \neq i} x _{\nu} ^p \right) v _i ^2 - 2 \sum _{i < j} x _i ^{p - 1} v _i x _j ^{p - 1} v _j \\ = &\, \sum _{i=1} ^{n} x _i ^{p - 2} \left( \sum _{\nu} x _{\nu} ^p - x _i ^p \right) v _i ^2 - 2 \sum _{i < j} x _i ^{p - 1} v _i x _j ^{p - 1} v _j \\ = &\, \left( \sum x _{\nu} ^p \right) \sum _{i=1} ^{n} x _i ^{p - 2} v _i ^2 - \sum _{i=1} ^{n} x _i ^{2p - 2} v _i ^2 - 2 \sum _{i < j} x _i ^{p - 1} v _i x _j ^{p - 1} v _j \\ = &\, \left( \sum x _{\nu} ^p \right) \left( \sum _{i=1} ^{n} x _i ^{p - 2} v _i ^2 \right) - \left( \sum _{i=1} ^{n} x _i ^{p - 1} v _i \right) ^2 \\ \geq &\, 0 \end{aligned}} }$$

by Cauchy-Schwarz between the vectors ${ (x _i ^{p /2}) }$ and ${ (x _i ^{p/2 - 1} v _i) . }$

Hence ${ A(x) }$ is positive semidefinite.

Hence ${ \lVert x \rVert _p := \left( \sum _{i=1} ^n \vert x _i \vert ^ p \right) ^{\frac{1}{p}} }$ is a norm on ${ \mathbb{R} ^n , }$ as needed.