Let $X$ and $Y$ be finite dimensional vector spaces. Say that a continuous function $f:X \to Y$ is piecewise affine if there is a finite polyhedral decomposition $\mathcal{P}$ of $X$ such that for every polyhedral cell $P \in \mathcal{P}$, the restriction $f_P = f \big \rvert_P$ is given by $f_P(x) = A_Px + b_P$ for some linear map $A_P:X \to Y$ and a vector $b_P \in Y$.
I am investigating Filippov solutions to the generalized (Clarke) gradient descent of the potential function $V(x) = \frac{1}{2} \|f(x)\|^2$ for a piecewise affine function $f$. That is, an absolutely continuous path $x(t)$ in $X$ such that $-\dot{x}(t)$ is in the Clarke gradient of $V$ for all $t$. On the interior of a top-dimensional cell $P$, these dynamics are given by $\dot{x} = - A_P^*A_Px - A_P^*b_P$. In particular, if the piecewise affine function is simply an affine function with one cell $P = X$, this flow converges to a least-squares minimizer for the equation $A_Px = b_P$. In particular, I'm interested in when this system converges to a point in the zero-set of $f$.
I’m primarily interested because of a connection to feed forward neural networks. Freely composing affine maps and the “projection” onto the positive orthant gives such a function. I would like to understand the landscape of solutions with the quadratic potential function $V$.
One issue I'm running into is checking if trajectories are bounded. I believe I have shown that if $x(t)$ diverges to infinity, it must infinitely bounce between multiple cells in the polyhedral decomposition $\mathcal{P}$. However, I have been unable to exhibit such a trajectory (even for bounded trajectories). Is such a trajectory possible? Intuitively, it seems like the continuity of $f$ plus the fact that $A^*A$ is positive semidefinite rules out such behavior, although I haven't been able to prove it.
I would be greatly interested in suggestions for either exhibiting a trajectory that infinitely moves between cells, or for a proof that such behavior is impossible. Thank you!
Edit: A few observations Since original asking this question, I have made a few observations that may be valuable. I feel like I can see the pieces of a tapestry, but I can't quite assemble them. I will enumerate a few of them here:
The continuity condition for a piecewise affine map is quite restrictive; in particular, if $P$ and $Q$ are two top-dimensional cells that meet on a boundary plane $B$, then $B$ can be uniquely extended to an affine hyperplane $W$ of codimension 1. Writing $W = U + u^{\perp}$, where $U$ is a linear subspace of $X$ and $u^\perp$ is in the orthogonal complement $U^\perp$, we may easily derive that: (a) $A_P$ and $A_Q$ agree on $U$; (b) $\ker(A_P) \cap B = \ker(A_Q) \cap B$; (c) $(A_P - A_Q) u^\perp = b_Q - b_P$.
Suppose $x_t$ is a bounded Filippov trajectory for the generalized gradient descent $\dot{x} \in - \partial V$. Then this paper proves that the trajectory converges to a generalized critical point $\overline{x}$ such that $0 \in \partial V(\overline{x})$. It follows that a bounded infinitely cycling trajectory is impossible.
If $x \in P$ for a cell $P$, then $\dot{x} \in \ker(A_P)^\perp$. From this it follows that if $\ker(A_P)$ does not contain an extreme direction of the polyhedral cell $P$, then for all boundary points $x$ of $P$ sufficiently far from the origin, the velocity vector $\dot{x}$ points inward (in a direction that reduces $\|x\|$).
More generally, for a top-dimensional cell, we have a few possibilities worth considering. Let $$Z_P := \{\text{OLS minimizers to the equation } A_P x = - b_P \}$$ which is a non-empty affine subspace of $X$. For $x \in P$, writing $x = z + \lambda y$ where $z \in Z_P$, $y \perp Z_P$, and $\lambda \in \mathbb{R}$, the angle between the gradient vector $-(A_P^*A_Px + A_P^*b_P)$ and $y$ is constant in $\lambda$.
I am extremely interested in seeing when (if ever) there is an unbounded trajectory. I am starting a bounty for this question. Thank you again.