0

For the standard definition of Lipschitz continuity we have $$||f(x) - f(y)|| \leq L||x-y||$$

But what do we do if we define $$f(a,b) = \gamma_1(a) + \gamma_2(b) : a \in R^{n \times m}, b \in R^{n \times k} $$ $\gamma_1, \gamma_2$ both have Lipschitz constants $\gamma_{1_L},\gamma_{2_L}$ respectively.

How can we define the Lipschitz constant for $f$?

Bernard
  • 175,478
Armen Aghajanyan
  • 233
  • 1
    $\sup_{x}\nabla_x f(x) = \sup_{a,b} \nabla_{a,b}f(a,b) = \sup_{a,b} \nabla_a \gamma_1(a) + \nabla_b\gamma_2(b) = \gamma_{1_L} + \gamma_{2_L}$ – Dhruv Kohli Jul 03 '17 at 19:11
  • @Bob1123 $|a-b|$ doesnt neccessarily have to be defined since we treat variable $x$ in our definition as $\gamma_1(a) + \gamma_2(b)$. Am I correct in this reasoning? – Armen Aghajanyan Jul 03 '17 at 19:27
  • https://math.stackexchange.com/questions/1253552/how-do-i-show-that-for-a-multivariate-function-lipschitz-continuity-in-each-var – tryst with freedom Mar 05 '22 at 10:00

1 Answers1

1

First, the answer depends on the choice of norm/metric on $\mathbb{R}^{n\times m} \times \mathbb{R}^{n\times k}$. Now let's estimate:

$$\|f(a,b)-f(c,d)\| = \|\gamma_1(a)+\gamma_2(b) - \gamma_1(c)-\gamma_2(d)\|,$$

so rearranging and using the triangle inequality we obtain

$$\|f(a,b)-f(c,d)\| \leq \|\gamma_1(a)-\gamma_1(c)\| + \|\gamma_2(b)-\gamma_2(d)\| \\\leq \text{Lip}(\gamma_1) \|a-c\| + \text{Lip}(\gamma_2)\|b-d\|.$$

Hence

$$\|f(a,b)-f(c,d)\| \leq \max\{\text{Lip}(\gamma_1),\text{Lip}(\gamma_2)\}\left(\|a-c\| + \|b-d\|\right).$$

If you use the norm $\|(a,b)\|:=\max\{\|a\|,\|b\|\}$ on $\mathbb{R}^{n\times m} \times \mathbb{R}^{n\times k}$, then $L:= \max\{\text{Lip}(\gamma_1),\text{Lip}(\gamma_2)\}$ is a Lipschitz constant for $f$. If you use a different norm, then you need to do a little more work.

Matthew Kvalheim
  • 1,990
  • Thank you for the answer. I have a couple of questions. If we say that both $\gamma$ have a bounded first derivative then the Lipshitz constant will be the $sup f'(x)$ which is consistent with @expittpz0 comment. But it seems to me like your answer is also correct. Could you explain why we get the max vs the sum of the individual Lipshitz constants? Thanks you! – Armen Aghajanyan Jul 03 '17 at 20:03
  • I don't think the comment you mentioned make sense. How do you take the supremum of a linear operator (the derivative)? There should be norms of those derivatives in the expression in that comment -- but you need to specify explicitly what those norms are. – Matthew Kvalheim Jul 03 '17 at 20:17