1

I want to find the following saddle point of the function of 10 variables:

saddle_point.

I am not able to find the stationary points by setting the first derivatives to 0, so I need do optimize the function numerically. I think that that performing the above-mentioned optimization is equivalent to finding a set of {a1,a2,a3,a4,a5} that would maximize the following function:

InnerMin[a1,a2,3,a4,a5]:=NMinimize[f[a1,a2,a3,a4,a5,b1,b2,b3,b4,b5],{b1,b2,b3,b4,b5}]

That is how I would do the inner minimization, so I get the value of that function for the values of a1,a2,a3,a4 and a5 I provide as an input. Does anybody know how to find in practice a set of a1, a2, a3, a4 and a5 that would maximize the InnerMin function, or in the other words how to perform the outer optimization numerically (coupled with the inner minimization via the InnerMin function)?

Glorfindel
  • 547
  • 1
  • 8
  • 14
ogledala
  • 367
  • 2
  • 10

1 Answers1

2

What you have proposed works in principal: 

inner[y_ /; NumericQ[y]] := (
                  lastx =  NMinimize[-7 - 6 x + x^2 - 8 y - y^2, x];
                  First@lastx )
NMaximize[inner[y] , y ]
lastx

{-1.77636*10^-15, {y -> -4.}} {-1.77636*10^-15, {x -> 3.}}}

When you see how slow this is ( a whole minute) with my simple example I think you'll reconsider though..

Re comment, here is a 4-variable example:

 inner[y_, z_] /; And @@ NumericQ /@ {y, z} := (
        lastwx = 
          NMinimize[
          -4 w + 6 w^2 - 4 w^3 + w^4 - 4 x + x^2 + 108 y 
               - 54 y^2 + 12 y^3 - y^4 + 8 z - z^2 , {w, x}];
           First@lastwx )
 NMaximize[inner[y, z], {y, z}]
 lastwx

{92., {y -> 2.99739, z -> 4.}}

{92., {w -> 1.00001, x -> 2.}}

Exact answer should be 1,2,3,4 ..

This only works because we know a a priori which variables to min/max and the function is really nice with just one stationary point.

I think for the general problem you should be doing your own gradient descent.

george2079
  • 38,913
  • 1
  • 43
  • 110
  • Thanks a lot, what should I put in my case for NumericQ[y], if I predifined function f[a1_,a2_,a3_,a4_,a5_,y1_,y2_,y3_,y4_,y5_], and want to optimize it how I have stated in the question. Just to make sure I don't make a mistake in the input, is this correct: inner[a1_, a2_, a3_, a4_, a5_ /; NumericQ[{a1,a2,a3,a4,a5}]] := (lastx = NMinimize[ f[a1, a2, a3, a4, a5, b1, b2, b3, b4, b5], {b1, b2, b3, b3, b4, b5}]; First@lastx) NMaximize[inner[a1, a2, a3, a4, a5], {a1, a2, a3, a4, a5}] lastx. It gives me some results, and reports the errors: – ogledala Feb 07 '14 at 22:50
  • NMinimize::cvdiv: Failed to converge to a solution. The function may be unbounded. Are therefore results meaningful? – ogledala Feb 07 '14 at 22:53
  • It is so strange to me, I have not changed anything and know the error is:

    NMaximize::nnum: The function value -inner[0.643476,0.280303,0.964763,0.766972] is not a number at {b1,b2,b3,b4} = {0.643476,0.766972,0.280303,0.964763}. >>. Any ideas how to tackle this? @george2079

     – ogledala Feb 08 '14 at 00:15
  • i'd suggest working on a simple solvable problem (eg w^2+x^2-y^2-z^2 ) to sort out the syntax. Your errors relate to your function which you havent given.. But really i think you need to do something more sophisticated. (study the links Daniel posted for starters ) – george2079 Feb 08 '14 at 13:12
  • Before going to something sophisticated, I'd like to understand the principles of your code. Here is what I did with your example:

    f[x_, y_, z_, w_] := w^2 + x^2 - y^2 - z^2 inner[x_, y_ /; NumericQ[{x, y}]] := (lastx = NMinimize[f[x, y, z, w], {z, w}]; First@lastx) NMaximize[inner[x, y], {x, y}] lastx

    and this was the error message:

    NMaximize::nnum: The function value -inner[0.918621,0.716689] is not a number at {x,y} = {0.918621,0.716689}. >>

    What I am doing wrong?

     – ogledala Feb 08 '14 at 21:49