How can I set constraints on parameters in NelderMeadMinimize package?

Question

I read about NelderMeadMinimize package. How can I set constraints on the parameters? For example I want to minimize an equation with $10$ parameters such as ${a_1, a_2, ....}$. I want to put the condition $a_1 > 0$. How can I do that?

EDIT 1: As an example I have $10$ equations and $10$ unknown parameters which I want to find them by minimizing problem1:

problem1 = (-30 + A3/A1)^2 + (4 e3a α1 α3 + 2 d2 β1)^2 
  + (-2 A3 - 2 c2 α3 + 4 c4a α3^3 + 8 e3a α1 β1)^2 + (4 e3a α1^2 + 2 d2 β3)^2 
  + (-3.13995 - 2 c2 + 2 d2 + 12 c4a α1^2 - 4 e3a β3)^2 
  + (-1.49499 - 2 c2 + 2 d2 + 4 c4a α1^2 + 4 e3a β3)^2 
  + (-2.10157 - 4 c2 d2 + 24 c4a d2 α1^2 - 16 e3a^2 α3^2 - 8 d2 e3a β3)^2 
  + (-0.0277073 - 4 c2 d2 + 8 c4a d2 α1^2 - 16 e3a^2 α3^2 + 8 d2 e3a β3)^2 
  + (-2 A1 - 2 c2 α1 + 4 c4a α1^3 + 4 e3a α3 β1 + 4 e3a α1 β3)^2 
  + (0.00429025 α1^2)/((β1 Sqrt[1 - (2 c2 + 2 d2 - 4 c4a α1^2 - 4 e3a β3)/Sqrt[ 64 
         e3a^2 α3^2 + (2 c2 + 2 d2 - 4 c4a α1^2 - 4 e3a β3)^2]])/(Sqrt[2] α1) 
  + Sqrt[1 + (2 c2 + 2 d2 - 4 c4a α1^2 - 4 e3a β3)/Sqrt[64 e3a^2 α3^2 
                       + (2 c2 + 2 d2 - 4 c4a α1^2 - 4 e3a β3)^2]]/Sqrt[2])^2;        

parameters1 = {c2, d2, e3a, c4a, α1, α3, β1, β3, A1, A3};    

function1 = Function @@ {parameters1, problem1};    

minimizer = 
  With[{nelderMead = NelderMeadMinimize`Dump`CompiledNelderMead[function1, parameters1], 
        bounds = {-1, 1}, dimension = Length[parameters1], 
        tolerance = Sqrt[$MachineEpsilon]}, 

        Compile[{{dummy, _Integer, 0}}, 
           nelderMead[RandomReal[bounds, {dimension + 1, dimension}], tolerance, -1], 
           CompilationOptions -> {"ExpressionOptimization" -> True, 
                                  "InlineCompiledFunctions" -> True}, 
           RuntimeOptions -> {"CompareWithTolerance" -> False, 
                              "EvaluateSymbolically" -> False}, 
           RuntimeAttributes -> Listable(*"Parallelization"->True*), 
           CompilationTarget -> "C"]];  

       ans = {First[#], Thread[parameters1 -> Rest[#]]} & /@ 
                Take[minimizer@Range[3000] // Sort, 10]

I know that $\alpha_1$, $\alpha_3$, $\beta_1$, $\beta_3$, $A_1$, and $A_3$ should be positive. How can I put these constraints on these parameters when applying NelderMead minimization? Is it possible?

EDIT 2

First one should download the package and also needs to run:

Needs["CCompilerDriver`"]

Could you provide a small example ? – b.gates.you.know.what Nov 09 '13 at 09:57 — b.gates.you.know.what, Nov 09 '13 at 09:57
Sure, I put an example in EDIT 1. – Kheeyal Nov 09 '13 at 12:04 — Kheeyal, Nov 09 '13 at 12:04

score 5 · Answer 1 · answered Nov 25 '13 at 21:17

Unfortunately, as mentioned in the answer in which I presented it, the package does not support constraints at the moment. Since the Nelder-Mead method only deals with unconstrained problems anyway, the most realistic approach is probably to transform your constrained problem into an unconstrained one with the same solution. This is not as easy as it sounds, since one must take care not to introduce spurious local minima in the process. NMinimize does it, for example, with penalty/barrier function methods combined with co- and post-processing steps using explicitly constrained algorithms such as the nonlinear interior point method.

In particular cases (especially those about which you have some knowledge of their characteristics) this may not be too difficult to do manually and in an a priori fashion, e.g. by the method of Lagrange multipliers. However, to automatically and robustly deal with all possible constraint types is quite complicated, requiring the construction of a new objective function that depends not only on the problem and its constraints but which also has parameters that vary with the progress of the optimization process. This adds considerable overhead to the algorithm (NMinimize spends a lot of its time checking whether constraints are being fulfilled, and if not, dynamically fixing up the process), and is also rather difficult to implement in compiled code because it usually requires some kind of symbolic manipulation.

The primary purpose of the NelderMeadMinimize` package is to provide a minimalistic implementation of the Nelder-Mead algorithm that is fast and can be expressed entirely in compiled code. I am not sure if it is even possible to handle general constraints in the way that NMinimize does while retaining the performance advantage given the current limitations of the Mathematica compiler and virtual machine. Nonetheless, I do appreciate that not being able to work with constraints at all is a significant limitation. If someone has written code that accomplishes similar preprocessing work to that which NMinimize performs (e.g. taking a constrained problem and using the KKT conditions to produce an equivalent unconstrained problem), I would be very interested to see it posted as an answer here.

How can I set constraints on parameters in NelderMeadMinimize package?

1 Answers1