1

1.Trying to add an interpolated function to neural networks.

ifun = Interpolation[Table[{x, Tanh[x]}, {x, -100, 100, 0.2}], 
  InterpolationOrder -> 1]
Plot[{ifun[x], Tanh[x]}, {x, -4, 4}]

2.We implement the function.

net = NetChain[{30, ElementwiseLayer[ifun], 20, 
   ElementwiseLayer[ifun], 3, SoftmaxLayer[]}, "Input" -> {2}, 
  "Output" -> NetDecoder[{"Class", {Red, Green, Blue}}]]

3.Error

ElementwiseLayer::invscf: InterpolatingFunction[{{-100.,100.}},... could not be symbolically evaluated as a unary scalar function.

4.Please tell me how to fix it.

Michael E2
  • 235,386
  • 17
  • 334
  • 747
Глеб
  • 11
  • 2

2 Answers2

4

Using @CarlWoll's InterpolationToPiecewise will work. ElementWiseLayer complains that the InterpolatingFunction does not "symbolically evaluate." I take that to mean that it accepts only certain expressions as symbolic expressions.

(* https://mathematica.stackexchange.com/a/212753 *)
InterpolationToPiecewise[if_, x_] := 
 Module[{main, default, grid}, grid = if["Grid"];
   Piecewise[{if@"GetPolynomial"[#, x - #], x < First@#} & /@ 
     grid[[2 ;; -2]], if@"GetPolynomial"[#, x - #] &@grid[[-1]]]] /; 
  if["InterpolationMethod"] == "Hermite";

pwfun[x_] = InterpolationToPiecewise[ifun, x];

net = NetChain[{30, ElementwiseLayer[pwfun], 20, 
   ElementwiseLayer[pwfun], 3, SoftmaxLayer[]}, "Input" -> {2}, 
  "Output" -> NetDecoder[{"Class", {Red, Green, Blue}}]]
Michael E2
  • 235,386
  • 17
  • 334
  • 747
  • Thanks, it helped.Another error occurred while training the network. NetTrain::interr2 : An unknown internal error occurred.Consult Internal$ \ LastInternalFailure for potential information.``

    Consult Internal` $ LastInternalFailure

    InvalidJson Non - JSON value : {{KeyAbsent, $Failed}, {KeyAbsent, $Failed}}.

     – Глеб May 15 '20 at 15:02
  • 1
    @Глеб I'm not sure what causes that. Maybe ask another question about that issue. Or ask Wolfram Support. (This is somewhat similar but unanswered: https://mathematica.stackexchange.com/questions/212986/how-can-i-prevent-errors-with-replicatelayer) – Michael E2 May 15 '20 at 16:11
0

Currently, piecewise function support in ElementwiseLayer is somewhat limited. It doesn't allow logical disjunctions (e.g. x<10. || x<9. ...) which were generated when PiecewiseFunction automatically unified all of the -1s and 1s in the function value list. Simplifying the expression with PiecewiseExpand would help, but then again, this would generate unsupported conditions of the form a < x <= b. Seems that ElementwiseLayer currently only supports simple inequalities inside of PiecewiseFunction.

The quickest solution here (in terms of my work) is to use the Which construct instead of PiecewiseFunction:

InterpolationToWhich[if_, x_] := 
  Module[{grid}, grid = if["Grid"];
    Which @@ Flatten[
      {
       {x < First@#, if@"GetPolynomial"[#, x - #]} & /@ grid[[2 ;; -2]],
       True,
       if@"GetPolynomial"[#, x - #] &@grid[[-1]]
      }
    ]
  ] /; if["InterpolationMethod"] == "Hermite";

wfun[x_] = InterpolationToWhich[ifun, x];
ElementwiseLayer[wfun]

There's much to be optimized, though. One could use a binary tree of Ifs to reduce the layer's time complexity from linear to logarithmic.

aooiiii
  • 609
  • 3
  • 7