Shell Emulation inside Mathematica

Question

I am trying to create a Mathematica/shell syntax hybrid. I would like the following command

wget -qO- "http://google.com/space line
break\"" $var

to be interpreted as the following.

wget["-qO-","http://google.com/line
break\"",var]

Notice there are both line breaks and escaped quotes in the command.

I will add a bounty soon. I'm not the best at Regular Expressions but will try it myself.

Answer 1

First of all, I agree, as OP mentioned in his comment, ANTLR is one of the proper ways to go.

Now for this specific task, it might be easier to just compose a parser in the "dirty" way, except we don't have to go so far to regex. In my opinion Mathematica's StringExpression is much more powerful and very suitable for the job.

All we have to do is (as OP already did in his answer) to write a parser for CellEvaluationFunction.

However one thing should be take care of when creating the Cell:

The shell emulator is going to process customized commands, which are unlikely to be similar to Mathematica's syntax, so the created Cell should not involve any of the Box typesetting system. One way to ensure this is to create the Cell as Cell[ "", ... ] instead of Cell[ BoxData[""], ... ] (just like what happens when you convert a Cell to the "Raw InputForm"). That case the CellEvaluationFunction receives input as a raw string rather than Boxes, so we can avoid lots of unnecessary work.

One simple parser could be like the following:

Clear[shellEmu`interpretRules, shellEmu`stringMismatchMsg, shellEmu`parser]

shellEmu`interpretRules = {
            str_String /; 
                    StringMatchQ[str, "$" ~~ LetterCharacter ~~ WordCharacter ...] :> ToExpression[StringDrop[str, 1]],
            str_String /;
                    StringMatchQ[str, NumberString]                                :> ToExpression[str]
                          };

shellEmu`stringMismatchMsg[strMarkerPos_] :=
    Failure["OddQuotations", <|
            "MessageTemplate"   -> StringTemplate["Number of quotation markers should be even and positive rather than `Number`"],
            "MessageParameters" -> <|"Number" -> Length[strMarkerPos]|>
                             |>]

shellEmu`parser[inStr_String, form_] :=
    Module[{
               workStr = StringJoin["< ", inStr, " >"],
               strLen,
               strMarkerPos,
               strMarkerPosPart, nonstrMarkerPosPart,
               posMergFunc = Function[pos, Interval @@ pos // List @@ # &],
               res
           },
           strLen           = StringLength@workStr;
           strMarkerPos     = StringPosition[workStr, #][[;; , -1]] & /@ {"\"", "\\\""} // Complement @@ # &;
           strMarkerPosPart = Partition[strMarkerPos, 2, 2, {1, 1}, {}];
           If[Length[strMarkerPos] != 0 && Length[strMarkerPosPart[[-1]]] != 2,
                   Return@shellEmu`stringMismatchMsg[strMarkerPos]
             ];
           strMarkerPosPart    = strMarkerPosPart // posMergFunc;
           nonstrMarkerPosPart = {0, Sequence @@ strMarkerPosPart, strLen + 1} // Flatten // (Partition[#, 2] + {1, -1}) & // Transpose // posMergFunc;
           res = StringTake[workStr, #] & /@ {nonstrMarkerPosPart, strMarkerPosPart};
           res //
               RightComposition[
                   MapAt[ToExpression /@ # &, #, 2] &,
                   MapAt[StringSplit[#, Whitespace] & /@ # &, #, 1] &,
                   Riffle @@ # &,
                   Flatten, #[[2 ;; -2]] &,
                   MapAt[ToExpression, #, 1] &,
                   # /. shellEmu`interpretRules &,
                   #[[1]] @@ Rest[#] &
                   ]
          ]

With that we can now create our ShellEmu Cell as following:

Cell["", "ShellEmu",
        Evaluatable -> True,
        CellEvaluationFunction -> shellEmu`parser,

        Background -> Hue[0.09, 0.41, 0.33],

        CellMargins -> {{50, 0}, {0, 0}},
        CellFrame -> {{False, False}, {5, 5}},
        CellFrameMargins -> {{10, 10}, {10, 10}},
        CellFrameColor -> Hue[0.09, 0.41, 0.56],

        FontFamily -> "Consolas",
        FontColor -> GrayLevel[0.95],
        FontWeight -> Bold,
        Hyphenation -> False,
        FrontEnd`AutoQuoteCharacters -> {},
        FrontEnd`PasteAutoQuoteCharacters -> {}
        ] // CellPrint

Note for a clearer evidence that the command has been correctly parsed, I manually copied the output as an "Input"-style Cell with syntax highlight.

And mismatch of quotation markers will cause a failure:

Now obviously CellPrint is too cumbersome, so we're going to get the job done more automatically.

By defining a new style in the stylesheet as following

Cell[StyleData["ShellEmu"],
      CellFrame                -> {{False, False}, {5, 5}},
      CellMargins              -> {{50, 0}, {0, 0}},
      Evaluatable              -> True,
      CellEvaluationFunction   -> shellEmu`parser,
      GeneratedCell            -> True,
      CellAutoOverwrite        -> True,
      CellFrameMargins         -> {{10, 10}, {10, 10}},
      CellFrameColor           -> Hue[0.09, 0.41, 0.56],
      Hyphenation              -> False,
      AutoQuoteCharacters      -> {},
      PasteAutoQuoteCharacters -> {},
      MenuCommandKey           -> "L",
      FontFamily               -> "Consolas",
      FontWeight               -> Bold,
      FontColor                -> GrayLevel[0.95],
      Background               -> Hue[0.09, 0.41, 0.33]]

we should be able to create "ShellEmu" Cell simply by a shortcut Alt+L.

We can define functions matching the parsed result, say:

Clear[plot]
plot[f_, var_, "from", start_, "to", end_] :=
    Module[{interm},
        Echo[Inactive[plot][f, var, "from", start, "to", end] // FullForm];
        interm = 
            Inactive[Plot][ToExpression@f, ToExpression /@ {var, start, end}];
        Echo[interm // FullForm];
        Activate[interm]
        ]

Answer 2

I extended the original question to support piping like the following.

wget -qO- "http://google.com" // cat

and it outputs the following.

cat[][wget[-qO-,http://google.com][]]

To get the following.

CellPrint@
  Cell[BoxData[""], "Input", Evaluatable -> True, 
   CellEvaluationFunction -> 
    Function[
     Module[{t}, 
      t = List@
        ReplaceRepeated[NotebookRead[EvaluationCell[]][[1, 1]], 
         RowBox[{x__}] :> x];
      t = 
       Map[Module[{t = #, a, addbacksemicolon = ""}, 
          If[t[[-1]] == ";", addbacksemicolon = ";";
           t = Delete[t, -1]];

          If[Length@t >= 2 && 
            StringMatchQ[t[[1]], 
             RegularExpression["([a-zA-Z0-9])*"]] && t[[2]] == " ", 
           t = SplitBy[t, " " == # &] // DeleteCases@{" "};
           t = SplitBy[t, {"//"} == # &] // DeleteCases@{{"//"}};

           t = Map[
             Map[If[(Characters[#])[[1, 1]] == "\"", #, 
                 StringJoin[{"\"", #, "\""}]] &, #] &, t];

           t = Map[
             StringJoin[StringTrim[#[[1]], "\""], "[", 
               StringJoin@Riffle[#[[2 ;;]], ","], "]"] &, t];
           (*Print["beforeRiffleStringJoin",t];*)

           t[[1]] = t[[1]] <> "[]";

           t = StringJoin@Riffle[t, " // "] <> "" <> 
             addbacksemicolon;
           (* Print@t;*)
           t, t]] &,
        t = (SplitBy[If[Depth[t] == 2, t, t[[1]]], 
            "\[IndentingNewLine]" == # &] // 
           DeleteCases@{"\[IndentingNewLine]"});
        t
        ];
      t = ToExpression@StringJoin@Riffle[t, "\[IndentingNewLine]"];
      t
      ]]];
cd[x_] := Function[SetDirectory[x];]
fn = Function[Null, 
   ReplacePart[
    Function @@ 
     Hold[Null, Quiet[## &, {Function::slotn}], HoldAll], {2, 1, 0} ->
      Function @@ Hold[##]], HoldAll];
cmds = {echo, cat, ls, wget, grep, ps, top, df, killv, rm, cp, mv, 
   cat, mount, chmod, chown, passwd, mkdir, ifconfig, uname, whereis, 
   whatis, locate, find, sed, awk, diff , sort, xargs, uname, 
   ifconfig, locate , man, tail, less, ping, date};
Map[Function[{f},
   With[{n = ToString[Unevaluated[f]]},
    f[x___ : String] := fn[
       Import[
        "!LD_LIBRARY_PATH= " <> 
         If[ToString@#1 == "#1", "", "echo \"" <> #1 <> "\" | "] <> 
         n <> " " <> StringJoin@Riffle[{x}, " "], "Text"]
       ];
    ]
   ], cmds];