10

This question is motivated by the recent question about searching inside of the NB files.

According to the Documentation, ToString expects a high-level WL expression as the first argument: 

expr = Row[{Style["Format", Bold], "ted", " string"}]
ToString[expr]

Formatted string
Formatted string

Unlike Rasterize it does not interpret raw boxes as an expression in the low-level box language:

cell = Cell[BoxData@ToBoxes@expr, "Output"];
Rasterize[cell]
ToString[cell]

Cell[BoxData[TemplateBox[{StyleBox["Format", Bold, StripOnInput -> False], "ted", " string"}, RowDefault]], Output]

Unfortunately MakeExpression isn't always able to convert a valid raw boxes into the corresponding high-level expression (although in the above simple case it can). Also conversion from raw boxes into high-level expression breaks the original formatting (for example, it does not preserve the infix, prefix and postfix notations, and the author's line breaks etc.) and should best be avoided.

Is it possible to convert arbitrary raw boxes directly into the corresponding String without resorting to MakeExpression?

I'm especially interested in converting raw Cells into pure strings for the purposes of textual searching inside of NB files.

Community
  • 1
Alexey Popkov
  • 61,809
  • 7
  • 149
  • 368
  • @Kuba The example is in the question (cell). It is about the raw boxes input and the expected output (what Rasterize gives as an image which I don't want) and wanted String without formatting and suitable for StringMatchQ etc. – Alexey Popkov May 18 '16 at 10:13
  • @Kuba Since I need to convert all Cells in a Notebook to plaint text, conversion cell-by-cell will be extremely slow (and the final goal is to convert a lot of Notebooks for the purposes of textual searching inside of them). Also is it possible to pass the PageWidth -> Infinity option somehow? I tried to include it into Cell expression and set at the global level but FrontEnd`ExportPacket ignores it. – Alexey Popkov May 18 '16 at 13:42
  • 3
    Notebook works too: First[FrontEndExecute[ FrontEnd`ExportPacket[NotebookGet@EvaluationNotebook[], "PlainText"]]] – Kuba May 18 '16 at 13:44
  • @Kuba That's great! And it seems to have PageWidth -> Infinity set by default! You can post this as an answer. – Alexey Popkov May 18 '16 at 14:08
  • But don't you feel it would be better to mark it as a duplicate of the linked topic? – Kuba May 19 '16 at 05:30
  • accidentally deleted the comment with the link, here it is: How can I get the unchanged Box form of an arbitrary expression? – Kuba May 19 '16 at 05:37
  • 1
    @Kuba Current question can be considered as a duplicate of How do I extract the contents of a selected cell as plain text? but has an emphasis on conversion of a very large amounts of raw boxes into text. The FrontEnd`ExportPacket solution mostly works but sometimes returns just Null (second attempt usually gives success). Do you have any idea how to deal with this problem? It looks like sometimes the call to FrontEnd`ExportPacket is ended before the data is returned. Or may be FrontEnd sometimes just doesn't accept a call? – Alexey Popkov May 19 '16 at 06:06

1 Answers1

8

As Kuba notices in the comment, undocumented FrontEnd`ExportPacket allows conversion of a whole Notebook into plain text:

nb = NotebookGet@EvaluationNotebook[];
First[FrontEndExecute[FrontEnd`ExportPacket[nb, "PlainText"]]]

"nb=NotebookGet@EvaluationNotebook[];

First[FrontEndExecute[FrontEnd`ExportPacket[nb,"PlainText"]]]"

But it isn't the whole story. Surprisingly FrontEnd evaluates and updates the Dynamic objects in the Notebook sent as the first argument of FrontEnd`ExportPacket!

With Mathematica 10.4.1 when trying to convert to plain text the "Views.nb" Documentation Notebook (warning: evaluation of the following code can make Mathematica not responding for several minutes!)

nbWithDynamic = 
  Get@FileNameJoin[{$InstallationDirectory, "Documentation", "English", 
     "System", "Tutorials", "Views.nb"}];

First[FrontEndExecute[FrontEnd`ExportPacket[nbWithDynamic, "PlainText"]]];

I observe that Kernel downloads a lot of data from the Wolfram paclet server obviously in order to update the Dynamic expressions in this Notebook (the most of the data should be due to the second example under the "SlideView" section which downloads shapes for all countries available via CountryData[]).

This behavior is especially strange because no Dynamic objects from that Notebook are displayed on the screen (according to the Documentation, Dynamic is updated only when displayed on the screen). So this is a bug and serious security violation. (UPDATE: As explained by John Fultz, exporting in some fundamental sense is equivalent to displaying and this behavior is by design.)

To prevent this one should set the DynamicUpdating -> False option for a Notebook before sending it to the FrontEnd:

nb = NotebookGet@EvaluationNotebook[];
nbDynamicOff = Insert[nb, DynamicUpdating -> False, 2];
First[FrontEndExecute[FrontEnd`ExportPacket[nbDynamicOff, "PlainText"]]]

"nb=NotebookGet@EvaluationNotebook[];

nbDynamicOff=Insert[nb,DynamicUpdating->False,2];


First[FrontEndExecute[FrontEnd`ExportPacket[nbDynamicOff,"PlainText"]]]"

Disabling the DynamicUpdating also solves the problem with FrontEnd`ExportPacket sometimes returning just Null (but at the second attempt on the same file it returns the expected result). Of course it also speeds up the conversion several times and solves the problem with the Dynamic updating timeout dialog appearing during the conversion (what stops the process of computation until you press the "Continue waiting" button).

Community
  • 1
Alexey Popkov
  • 61,809
  • 7
  • 149
  • 368
  • Just want to let you know that I've seen your comments. I will respond as soon as I have something to say :) – Kuba May 19 '16 at 19:29
  • 3
    I don't see the problem with this. It is perfectly reasonable (in fact, I claim desirable) to say that "Plain text" means "what the Dynamic displays). In some fundamental sense, you are "displaying" the Dynamic to the exporter. Would you really expect Export["foo.png", expr] to not render dynamics reference by expr? Now, it might be that it's doing too much silly work in the case of some views. But while I'd probably concede some of the details could be better managed, I think the fundamental direction of the implementation is right. – John Fultz May 26 '16 at 20:01
  • @JohnFultz Thank you for the explanation. But what if I want to export only cached (pre-rendered) versions of Dynamic saved in the Notebook (i.e. as when DynamicUpdating -> False)? My solution via adding DynamicUpdating -> False into Notebook code looks a bit ugly. Is there a better way? Also does DynamicUpdating -> False disable NotebookDynamicExpression and CellDynamicExpression (it is not exactly clear from the Documentation)? – Alexey Popkov May 27 '16 at 05:07
  • 2
    DynamicUpdating->False doesn't cache anything into the notebook. MaintainDynamicCaches->True does. I would expect the combination of those two options probably does what you want, but I haven't tried it. And yes, I would also expect DynamicUpdating->False to disable NotebookDynamicExpression and CellDynamicExpression for any notebook it's been set on. – John Fultz May 27 '16 at 23:57