6

I've tried to give this a generic title to help future searchers, but concretely I am trying to parse YAML data. Import doesn't seem to support it natively and I can't find any code for it online, so I thought the easiest approach would be to wrap a C library (libyaml).

I haven't used the WSTP stuff yet, but the documentation looks pretty good at least for the easy cases, I was wondering if someone give give some tips though on

Dan
  • 870
  • 4
  • 9
  • To send an association using MathLink, just send an expression such as Association[Rule["foo", 1], Rule["bar", 2]]. There's nothing special about associations compared to other expressions when sending them through MathLink. Can you explain about streaming APIs to those of us who are not familiar with this concept? What difficulty do you expect and why? – Szabolcs Sep 27 '16 at 10:41
  • If you mean that you read the parts one by one, you can also just keep writing the result to the (Math)Link one by one, as you receive them. However, it is necessary to know when there are no more items, so you can wrap up things. – Szabolcs Sep 27 '16 at 10:50
  • How's this project going? – Szabolcs Sep 29 '16 at 11:00
  • I'm ashamed to admit that as I read this I became aware of J/link and kind of side-stepped the issue this time by using a Java based YAML parser (snakeyaml, in case anyone is interested). I do still expect to use this in the future though (for more specialized libs). – Dan Oct 02 '16 at 06:21
  • If you have a working YAML importer, it would be nice if you could post it. Maybe a self-answered question separate from this? – Szabolcs Oct 02 '16 at 09:51
  • Sure, I need to figure out some packaging but I'd be happy to contribute it. – Dan Oct 04 '16 at 03:17
  • I posted it here: https://github.com/dfarmer/MathematicaYamlSupport I want to address my top 2 TODO items before I make a self-answered question, but just in case anyone is desperate. – Dan Oct 07 '16 at 16:01
  • 1
    Another YAML importer was just published. You can still post yours as an answer. http://mathematica.stackexchange.com/a/128506/12 – Szabolcs Oct 12 '16 at 06:44
  • 1
    @Dan haha, the odds that we both write a YAML importer on SnakeYAML within 48 hrs of each other... :) Tip: calling into Java is expensive, so doing the parsing from Mathematica is slower than doing it in Java. For a 30kb document, for example, yours takes 0.6 seconds; vs. 0.02 sec done in Java. Compare: https://github.com/zbjornson/MYaml/blob/master/javaSource/YamlLinkUtils.java#L47 – ZachB Oct 12 '16 at 17:26

1 Answers1

6

You can use MathLink (recently called WSTP) for this.

MathLink is an interprocess communication protocol. You can create links and send arbitrary Mathematica expressions through it. This is the must-read tutorial for MathLink. Use it in conjunction with the documentation.

Typically, you would create a function that is callable directly from Mathematica, e.g. by creating an installable MathLink program. You will need to write the expression to be returned by this function to a MathLink link. When using Installable programs, you simple set the return type to Manual in your template file and write to stdlink.

When parsing a YAML file, you probably need to anticipate errors and maybe just return $Failed if an error has occurred. To deal with this I suggest not writing the expression to stdlink at first. Instead, create a loopback link. Write the parts of the expression to the link one by one, as you receive them from libyaml. When you're finished, just transfer everything to stdlink. If you must abort because of the error, throw away the contents of the loopback link, and MLPutSymbol a $Failed symbol in stdlink.

Sending associations is the same as sending any other expression. Send them in the form Association[Rule["a", 1], Rule["b", 2]], etc.

Community
  • 1
Szabolcs
  • 234,956
  • 30
  • 623
  • 1,263