10

I've been trying to implement a macro that reads and transforms its argument into a different form...but I'm having quite a bit of trouble finding or creating even the most basic parsing tools.

Case in point: I can't find a way to make a safe version of either car (which returns the first element of a list) or cdr (which returns the rest of the list).

I learned from this answer that LaTeX includes implementations named \@car and \@cdr, defined as follows:

\def\@car#1#2\@nil{#1}
\def\@cdr#1#2\@nil{#2}

However, both of these can only be used alone, no nesting is possible.

For example, the following produces an error:

\@car \@cdr content\@nil\@nil

The correct output, if \car and \cdr worked as in Lisp, would be 'o': the first element of the rest of the argument passed to \cdr.

However, \@car runs first, and doesn't execute its input, taking \@cdr as #1 and content as #2, and returns \@cdr.

This leaves the remaining input as \@cdr\@nil, which results in the error "Paragraph ended before \@cdr was complete".

We also can't use \@cdr more than once to trim off more than the first token, if we try:

\@cdr \@cdr content\@nil\@nil

...the result is content\@nil, because the first \@cdr discards the second before it is run.

I feel like the answer would be something along the lines of

\def\car#1{\@car #1\@nil}
\def\cdr#1{\@cdr #1\@nil}

...if it were possible to tell TeX to evaluate #1 until it contains no macros before passing it to \@car or \@cdr.

To that end, I've also tried:

\def\car#1{\edef\temp{#1}\expandafter\@car\temp\@nil}
\def\cdr#1{\edef\temp{#1}\expandafter\@cdr\temp\@nil}

...which again works correctly for \car{content} or \cdr{content}, but causes compilation of the document to silently fail for \car{\cdr{\cdr{content}}}, and I have no idea why.

Any ideas on how car and cdr could be implemented safely?

Community
  • 1
Theodore Murdock
  • 203

3 Answers3

7

Werner has addressed the specific issue very well. More broadly, TeX is a macro expansion language, which work very differently from languages which use functions. There are tricks to produce function-like behaviour in some cases (expandably using \romannumeral, non-expandably using \edef) but this depends on the exact requirements. Here, depending on exactly what you want, something like

\documentclass{article}
\makeatletter
\def\car#1{\expandafter\@car\romannumeral-`\Q#1\@nil}
\def\cdr#1{\expandafter\@cdr\romannumeral-`\Q#1\@nil}
\makeatother
\begin{document}
\car{\car{content}}

\cdr{\cdr{content}}

\cdr{\cdr{\cdr{content}}}
\end{document}

might be suitable. This will fully expand the content, which is fine if that is simply text but may not be in other cases. (Note: \romannumeral expansion is ended by a space or other non-expandable token, so this trick still depends on the exact nature of the content. It is possible to set up a loop to try to deal with this: see for example Expandable full expansion of tokens that preserves catcodes and How to extend the \romannumeral-based \fullyexpand to handle empty/all-space arguments?. You don't say what is supposed to happen for non-expandable material, which is another issue with that approach.)

Community
  • 1
Joseph Wright
  • 259,911
  • 34
  • 706
  • 1,036
  • Thanks, I think this solution will work...it occurs to me that with \car we clearly don't need to evaluate anything else once we know that the first token isn't a macro, and for \cdr, as long as we're throwing away a non-macro token, lazy evaluation of the rest should give the same result. So if the expansion stops early it shouldn't be a problem, as long as it leaves a non-macro token at the start. – Theodore Murdock Jul 26 '12 at 07:20
  • After testing it a bit, it looks like this solution seems to throw away white-space tokens without counting them, rather than stopping, e.g. \cdr{ f o o } returns o o, throwing away the leading spaces without counting them as tokens. That could ruin it for some applications, but not for what I have in mind immediately, as the text I'm parsing never uses whitespace as a meaningful delimiter. – Theodore Murdock Jul 26 '12 at 07:31
  • @TheodoreMurdock As I say, TeX is a macro language so you have to decide what exactly it is you want. One thing to watch with your 'as long as the first thing is not a macro' is that there are many non-expandable 'string manipulation' approaches which surprise people when they try to \edef or similar and don't get what they 'see' when typesetting. So again you need to be careful. – Joseph Wright Jul 26 '12 at 07:36
5

The following produces the result you're after, namely o:

\def\@car#1#2\@nil{#1}%
\def\@cdr#1#2\@nil{#2}%
\expandafter\@car \@cdr content\@nil\@nil

The issue you have is with the way LaTeX gobbles its arguments, and therefore indirectly with the way things are expanded.

Using only

\@car \@cdr content\@nil\@nil

would mean that \@car is executed first, yes. It grabs, according to its definition, three tokens as arguments (the last being a "delimiter"), in this case \@cdr (the first or #1), content (the second or #2) and \@nil (the third). It returns to the input stream #1, which is \@cdr. So, what's left in the input stream is \@cdr\@nil which, according to \@cdr's definition, grabs three tokens. The first is \@nil. While the second could potentially be anything, there's no match for the third (or "delimiter"), before the paragraph ends, causing the error.

Using \tracingall, the following is output in the .log:

\@car #1#2\@nil ->#1
#1<-\@cdr 
#2<-content

\@cdr #1#2\@nil ->#2
#1<-\@nil 
#2<-...

In the execution

\expandafter\@car \@cdr content\@nil\@nil

the "inner function" is expanded (or executed) first, leaving \@car ontent\@nil in the input stream, which correctly extracts o (as #1, with ntent as #2).

Using \tracingall, the following is output in the .log:

\@cdr #1#2\@nil ->#2
#1<-c
#2<-ontent

\@car #1#2\@nil ->#1
#1<-o
#2<-ntent
Werner
  • 603,163
  • Thanks, that's a bit of a step in the right direction...but I still don't think it's sufficient to make car and cdr safe when used in macros that call other macros and built up lists of unevaluated tokens that eventually come to include nested calls to @car or @cdr. There's no way, for example, to use \expandafter in defining \car and \cdr to make both \car{\cdr{foo}} and \car{\cdr{\cdr{foo}}} work, as \car would need to use different numbers of \expandafters before calling @car...it's fine if we know exactly what tokens are coming before hand, but not safe in the general case. – Theodore Murdock Jul 26 '12 at 05:48
  • 2
    It's inaccurate to say that \@car gobbles three tokens as arguments: in the case of \@car contents\@nil we have #1<-c and #2<-ontents. But with \@car {abc}def\@nil we'd have #1<-{abc} (the braces would be stripped off at a later stage) and #2<-def. The fact is that \@car has an undelimited argument and a delimited one. – egreg Jul 26 '12 at 08:30
2

You might find the package lambda-lists interesting. There is also a TUGboat article regarding the package, both fascinating read.

Example of use:

\catcode`@=11
\input lambda.sty
\let\car\Head
\let\cdr\Tail
\def\mylist{\Listize[a,b,c]}
\car{\cdr{\mylist}} % => b
\bye
morbusg
  • 25,490
  • 4
  • 81
  • 162