Why does this command defined with \NewDocumentCommand behave differently from \def inside tabular column specification?

Question

On the first glance

\NewDocumentCommand{\foo}{m}{#1}

and

\def\foo#1{#1}

should be identical and in most scenarios they actually are, but when the argument is not explicitly follows \foo, the command defined via \NewDocumentCommand gives Missing { error whereas with \def it doesn't

This question is kinda a follow up to: Delay expansion of macro in >{...} column specification , so I will give that example to show the problem. Look at the MWE:

\documentclass{article}
\usepackage{array}
\def\test#1{
\newcommand{\temp}[1][default]{##1}
#1\temp
}
\begin{document}
\begin{tabular}{>{\test}l}
[one]1\
1\
1\
1
\end{tabular}
\end{document}

This one works perfectly fine, but once I define \test like so

\NewDocumentCommand{\test}{m}{
\newcommand{\temp}[1][default]{##1}
#1\temp
}

Errors rise up.

I figured out myself that if you add \expandafter before \test in column specification, it works with \NewDocumentCommand definition

\begin{tabular}{>{\expandafter\test}l}
[one]1\\
1\\
1\\
1
\end{tabular}

But I want to add additional argument, so that

\documentclass{article}
\usepackage{array}
\NewDocumentCommand{\test}{mm}{
\newcommand{\temp}[1][default]{##1}
#2#1\temp
}
\begin{document}
\begin{tabular}{>{\test{foo}}l}
[one]1\
1\
1\
1
\end{tabular}
\end{document}

And here I cannot do the trick with \expandafter.

---

So my question is how it make \NewDocumentCommand work the same as \def without throwing bunch of \expandafter or any other bulk solutions.

Answer 1

As I mentioned before, this is officially "undefined behavior", use collcell package then process the data as usual. But this is an answer anyway.

You must understand how \halign primitive works and all the 3 kinds of brace hacks do (TeXbook appendix D, or Understanding Brace Hacks / Showcase of brace tricks: }, \egroup, \iffalse{\fi}, etc.) to understand this answer.

---

First, make a MWE:

\documentclass{article}
\begin{document}
\ExplSyntaxOn
\protected\def\test     {\group_align_safe_begin: \testb}
\protected\def\testb #1 {\group_align_safe_end: #1}
\halign{\test\ignorespaces# \cr
123 \cr}
\ExplSyntaxOff
\end{document}

What's going on here?

the \test macro, as defined, will do the following...

• expand \group_align_safe_begin:
• grab the \ignorespaces token
• expand \group_align_safe_end:

Problem arises because, while it does that...

• expand \group_align_safe_begin: → increase the master counter by 1
• grab the \ignorespaces token → runs into the end of the u part of the template. TeX marks that the target master counter should be 1 at the end of the alignment entry
• expand \group_align_safe_end: → return the master counter. Too late.

when the & is seen, it will rightfully complain that the current master counter value is 0, while it should be 1 to properly end the alignment entry.

Here the problem is the explicit adjustment of the master counter, which is obviously desired in most common cases.

(side note, in this particular MWE you can avoid touching the # part with

\halign{\test\ignorespaces\empty# \cr
123 \cr}

but this is not applicable if you want to peek into the entry itself)

(by the way, the \expandafter you put before \test actually have no effect as \ignorespaces is not expandable; nevertheless it makes TeX "see" the \ignorespaces token and that "touches" the # part, thus set the target master counter value to 0 instead of 1)

---

Nevertheless you can choose to "opt out" of it by "undo" the effect (highly not recommended):

\protected\def\test{\group_align_safe_end: \testa}  % remember to protect this, because the start of an alignment entry is initially expanded to look for e.g. \omit
\NewDocumentCommand\testa{m}{\group_align_safe_begin: #1}

It does work in the \halign example above.

---

Needless to say you can find an example where disabling alignment-protect is harmful.

For things like \peek_analysis_map_inline:n protecting is obviously useful (https://github.com/latex3/latex3/issues/1090),
for things like clist_map_ etc. it's kind of useful,
for argument grabbing, I'm not sure. (maybe grabbing & inside an optional argument counts.)

There's this one (although a little contrived... who needs to take the \cr as input anyway...?)

\documentclass{article}
\begin{document}
\ExplSyntaxOn
% the following 4 lines are equivalent to the simple \protected\def  below
%\protected\def\test{\group_align_safe_end: \testa}  % remember to protect this, because the start of an alignment entry is initially expanded to look for e.g. \omit
%\NewDocumentCommand\testa{+m}{\group_align_safe_begin:
%   #1
%}
\protected\def\test #1{#1}
\halign{# \cr
    \test \cr}
\ExplSyntaxOff
\end{document}