Passing multiple arguments from ProcessList (xparse) to a macro. Result to be used in tabular

Question

I am making a macro that will take user input of a grading scheme and place it into a tabular environment. I would like the syntax to be as simple as possible and decided on:

\tabulate{ eval1,%1 ; eval2,%2 ; ... ; evaln,%n}

as the input scheme, e.g.

\tabulate{Midterm 1,15\% ; Midterm 2,20\%}

Having asked question about building a table from a macro previously, I know that I should (need to?) use toks to build the rows of the table. My first attempt was the following:

\documentclass{article}
\usepackage{xparse}

\makeatletter
\newtoks\@tabtoks
\newcommand\addtabtoks[1]{\@tabtoks\expandafter{\the\@tabtoks#1}}
\newcommand*\resettabtoks{\@tabtoks{}}
\newcommand*\printtabtoks{\the\@tabtoks}
\makeatother

\DeclareDocumentCommand \tabulate%
{ > { \SplitList { ; } } m }%
{\ProcessList {#1} {\mycommand}}
\def\mycommand#1,#2{\addtabtoks{#1&#2\\ }}

It took me a while to sort of figure out why this didn't work: the #1 is passed from ProcessList to mycommand as a single token and can't be read as individual tokens? Question: Is my understanding of why the above doesn't work reasonably correct? If not, could someone please clarify? Based on that fuzzy understanding, I came up with a solution that works but deviates from my desired input scheme (this is the almost working example):

\documentclass{article}
\usepackage{xparse}

\makeatletter
\newtoks\@tabtoks
\newcommand\addtabtoks[1]{\@tabtoks\expandafter{\the\@tabtoks#1}}
\newcommand*\resettabtoks{\@tabtoks{}}
\newcommand*\printtabtoks{\the\@tabtoks}
\makeatother

\DeclareDocumentCommand \tabulate%
{ > { \SplitList { ; } } m }%
{\ProcessList {#1} {\mycommand}}
\newcommand{\mycommand}[1]{
\def\temp{#1}
\expandafter\mycommandTwo\temp
}
\def\mycommandTwo#1,#2{\addtabtoks{#1&#2\\ }}

%{\ProcessList {#1} {\mycommand}}
%\def\mycommand#1,#2{\addtabtoks{#1&#2\\ }}


\begin{document}

\resettabtoks
\tabulate{Midterm 1,{15\%};Midterm 2,{20\%};Midterm 3,{25\%}}

\begin{tabular}{cc}
    \printtabtoks
\end{tabular}

\end{document}

Again, based on my fuzzy understanding, I stored the argument from ProcessList in a macro that gets expanded and can therefore be read as individual tokens by mycommandTwo. Depending on whether I understood the initial problem correctly, this may not be the correct interpretation. My problem is that without grouping the percentages as above, ie. {15\%} rather than just 15\% I don't get the output that I want. I'm not sure why that is the case. Could anyone help me understand why the percentages need to be grouped and/or suggest a possible fix/improvement on what I currently have. My previous attempt involved parsing the csv eval,%, prepending each entry with an &, and then deleting the first & in the resulting list but I couldn't make it work. Thanks in advance!

Edit 1: Thinking about it now, I would prefer to insulate the user from even needing to enter the \ prior to the %'s, although I suspect that would require more knowledge than I have.

Answer 1

When doing tabular things, the moment when TeX sees the & is crucial and it's best to collect in advance the data and present it to TeX once.

Here's a way that uses expl3 syntax:

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn
\NewDocumentCommand{\tabulate}{ O{cc} m }
 {
  \tl_clear:N \l_scott_body_tl
  \seq_set_split:Nnn \l_scott_body_seq { ; } { #2 }
  \seq_map_inline:Nn \l_scott_body_seq { \scott_do_line:n { ##1 } }
  \begin{tabular}{#1}
  \tl_use:N \l_scott_body_tl
  \end{tabular}
 }
\cs_new:Npn \scott_do_line:n #1
  {
   \tl_clear:N \l_scott_row_tl
   \clist_map_inline:nn { #1 } { \tl_put_right:Nn \l_scott_row_tl { & ##1 } }
   \tl_put_right:Nn \l_scott_row_tl { \\ }
   \tl_set:Nx \l_scott_row_tl { \tl_tail:N \l_scott_row_tl }
   \tl_put_right:NV \l_scott_body_tl \l_scott_row_tl
  }
\seq_new:N \l_scott_body_seq
\tl_new:N \l_scott_body_tl
\tl_new:N \l_scott_row_tl
\ExplSyntaxOff
\begin{document}

\tabulate{Midterm 1,{15\%};Midterm 2,{20\%};Midterm 3,{25\%}}

\tabulate[cr]{Midterm 1,{15\%};Midterm 2,{20\%};Midterm 3,{5\%}}

\end{document}

One might do with \ProcessList, at least for getting the rows:

\ExplSyntaxOn
\NewDocumentCommand{\tabulate}{ O{cc} >{ \SplitList { ; } } m }
 {
  \tl_clear:N \l_scott_body_tl
  \ProcessList { #2 } { \scott_do_line:n }
  \begin{tabular}{#1}
  \tl_use:N \l_scott_body_tl
  \end{tabular}
 }
\cs_new:Npn \scott_do_line:n #1
  {
   \tl_clear:N \l_scott_row_tl
   \clist_map_inline:nn { #1 } { \tl_put_right:Nn \l_scott_row_tl { & ##1 } }
   \tl_put_right:Nn \l_scott_row_tl { \\ }
   \tl_set:Nx \l_scott_row_tl { \tl_tail:N \l_scott_row_tl }
   \tl_put_right:NV \l_scott_body_tl \l_scott_row_tl
  }
\tl_new:N \l_scott_body_tl
\tl_new:N \l_scott_row_tl
\ExplSyntaxOff

This is essentially the same as before: \SplitList splits the argument into a sequence and then \ProcessList does a mapping to this sequence (which is what is done directly in the first version).

Each row is processed as a clist; each item is preceded by &, a row is stored in a token list and the first & is pruned off. Then each row is added to a "body" token list, which is then delivered to the tabular environment.

---

If you really want to allow % directly, then you can use a common trick:

\newcommand{\tabulate}{\begingroup\catcode`\%=12 \innertabulate}

and then define \innertabulate just like \tabulate before, but adding \endgroup at its end.

Answer 2

The \SplitList in this command

\DeclareDocumentCommand \tabulate { > { \SplitList { ; } } m }%
  {\ProcessList {#1} {\mycommand}}
\tabulate{Midterm 1,15;Midterm 2,20;Midterm 3,25}

will give {Midterm 1,15}{Midterm 2,20}{Midterm 3,25} for #1 so indeed the single items are single tokens from the perspective of a macro.

Your attempt has another problem. Consider this command:

\def\test#1,#2{1: #1; 2: #2; that's it! }
\test Midterm,15\%

will give you

1: Midterm; 2: 1; that's it! 5%

Only the first token after the comma is seen. That's why you have to add the braces around 15\%.

In order to see the whole thing you'll need another marker:

\def\test#1,#2\stop{1: #1; 2: #2; that's it! }
\test Midterm,15\%\stop

---

IMHO the easiest thing to do would be to use expl3's mapping functionality:

\documentclass{article}
\usepackage{xparse}

\ExplSyntaxOn
\tl_new:N \l_tab_interior_tl
% this maps through the token list created by \SplitList and will apply
% \add_to_tab:w to every item in a way that it doesn't have to contain the
% comman and the secoind argument:
\DeclareDocumentCommand \tabulate { > { \SplitList { ; } } m }
  { \tl_map_inline:nn { #1 } { \add_to_tab:w ##1,, \q_stop } }

% this adds every entry from above in the form #1 & #2\% \\ to the token list
% that builds the table; it works similar to the \def\test above:
\cs_new:Npn \add_to_tab:w #1,#2,#3 \q_stop
  {
    \tl_put_right:Nn \l_tab_interior_tl
      { #1 & \tl_if_blank:nF { #2 } { #2 \% } \\ }
  }

% this prints the actual table material:
\cs_new_nopar:Npn \writetable { \tl_use:N \l_tab_interior_tl }

% this clears the tabular material; this functionality could of course be added
% to the \writetable command if you're sure you're not going to use it twice:
\cs_new_nopar:Npn \cleartable { \tl_clear:N \l_tab_interior_tl }
\ExplSyntaxOff

\begin{document}

\tabulate{Midterm 1,15;Midterm 2,20;Midterm 3,25}

\begin{tabular}{cc}
 \writetable
\end{tabular}

\cleartable
\tabulate{Midterm 1;Midterm 2,20;Midterm 3}

\begin{tabular}{cc}
 \writetable
\end{tabular}

\end{document}