Lua loop in edef gives error, but not in newcommand! How to fully expand macros with lua loops?

Question

How can I have an \edef'ed (expanded) macro that has a lua loop with tex.sprint in it? I get an error when tex.sprint has another macro like \blindtext[<n>] in it. I do not get error if I print some non-macro strings like Hello world! using tex.sprint in the lua loop. The error also does not happen in case I do not expand (do not use \edef) the macro definition, and use \newcommand instead. Why is that so?

Below is the code that first uses \newcommand, and then \edef. First run the code as it is, in the second run uncomment the \edef version to see the error:

! Use of \\blindtext doesn't match its definition.
\kernel@ifnextchar ...rved@d =#1\def \reserved@a {
                                                  #2}\def \reserved@b {#3}\f...
l.26    \directlua{dofile("blindtextloop.lua")}
                                           %
?

% lualatex edefloop.tex
\documentclass[notitlepage,letterpaper]{article}
\usepackage[english]{babel}
\usepackage{blindtext}

\begin{document}

% Note: This will write file blindtextloop.lua in you current directory
\begin{filecontents*}{blindtextloop.lua}
    for i=0,3 do
        tex.sprint(" \\blindtext[1] \\par")
    end
\end{filecontents*}

% Unexpanded blindtext
\newcommand{\myblindtext}{%
    \directlua{dofile("blindtextloop.lua")}%
}%

Expanding next:

\myblindtext

%% Uncomment following lines in second run to see the error:
% %Expanded blindtext
% \edef\myblindtextexpanded{%
%   \directlua{dofile("blindtextloop.lua")}%
% }%

% Already expanded:

% \myblindtextexpanded


\end{document}

Screenshot:

\blindtext is not expandable (like everything that scans for optional arguments). — Henri Menke, Sep 14 '20 at 03:37
@HenriMenke Thanks. Everything at some point gets expanded, so what do you mean by "anything that scans for optional arguments is not expandable"? Why does TeX put this restriction? — codepoet, Sep 14 '20 at 03:45
Some primitives are not expandable and \futurelet, which is used to implement any type of optional argument scanning is one of them. — Henri Menke, Sep 14 '20 at 05:21
the question isn't really related to Lua, \edef\foo{\blindtext} generates the error you show, and doing that in a lua (or tex) loop doesn't stop that. — David Carlisle, Sep 14 '20 at 06:35
see https://tex.stackexchange.com/questions/66118/advantages-and-disadvantages-of-fully-expandable-macros/66168#66168 — David Carlisle, Sep 14 '20 at 06:35
@HenriMenke If the only thing making macro \blindtext unexpandable is the presence of optional arguments, can such problem be cleanly solved by taking care of optional arguments in lua? (while tex thinks there are no optional arguments?) Just curious... — codepoet, Sep 14 '20 at 06:44
@DavidCarlisle Thanks for the pointer. The question is related to lua in the sense that I intended to get a solution for the problem (non expandability) while executing in a lua loop. As you correctly point out, the root of the problem is not lua; I don't think I said that either. — codepoet, Sep 14 '20 at 06:51
@reportaman Just return it inside \unexpanded, i.e. tex.sprint("\\unexpanded{\\blindtext[1]}\\par"), but it's really an XY-problem. — Henri Menke, Sep 14 '20 at 07:07
you may be able to do something specifically for blindtext but that is presumably not your real use case.Most latex constructs will fail in an edef. Accented (or any non ascii) letters for example. Unless you are going to re-implement all of latex in lua, not using edef is the most likely solution. As Henri says you could use \unexpanded but using \unexpanded in \edef is more or less the same as just using \def. — David Carlisle, Sep 14 '20 at 07:13

score 1 · Answer 1 · answered Feb 26 '21 at 21:49

You can expand a Lua loop in edef, just not \blindtext in this case though you don't need to expand the macro, as shown in the terminal output this results in defining \myblindtextexpanded as

macro:-> \blindtext [1] \par \blindtext [1] \par \blindtext [1] \par \blindtext
 [1] \par

so the loop has been unrolled to repeated calls of \blindtext.

% lualatex edefloop.tex
\documentclass[notitlepage,letterpaper]{article}
\usepackage[english]{babel}
\usepackage{blindtext}
\begin{document}
% Note: This will write file blindtextloop.lua in you current directory
\begin{filecontents}{blindtextloop.lua}
    for i=0,3 do
        tex.sprint(" \noexpand\blindtext[1] \par")
    end
\end{filecontents}
% Unexpanded blindtext
\newcommand{\myblindtext}{%
    \directlua{dofile("blindtextloop.lua")}%
}%
Expanding next:
\myblindtext
%% Uncomment following lines in second run to see the error:
 %Expanded blindtext
 \edef\myblindtextexpanded{%
   \directlua{dofile("blindtextloop.lua")}%
 }%
% Already expanded:
\typeout{\meaning\myblindtextexpanded}
\myblindtextexpanded
\end{document}

Lua loop in edef gives error, but not in newcommand! How to fully expand macros with lua loops?

1 Answers1