1

Consider

\documentclass[12pt]{article}
\usepackage{luacode}
%\usepackage{luacr}
%\usepackage{miscellaneous}
\begin{document}

\global\def\F{\directlua{f()}}
\begin{luacode}
i=0
function f()
    if i<30 then
        i=i+1
        tex.sprint(i .. [[, \F]])
    end
end
\end{luacode}
\F


\end{document}

Problem:

TeX capacity exceeded, sorry [text input levels=15].

As far as I can see tex.sprint is "equivalent" to a \input that contains the printed content, and you can \input 15 levels at most.

Question:

  • why does the tail recursion elimination does not work when the input is from a line instead of a token list?
  • how to fix the issue? Assuming I need tex.sprint() e.g. to print some more string content to the right of \F.

in the meantime I find a workaround using token.put_next(), put a \relax and get it later.

user202729
  • 7,143
  • for the background, I'm making some convenience wrapper to run TeX code from Lua using coroutine. – user202729 Apr 15 '22 at 18:18
  • 1
    You're probably looking for this: https://tex.stackexchange.com/a/470667/10995 – Henri Menke Apr 15 '22 at 19:10
  • Tail call elimination obviously does not work because there is no return. Lua doesn't know that you are tail calling as part of tex.sprint. – Henri Menke Apr 15 '22 at 19:22
  • 1
    This also sounds awfully like an XY problem. What are you actually trying to do? There is probably a much simpler solution. – Henri Menke Apr 15 '22 at 19:26
  • @HenriMenke I already explained above, didn't I? "I'm making some convenience wrapper to run TeX code from Lua using coroutine" inspired from https://tex.stackexchange.com/questions/20893/concurrently-interleaving-execution-of-lua-and-tex-in-luatex – user202729 Apr 16 '22 at 00:20
  • Hm, on a second read, the answer there writes a loop in TeX... will think about it. – user202729 Apr 16 '22 at 00:22
  • @HenriMenke What do you mean by "there's no return" here? if you mean Lua'sreturn from f() function, it already returns normally by reaching the end of the function – user202729 Apr 16 '22 at 03:40
  • To qualify for tail call elimination the function has to be of the form function f() ... return f() end which this one clearly isn't. – Henri Menke Apr 16 '22 at 07:48
  • With XY problem I mean that to print a list of numbers you don't need a tail recursive hybrid Lua/TeX function, so the example is simplified a bit too much from what you are actually trying to do. – Henri Menke Apr 16 '22 at 07:50
  • @HenriMenke (anyway the closeinput one does work) -- No I'm talking about TeX tail call recursion (i.e. remove the input level when it's consumed), not Lua tail call recursion – user202729 Apr 16 '22 at 08:03

2 Answers2

4

You can force tex to unwind the input stack before recursing by looking ahead with \expandafter

enter image description here

\documentclass[12pt]{article}

%\usepackage{luacr}
%\usepackage{miscellaneous}
\begin{document}


\def\F{\expanded{\noexpand\directlua{f()}\expandafter}}
\directlua{
i=0
function f()
    if i<30 then
        i=i+1
        tex.sprint(i .. [[, \string\F]])
    end
end
}
\F


\end{document}
David Carlisle
  • 757,742
2

0. texio.closeinput()

Suggested by a comment

Code:


%! TEX program = lualatex
\documentclass[12pt]{article}
\usepackage{luacode}
\begin{document}

\def\F{\directlua{f()}}
\begin{luacode}
i=0
function f()
    if i~=0 then
        texio.closeinput()
    end
    if i<20000 then
        i=i+1
        tex.sprint(i .. [[, \F]])
    end
end
\end{luacode}
\F


\end{document}

The manual explains...

This function that should be used with care. It acts as \endinput but at the Lua end. You can use it to (sort of) force a jump back to TeX.

which means that it drops the remaining content of the topmost "file".

Note/clarification of the manual,

  • it doesn't really jump back to TeX, the following Lua content is still executed
  • tex.print() commands preceding it is also dropped
  • unlike \endinput the following content on the line is dropped
  • but if there are some pending tokens put by token.put_next() they will be kept as long as those after it
  • only apply to real "file"/pseudofile instead of e.g. argument token list

1. Use token.put_next on the continuation token

%! TEX program = lualatex
\documentclass[12pt]{article}
\usepackage{luacode}
\begin{document}

\def\F{\directlua{f()}}
\begin{luacode}
i=0
function f()
    if i<20000 then
        i=i+1
        tex.sprint(i .. [[, ]])
        token.put_next(token.create("F"))
    end
end
\end{luacode}
\F


\end{document}

(experimentally, all token.put_next() comes after all tex.*print(), regardless of their order in the code)

I think that the reason this method works is that while expanding a macro, unlike when token.get_next() or token.scan_toks() etc. is executed, TeX unwinds the input stack (even in case the macro does not have any argument such as in this case)

2. Use futurelet

%! TEX program = lualatex
\documentclass[12pt]{article}
\usepackage{luacode}
%\usepackage{miscellaneous}
%\tracingmacros=1
\begin{document}

\def\F{\directlua{f()}}
\begin{luacode}
i=0
function f()
    if i<20000 then
        i=i+1
        tex.sprint(i .. [[, \immediateassignment\futurelet\a\F]])
    end
end
\end{luacode}
\expanded{\F}
\end{document}

\immediateassignment used to make it work in expansion-only context.

Note that if the following token is a notexpanded token, it will be changed.

3. (partially works only, do not use) Use token.put_next on another token and get_next it from inside Lua

%! TEX program = lualatex
\documentclass[12pt]{article}
\usepackage{luacode}
%\usepackage{miscellaneous}
%\tracingmacros=1
\begin{document}

\def\F{\directlua{f()}}
\begin{luacode}
i=0
function f()
    token.get_next()  -- the relax token, either the original one or result of put_next
    if i<3000 then
        i=i+1
        tex.sprint(i .. [[, \F]])
        token.put_next(token.create("relax"))
    end
end
\end{luacode}
\F\relax
\end{document}

This does not do proper tail-recursive (somehow token.get_next() does not eliminate the input level if it's exhausted), so if 3000 is increased to a larger value you'll see

TeX capacity exceeded, sorry [input stack size=5000].
user202729
  • 7,143