Variable processing, lualatex or expl3, or something else?

Question

I commonly find myself facing the same task of streamlining complex data entry for classes I write. I alway use the same front end. Data is entered and stored in variables, then a processing command uses the information in those variables to produce a complex formatted output.

For a simple example

\name{The name goes here}
\descr{The description goes here}
\tablecontents{column a}{column b}{column c}{column d}
\note{The note goes here}
\makeentry

Upon calling \makeentry whatever data was entered into the 'variables' would be used.

Over the years, I have made multiple backends, one in basic latex, one in expl3, and now most recently one in lua. Each has its own issues. I pretty quickly moved from basic latex to expl3 because handling any form of logic was a nightmare, and it took roughly 3 times the code to do the same thing.

I have issues with expl3 as there is so much erroneous stuff that goes into coding for it. And looking at old code looks like it came out of some sort of cypher. Dealing with all the strange data structures (looking at you token lists and sequences) is very tedious. And dealing with all the :Nn made me want to die.

Here is an example of my variable storage command in expl3

\tl_new:N \name %delcares the variable \name
\NewDocumentCommand{\name}{s+m} {%set command for names
    \tl_set:Nn \itemname {#2}
}

Lualatex seemed to offer a golden solution, a "real" programing language. The same code above can be handled with this.

function myluafile.setname(alpha)
    name = alpha
end
function myluafile.getname()
    tex.print(name)
end

And then the adjoining code in your .tex would be

\directlua{myluafile = require("myluafile")}%this links the .lua
\newcommand{\name}[1]{\directlua{myluafile("#1")}}

But diving further and I found myself wanting to build formatting in latex (because, you know, thats what its for) which actually would have required passing variables into lua, parsing them, passing them back to latex, storing them with expl3 variables, and then formatting them. I get around this by psudo processing them in lua and spitting gigantic print sequences back that latex then reads correctly, like so

local function buildnamedtable()
    tex.print("\\setlength{\\tabcolsep}{0pt}")
    tex.print("\\renewcommand{\\arraystretch}{1.2}")
    --Table settings for un-named entrys, D D D T
    tex.print("\\newcolumntype{N}{>{\\hsize=0.4\\linewidth \\raggedright\\arraybackslash}X}%")
    tex.print("\\newcolumntype{C}{>{\\hsize=0.15\\linewidth \\centering\\arraybackslash}X}%")
    tex.print("\\newcolumntype{T}{>{\\hsize=0.3\\linewidth \\raggedright\\arraybackslash}X}%")
    tex.print("\\begin{center}\\begin{tabularx}{0.98\\linewidth}{ N C C C T }")
    tex.print("& \\textbf{b} & \\textbf{c}& \\textbf{d}& \\textbf{e}\\\\")
    tex.print("\\hline")
    for k=0, (count - 1), 1 do
        if tocolorline[k] then
            tex.print("\\rowcolor{mycellcolor}")
        end
        tex.print(a[k] .. "&" .. b[k] .. "&" .. c[k] .. "&" .. d[k] .. "&" .. e[0] .. "\\\\")
    end
    tex.print("\\end{tabularx}\\end{center}")
end

Needless to say this feels very wrong, but it works. I'd say its better than dealing with expl3, but its not perfect, the joining commands between lua and latex seem to have problems. In particular im having trouble with the latex side of the storage commands \newcommand{\name}[1]{\directlua{myluafile("#1")}}. It seems if you try to pass complicated strings, such as those with backslashes \ quotes " single quotes ' or percent symbols % it up and dies. The error handling is also atrocious.

Is there a better way to do what I am trying to do? I am looking for input as to how others handle these kind of circumstances, as surely I cannot be the only one. I understand that the responses may be very opinionated, but that is exactly the kind of input I am looking for. I feel like my solutions up till now are all flawed in one way or another.

Edit: I have added clarity to the specific problem I mentioned about complicated strings. As well as added the corresponding getname function.

Using something like [=====[]=====] instead of "" allows you to avoid escaping characters every single time. Do you have your Lua code? — , May 11 '20 at 03:42
Im sorry but can you explain that again? I have no idea what you mean by [=====[]=====]. Yes I have the code. In this case it is as simple as function myluafile.getname() tex.print(name) end — Bob, May 11 '20 at 03:47
Yes, see, if you type tex.print([==[\textbf{Hello}]==]) you don't need to use the escape character \ — , May 11 '20 at 03:50
Btw, you're using tables whose definition is not included in your example, e.g. tocolorline, so your code remains a bit obscure — , May 11 '20 at 03:54
Im sorry there was a miscommunication. The last, large code block with the table was a separate example. My issue seems to be with this \newcommand{\name}[1]{\directlua{myluafile("#1")}} — Bob, May 11 '20 at 03:56
I tried using the tex.print([==[name]==]) and it seemed to have no change. If I pass \name{"value"} the nothing is printed, where if I pass \name{value} the value is printed. — Bob, May 11 '20 at 04:07
your way to set the data seems overcomplicated (in all variants). All this functions to read a simple variable. Why don't you use a key-val syntax or (in lua) read a table? — Ulrike Fischer, May 11 '20 at 07:22
@UlrikeFischer can you provide an example? I have not used keyvals before. As for tables, I do use tables, just in the name example there is no need, I just store it as a string. Or am I missing your point? can you perhaps elaborate more? — Bob, May 11 '20 at 09:05
I would set all the data in one go \setdata{name={Bob},desc={some description}, toc={toc1,toc2,toc3}, note={some note}} and not write for every one a command to set it. — Ulrike Fischer, May 11 '20 at 09:10
@UlrikeFischer is there a matching \getdata to access the values? The data entered into these variables can be be anything, paragraphs, tables, lists. Often a lot of parsing happens, such as text replacement and the like. I am reading up upon these keyvalues, seems like the two primary options are keyval and pgfkeys. Which of these would be most appropriate? — Bob, May 12 '20 at 02:29
IMHO you shouldn't mix expl3 and Lua. The latter is better for heavy programming tasks, whereas the former seems to be better for tasks which only allow TeX. — , Jan 20 '21 at 05:30
Somewhat related: luatex - Does lua scripting entirely replace the TeX macro-definition commands & allies? - TeX - LaTeX Stack Exchange — user202729, Jan 27 '22 at 13:59
Crosslink to related question: Can the Lua part of LuaTeX know about tokens? - TeX - LaTeX Stack Exchange — user202729, Apr 17 '22 at 00:42

score 4 · Accepted Answer · answered May 11 '20 at 05:30

Using the luacode package what do you want is possible. Basically \luastringN avoids any expansion, so it occurs on the TeX side. You can now pass complex arguments.

%!TEX program = lualatex
\documentclass[A5]{article}
\usepackage{luacode}
\begin{luacode*}
function setname(str)
    name = str
end
\end{luacode*}
\newcommand{\name}[1]{\luadirect{setname(\luastringN{#1})}}
\newcommand{\getname}{\directlua{tex.print(name)}}
\begin{document}
\name{\textbf{text}}\getname

\name{$x^2+1$}\getname

\name{"Jerky" comes from Quechua \emph{charqui}}\getname
\end{document}

score 0 · Answer 2 · answered Apr 01 '22 at 11:49

Now you have another choice: functional package. The package provides LaTeX2 interfaces for expl3. But the evaluation of composite functions is from inside to outside, which is the same as Lua language. Also you can make every function become a group, so that the values of local variables will be reset after the function.

\documentclass[A5]{article}
\usepackage{functional}
\PrgNewFunction \name {m} {%
  \TlSet \lNameTl {#1}%
}
\PrgNewFunction \getname {} {%
  \TlUse \lNameTl
}
\begin{document}
\name{\textbf{text}}\getname
\name{$x^2+1$}\getname
\name{"Jerky" comes from Quechua \emph{charqui}}\getname
\end{document}

Variable processing, lualatex or expl3, or something else?

2 Answers2