Speeding up latex/lua/tikz

Question

I have a rather large LaTeX document in that it generates a lot of LaTeX code using lua (small .tex file on disk but large amount of generated LaTeX code). It is very slow to compile and takes about 1 hour to do so.

Almost all of the TeX code is generated from a few lua functions. At the moment I bypass most of the code until it is needed but I'm curious if there is some way to speed up the compilation process when rendering the full blown document.

Is there any way to profile the code and possibly speed it up? I'm not sure where the bottleneck is but essentially the (pseudo)code looks like this:

latex:
   for i = 1, n {
      \myluafunction{\arg{i}}
   }

lua:
   function myluafunction(arg) { ... tex.print(result) }

where myluafunction parses arg using lpeg then generates a lot of TikZ code using tex.print. The lua code is quite involved. While I haven't tried, I believe one can't use compiled lua code in lualatex, and even if so, I'm not sure how much of a difference it would make?

I imagine that the tex.print, even of a large amount of code, would be relatively fast?

Obviously there is a lot of geometry being rendered using TikZ. myluafunction essentially prints a bunch of LaTeX TikZ commands. Possibly TikZ is the issue here?

Any ideas what generally is the bottleneck and how it could potentially be improved? (would a virtual drive work for the .aux files, etc...)

Answer 1

I imagine that the bottleneck is in running the TikZ code. However, to empirically check whether it is or not, a possible option would be to do make copies of the files that you're using and add:

function makefile(file)
   f = io.open(file,"w")
   f:write("\\documentclass{article}")
   -- you could put the rest of your preamble here, or just copy paste it later
   f:write("\\begin{document}")
end

and,

function closefile()
   f:write("\\end{document}"}
   f:close()
end

to the lua portion. Then, make wrappers:

\def\start#1{\directlua{makefile("\luatexluaescapestring{#1}")}}

\def\stop{\directlua{closefile()}}

in the tex preamble. You could then search/replace tex.print with f:write and run your file like:

\documentclass{article}
\usepackage{luacode}
\begin{luacode*} % or use a .lua file

function makefile(file)
   f = io.open(file,"w")
   f:write("\\documentclass{article}")
   f:write("\\usepackage{...}")
   f:write("\\begin{document}")
end

function closefile()
   f:write("\\end{document}"}
   f:close()
end

--your lua stuff here

\end{luacode*}

\def\start#1{\directlua{makefile("\luatexluaescapestring{#1}")}}
\def\stop{\directlua{closefile()}}

% your other preamble here

\begin{document}
\start{myfile.tex}

% your code

\stop{}
\end{document}

Some variation of the above should get you a compilable .tex file. Running it will take the lua stuff out of the equation. Thus, if it takes nearly an hour to compile you know that the bottleneck is on the TikZ side or at least get a feel for how much of the compilation time is actually taken up by running the code as opposed to producing it via lua. If it turns out that the lua is taking a significant portion of the compilation time, here are a couple of resources that might help you to speed it up:

http://www.lua.org/gems/sample.pdf

http://trac.caspring.org/wiki/LuaPerformance

and if you can confirm that TikZ takes too long, you should seriously consider the use of either TikZ's external library or the standalone package. Both can significantly speed-up the processing of documents with many tikz pictures (they are optimized for slightly different use-cases).