How do I compile an extremely large PDF?

Question

I am using LuaLaTeX to compile a file that will have estimated 4,000 pages. Quite obviously, though for Lua engine itself there are no complications in the file, my system runs out of memory on Page 944. How am I able to create such a file?

I hope for a method that would allow me to compile the file in separate seances and fuse the results. Just splitting will mess with the page numbering. I want to avoid actually creating multiple volumes in separate files.

The problem apparently is in the complicated font setup, with luatex-ja used to work with a CJK font parallel to the original one, and the font itself redefined with an intricate fallback similar to this:

\setmainjfont{SourceHanSans}[
Extension = .ttc,
UprightFeatures = {
    FontIndex=19,
        AltFont = {
        {   Range="2C9CA,
            Font="fallbackfont.otf"
        },
        {   Range="2C710,
            Font="fallbackfont.otf"
        },
        ... (20 more definitions)

Log files

When compiling, the process just stops:

...[941] [942]
[943] [944]
! TeX capacity exceeded, sorry [node memory size=1821640]
! Sorry, I ran out of memory.

Cutting down on fonts changes nothing, but removing all the fallback substitutions at all immediately lets the file to compile.

How do you manage to run out of memory? And which memory?4000 pages should normally not be a problem. — Ulrike Fischer, Aug 23 '19 at 13:37
You probably need to give us more information about your setup. Given that lualatex does not hold the entire document in memory, something else might be at play. — daleif, Aug 23 '19 at 13:37
Fonts, probably. Added a description of my font setup in the question. — Alexander Z., Aug 23 '19 at 13:42
Did you check, whether something on page 944 or previous make your system running out of memory? I had such problems with much smaller files but bad macros… — sztruks, Aug 23 '19 at 17:09

score 2 · Accepted Answer · answered Aug 25 '19 at 10:05

Self-answer: after taking away all the heavy fonts (fallback was manually edited to delete all but needed symbols, compressing from 10 Mb to 66 kb (!)) but nothing changing, even the page being the same, the culprit was determined to be lualatex.exe itsef, who ridiculously ate 2 Gb of RAM and got stuck.

Thus, the solution from Will TeXLive 2014 be 64 bit? worked. (Note: now the link is https://ctan.org/tex-archive/systems/win32/w32tex/TLW64/.)

How do I compile an extremely large PDF?

1 Answers1