1

here is my MWE:

documentclass[10pt, a4paper, twoside, ngerman]{scrartcl} 
\usepackage[utf8]{inputenc}
\usepackage[ngerman]{babel}
\usepackage{siunitx}    
\usepackage{geometry}
\usepackage{tikz}       
\usepackage{pgfplots}

\usepgfplotslibrary{colormaps, groupplots,external}
\usetikzlibrary{pgfplots.colormaps}
\tikzexternalize
\tikzsetexternalprefix{external_figs/}


\begin{document}


\begin{figure}
    \centering
    \input{fig.tex}
\end{figure}


\end{document}

The figure is a surface plot with a lot of datapoints which I cannot sample down further. I'm using TeXstudio as TeX-environment and Pdflatex with the following arguments

pdflatex -synctex=1 -interaction=nonstopmode -shell-escape %.tex

I still get the error

TeX capacity exceeded, sorry [main memory size=3000000].

I'm aware of the possibility to use LuaLaTeX (which I already used in a prior work) but the problem with it is (was), that it always took ages to build the file if the file contains more such memory intense images (apparently all images were recompiled everytime I built the document even though they didn't change).

Can anyone think of a good, stable and sustainable solution?

Steradiant
  • 705
  • How big is the file fig.tex? Can you show some representative lines from the file? – Mico Oct 24 '20 at 08:25
  • It's current research, hence, I can't really show what's inside the fig.tex file. But there are 6 surface plots with around 600 (3D)-datapoints each. – Steradiant Oct 24 '20 at 08:28
  • I would enter this as an answer, but you won't like it: "\includegraphics". Generate appropriate graphics-format files for your diagrams. Maybe you can get a successful run of individual tikz diagrams. – Donald Arseneau Oct 24 '20 at 23:20
  • Did that too already but it's more or less impossible to get the format (font style, look and feel) as with pgfplots and it always looks like 'stitched into it' and not like 'made as a whole'. – Steradiant Oct 25 '20 at 10:11

1 Answers1

2

Once I had a project with many (30+), very large, some more than 10k as exported from R, tikz graphs, maybe similar to yours, (lualatex MikTex 2.9, win10 x64, 8GB RAM) and I ran into a a similar problem.

I edited the file

texmfapp.ini

C:\Program Files\ MiKTeX\ miktex\config

Finally everything worked with these values

main_memory = 79999999

save_size = 79999


extra_mem_bot = 79999999


extra_mem_top = 79999999


font_mem_size = 799999


pool_size = 7999999


buf_size = 7999999


stack size = 200000


max-strings = 79999999

It was trial and error. I don't know exactly the impact of each parameter.

This answer it a reproduction of my previous

Again: ! TeX capacity exceeded, sorry [save size=500000] (LuaLaTeX, MikTeX)

Then I compiled the figures one by one (up to 15 min the largest!). At the end, monitoring the CPUs and memory while compiling the large figures, I found all 4 CPUs at mostly 100% and memory used at 80-90%. Then inserted them into the LaTeX document using this configuration

\tikzexternalize[%
up to date check={simple},
prefix=./GRAPHS/md5/]% Folder needs to be created before compiling

\tikzset{external/system call={%
        lualatex \tikzexternalcheckshellescape
        -halt-on-error -shell-escape -interaction=batchmode
        -jobname "\image" "\texsource"}}

in order to avoid a second compiling of the figures. There are several options of checks for changes to avoid the remaking.

Still some checks took up to 20-30, very long, seconds each time!
Now, with a little more wisdom afterwards, I guess I could have inserted the pdf generated files instead of the original tikz's, once each final version was reached and approved.

Take a look to http://www.bakoma-tex.com/doc/latex/pgfplots/pgfplots.pdf

section 6 for more detail instructions.

Simon Dispa
  • 39,141