tikzexternalize, list-and-make, list of figures to be updated

Question

I am using tikz' externalize library on windows. Since my document contains graphics with \ref, I need to use list and make instead of convert with system call. Since I am working on windows, I cannot use make main.makefile, as was also reported in this question.

Issuing the correct system calls is not a problem. However, I do not know how to find out which figures need to be updated since main.figlist seems to always list all figures. externalize obviously checks for whether a figure needs to be updated before issuing the system call, and I believe something like this is included in the makefile as well. Unfortunately I do not know how makefiles work. Is there a way to issue this check manually, or can I get a list of outdated figures?

The generated makefiles look as follows:

ALL_FIGURE_NAMES=$(shell cat thesis.figlist)
ALL_FIGURES=$(ALL_FIGURE_NAMES:%=%.pdf)

allimages: $(ALL_FIGURES)
    @echo All images exist now. Use make -B to re-generate them.

FORCEREMAKE:

include $(ALL_FIGURE_NAMES:%=%.dep)

%.dep:
    mkdir -p "$(dir $@)"
    touch "$@" # will be filled later.

fig-0.pdf: 
pdflatex -enable-write18 -halt-on-error -interaction=batchmode -jobname "fig-0" "\def\tikzexternalrealjob{main}\input{main}"

fig-0.pdf: fig-0.md5

Edit: tikz-externalize does collect the information on whether a figure is up-to-date or not. This is obvious because after a full run, it can issue a warning if there are figures that are not up-to-date. Furthermore, if externalize is set to convert with system call, only those figures are updated, that were outdated. So tikz has to have a way to invoke something like an md5 check. So instead of creating a list of outdated figures, is there maybe a way to invoke this up-to-date-check using a manual pdflatex command? Naivly put, something like pdflatex -jobname=figure0 --check-md5? I am perfectly happy to start updating figures manually, but creating all figures everytime I change one is - to put it mildy - time consuming... There has to be a solution that is working on windows as well, I suppose?

I had a look into the tikz-externalize files, but my tex is not good enough for this...

Edit 2: I realize I did not make my problem clear enough, or maybe the motivation for my question. My document includes several tikz pictures, many of those are very time consuming to create. I have tikzexternalize configured to consecutively number all figures in my document. Thus, I can manually invoke pdflatex -jobname=figure0 ..., and somehow pdflatex finds the right tikz code, although the tikzfiles are not named figure0.tikz and so on. I really can't do the manual md5 check unless I either find out where the externalize magic happens. This leaves me with the following choices:

• understand the magic (propably not going to happen anytime soon)
• find a way to invoke the tikze-md5check before typesetting
• find a way to get a list of outdated figures

Answer 1

This here may address the problem at hand, although I might have missed some point.

First, your basic assumption that you "need to use list and make instead of convert with system call" is outdated: Some decent version (*) of the external lib comes with support for \ref inside of pictures relying on the default convert with system call. The solution is based on a detection that the image contains an undefined reference. It then writes some special marker into the image's .dpth file. Later-on, this marker is read and the image will be reconverted during \end{document}, assuming that the reference is available then.

Thus, one solution to the problem "I need to handle \ref inside of my external images" might be to update to a decent version of the external lib and use convert with system call. Such a decent version is probably shipped with pgfplots, i.e.

\usepackage{pgfplots}
\usepgfplotslibrary{external}
\tikzexternalize

might do the trick.

(*) At the time of this writing, the "decent version" needs to be defined properly:

1. the external library as it is shipped and documented in the current stable PGF 3.0.1a does not contain the mentioned feature.
2. a copy of the external library is shipped with pgfplots. This one is always at least as recent as the version shipped with PGF. At the time of this writing, it is more recent and comes with the mentioned feature. If you write \usepgfplotslibrary{external}, the code checks if the version available as part of PGF happens to be more recent. If so, the version of PGF will be used. If not, the copy shipped with pgfplots will be used. Eventually, both will be in sync, but PGF's release cycle is way longer. That means that you are always better off with \usepgfplotslibrary{external} (at the cost of loading pgfplots).

---

That said, if you want to know which figures need to be remade due to a changed MD5, you can check if the file fig-0.pdf is older than the file fig-0.md5.

Note that the line

fig-0.pdf: fig-0.md5

in the Makefile does the same trick. And running LaTeX on the main file will update the .md5 file if and only if the MD5 has changed.

Answer 2

I've had several years of success using Windows 7 (and more recently using Windows 10) with tikz externalization under mode = list-and-make.

I cannot comment (not enough points), hence this answer instead, in hopes of resolving what may be some misconceptions such as

Since I am working on windows, I cannot use make main.makefile, as was also reported in this question,

where the linked question was closed based on the author's thought that

It seems that only I have the problem.

(I think I, too, experienced the same or similar problem).

The culprit appeared to me to be differences in the behavior of the cat command between linux and what was available in my case by default on Windows: one variant put a space between concatenations, the other produced a newline (I can't remember which did what). I installed RTools (with RTools ahead of related components on the Windows system path) so that the cat command on Windows changed to behaving as the linux version (I had issues I didn't understand with cygwin), and problems related to main.makefile and main.figlist (mentioned in the closed linked question) resolved, and everything worked. (I've been using externalization with mode = list-and-make on Windows for documents of classes beamer, article and tufte-handout, with the only hickup being that I cannot get the tufte class to work with lualatex as the externalization engine (have to stick with pdflatex), which is a different issue).