Integrating latexmk and TikZ external mode=list and make

Question

I've been looking into using latexmk to create my documents because it automatically handles how many passes the document needs to be fully processed etc. So, for example, a simple run of latexmk -pdf file.tex will run pdflatex and bibtex as many times as necessary to resolve all references.

I was wondering whether it's possible to not lose the ability to do so when using the TikZ/external library in the list and make mode. If list and make is used, the TikZ/external makefile is generated, but it is not automatically called by latexmk. You can run the makefile manually and, of course, that will generate the graphic files, but the problem is that, as far as latexmk is concerned, the .tex files remain unchanged, so running latexmk once more does not insert the newly-generated graphics into the document.

So, given file.tex that uses TikZ/external in the list and make mode, this is what happens:

• latexmk -pdf file.tex generates the pdf and inserts placeholders for the not-yet-generated graphics.
• make -f file.makefile processes and generates the pdf files for the graphics.
• latexmk -pdf file.tex does nothing because the source files have not changed.

I assume there's a way to force the generation of the pdf in the second latexmk pass, but that would mean that this would happen even if the source files remained unchanged.

I was wondering whether there was a way to use latexmk to handle all of this "automagicaly"? That would be great!

Here's a MWE to get everyone started:

\documentclass{article}

\usepackage{tikz,pgfplots}
\usetikzlibrary{external}
\tikzexternalize[mode=list and make]

\begin{filecontents}{plot.tikz}
\begin{tikzpicture}
  \begin{axis}
    \addplot coordinates {(1,1) (2,2) (3,3)};
  \end{axis}
\end{tikzpicture}%
\end{filecontents}

\begin{document}
  \input{plot.tikz}
\end{document}

Answer 1

This isn't latexmk automagic, but it avoids recompiling if the source .tex file hasn't changed:

latexmk -pdf file.tex
make -n -f file.makefile > /dev/null | grep pdflatex > /dev/null
if [ $? -eq 0 ] ; then
  make -f file.makefile
  latexmk -g -pdf file.tex
fi

If the auto-generated makefile isn't actually going to compile any figures, the string "pdflatex" won't appear in the output of the dry-run; otherwise, the figures are compiled and latexmk is forced to do at least one more compilation.

---

Alternatively, here is a latexmkrc file that works most of the time: it runs the external makefile once per latexmk invocation, after the first time pdflatex is called.

If you are compiling the document from scratch, latexmk will most likely run pdflatex multiple times, and so the figures will be included after the first run. If you are not building from scratch, then there is a good chance that latexmk will recognize that the external figures are being included, see that they have been updated by the external makefile, and then decide to run pdflatex again because of the changing dependencies.

Warning: this is the first time I've written perl:

our %externalflag = ();

$pdflatex = 'internal mypdflatex %O %S %B';

sub mypdflatex {
  our %externalflag;
  my $n = scalar(@_);
  my @args = @_[0 .. $n - 2];
  my $base = $_[$n - 1];

  system 'pdflatex', @args;
  if ($? != 0) {
    return $?
  }
  if ( !defined $externalflag->{$base} ) {
    $externalflag->{$base} = 1;
    system ("$make -j8 -f $base.makefile");
  }
  return $?;
}

Answer 2

This is not a straight forward answer to your question, but another way of achieving your overall goal: Outsourcing TikZ-pictures and rebuilding them only if they have changed.

This can be done using the standalone class:

\documentclass{article}
\usepackage[mode=buildnew]{standalone}
\begin{filecontents}{plot.tex}
\documentclass[tikz]{standalone}
\usepackage{pgfplots}
\begin{document}
\begin{tikzpicture}
  \begin{axis}
    \addplot coordinates {(1,1) (2,2) (3,3)};
  \end{axis}
\end{tikzpicture}%
\end{document}
\end{filecontents}
\begin{document}
  \includestandalone{plot}
\end{document}

This could fit your needs quite well, as you are already putting your TikZ-pictures in separate files and everything works "automagically" with latexmk. Caveat: The files have to have the extension .tex instead of .tikz and you have to add a few more lines of code.

Answer 3

Sorry for necroposting, but here's a hack I figured out a couple of days after first looking up this question here.

For context, I'm using latexmk -pvc (i.e. in "continuous preview" mode), with okular as pdf viewer and kate as text editor; shouldn't matter though.

---

latexmkrc:

$analyze_input_log_always=1;
$default_files=("file.tex");
$make="make -k -j 6"; # `-k`: keep trying on errors; -j 6: six parallel jobs
$use_make_for_missing_files=1; # probably only this is needed
$pdf_mode=4; # use lualatex
set_tex_cmds('--shell-escape --synctex=1 %O %S'); # generate source mapping

I don't know enough about latex etc to tell which parts are actually needed; maybe $use_make_for_missing_files=1 is sufficient.

Somewhere in the latexmk docs theres the hint to use \typeout{(filename)} to declare a dependency, however some digging through the code reveals that this is discarded if that file doesn't already exist. However, if we make it sound scarier, it'll stay:

Tikz creates a file.makefile with a dummy target allimages. However, if the error message specifies no extension, latexmk will try a couple of them, but not to actually build just the plain name. Thus, we use this fake error message:

\typeout{}
\typeout{No file "allimages.pdf".}
\typeout{}

And create a new makefile to redirect it to the target generated by tikz:
makefile

-include file.makefile
allimages.pdf: allimages

(Make sure that there's no allimages.tex or allimages.pdf)

---

caveats:

• latex first-timer here, so use at your own risk.
• I couldn't get this to work with a custom out_dir. \tikzsetexternalprefix{figures/} is fine though.
• latexmk will list the fake error message in its summary, even if everything went fine.
• I recommend setting a \tikzfigurename so that adding a new image in the beginning doesn't break everything,
• \tikzfiguredependsonfile needs to go after \tikzfigurename, but before \begin{tikzpicture}
• I encountered some situations where I broke some pre-existing images and latexmk decided to try makeing every individual image after the allimages target already failed. If it takes a long time before the error happens, this can be quite annoying.
• it can be quite hard to see whether all images compiled correctly, as any make output will be buried by the subsequent latex run. Try looking whether for every *.log there also is a *.pdf.

---

bonus: using the separate makefile, you can even define your own (implicit) recipes for data preprocessing. Trigger these via \tikzpicturedependsonfile.