4

I am trying to create a Makefile for my paper where I can type just make and it will compile everything in one go. The paper has several figures which are generated from scripts. However, everything I've tried requires several runs of make to get both the scripts to run and the paper itself.

I started with something based on this answer

all: paper.pdf

paper.pdf:

%.pdf: %.tex
    $(LATEXMK) -lualatex -halt-on-error -pdf -M -MP -MF $*.d $*

some-plot.pgf:
    script_to_generate_plot.py

-include *.d

However, from a clean build, or whenever I add a plot, this requires running make twice, once to generate the .d file and once to actually build the paper. And annoyingly the first time you just have to bypass an annoying "file not found" prompt.

I then noticed that latexmk has a flag to use make to build missing files, so I'm trying that:

all: paper.pdf

paper.pdf:

%.pdf: %.tex
    $(LATEXMK) -lualatex -interaction=nonstopmode -pdf -dvi- -ps- -use-make $*

some-plot.pgf:
    python script_to_generate_plot.py

Now what this does is finds one plot missing, build it, then fails with

Latexmk: 'pdflatex': source file 'some-plot.pgf' doesn't exist. I'll try making it...
------------
Running 'make "some-plot.pgf"'
------------
python script_to_generate_plot.py
Latexmk: Summary of warnings:
  Latex failed to resolve 1 reference(s)
Latexmk: Errors, so I did not complete making targets
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 256
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs of latex/pdflatex.
make: *** [paper.pdf] Error 12

If I run make again that plot was built, but it does the same thing with another one.

If I keep running make it eventually builds all my plots one by one (I have several plots, some of which take a while to generate so I'd like to keep the scripts to generate them separate).

How can I make it so that it builds everything in one go, regardless of which and how many files need to be regenerated?

EDIT I guess I forgot to mention that my other goal here is to avoid manually stating every .tex/.pgf/.pdf file dependency in my Makefile. If I do that, there's no point to using latexmk. I've done the manual Makefile before, but I'm always forgetting to update the dependency graph whenever I add a new file or rearrange things. My understanding is that the strength of latexmk is that it can figure that stuff out automatically. If I can get something that does that, that would be ideal. That and latexmk automatically rebuilds the paper and bib files as many times as necessary.

EDIT 2 I tried using the suggestion from the output, to use the -f flag to latexmk. This builds about 5 or so plots, then errors with

Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'origen-meeseeks.pgf'
Latexmk: Maximum runs of pdflatex reached without getting stable files
Failure to make 'paper.pdf'
Latexmk: Errors, in force_mode: so I tried finishing targets
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 256
Latexmk: Did not finish processing file 'paper':
   'pdflatex' needed too many passes
make: *** [paper.pdf] Error 12
transmutagen-papermaster*=$latexmk -

So it almost does what I want. How do I increase the max number of passes?

asmeurer
  • 454
  • I don't know about latexmk. However, for make, you need to tell it that the final PDF requires the plot's PDF. Then it will know not to apply the rule to make the final PDF until it has made the plot's PDF. (Or the plot's PGF or whatever it needs.) Then you just need a rule to make the plot's PDF. That's how make works: you tell it what ingredients each target needs and provide a recipe for turning the ingredients into the target. One target can have another target as an ingredient and make will figure out what to do in what order. – cfr Nov 16 '17 at 02:39
  • 1
    So %.pdf: %.tex shouldn't tell make that only one ingredient is needed for all PDFs because that's a lie! One of the PDFs needs another ingredient, so you need to list that ingredient, too. – cfr Nov 16 '17 at 02:41
  • I'm also by no means an expert on latexmk, but isn't it odd to use latexmk inside a make file, especially when latexmk is (additionally) told to (try to) 'use make? – jon Nov 16 '17 at 03:10
  • It is weird. My idea was that make would call latexmk (without me having to type all the arguments each time like -lualatex), then latexmk would call make to build the dependencies. So it shouldn't actually require the dependencies in the Makefile, because it's a separate make process that builds the plots. – asmeurer Nov 16 '17 at 21:15
  • In the way of using latexmk given in the first Makefile, the options given to latexmk tell it to generate dependency information in a form that is read by make when it executes the -include line. That's been removed from the second version, which shows an incorrect way of calling latexmk from a Makefile. The first Makefile is a good way of combining the use of latexmk and make. – John Collins Nov 18 '17 at 23:02
  • If you've a lot of files that need making, it's normally a bad idea to give the -halt-on-error option to latexmk, since then lualatex halts after the first missing file. Then you need repeated runs in order for latexmk to know about all the missing files: One run per missing file. It's better to use -interaction=nonstopmode. Then you get information on all the missing files in one run. – John Collins Nov 18 '17 at 23:07
  • Can you provide an example zip file to work from? I agree with @cfr that this should simply be a matter of telling make that the document PDF depends on the construction of the figure files. – Will Robertson Dec 19 '17 at 03:14

1 Answers1

1

As I noted in my edit, if you add the -f flag to latexmk, it continues after each plot is generated and tries again.

Unfortunately, these "re-runs" are counted as part of the number of times latexmk calls latex to build the document. By default, it only does 5 of these, but you can modify this with a configuration parameter. Create a file .latexmkrc in the directory with your tex files and add

$max_repeat = 10

(replace 10 with some number bigger than the number of plots you have, probably at least 5 more than that).

I personally would consider these to be bugs in latexmk (building more than a single thing with make shouldn't require -f, each call to make shouldn't count against the max_repeat).

You also do need to add the file dependencies manually in the Makefile

paper.pdf: some-plot.pgf

I don't know if there's a way to make the -MF stuff from latexmk work properly with just one Make run that both generates and uses them.

asmeurer
  • 454
  • I agree that if the only error from lualatex is a missing file error, and the file gets created, then it appears to be a good idea not to count the run towards the maximum. I'll consider this for another release. As I wrote in another comment, I normally find it better to use -interaction=nonstopmode to get all the missing-file errors in one batch. Only for later invocations would -halt-on-error be natural. – John Collins Nov 18 '17 at 23:16
  • 1
    However, for missing files, you shouldn't need the -f option. Latexmk is intended to continue processing after the missing file is made, which is different from it's behavior with other kinds of error. So there's probably some other difficulty either in latexmk's interaction with make or elsewhere. It's probably going to be too hard to work this out here. You could contact me offline (at the address in the latexmk documentation). – John Collins Nov 18 '17 at 23:22
  • I have a project, where non of -interaction=nonstopmode, -f and $max_repeat = 10; helped. Not even all of these options combined... Any other option that could fix this? – orzechow Dec 11 '19 at 16:24