latexmk reaches maximum runs when including new glossaries entries

Question

I have a problem with latexmk and the glossaries package. I have a large `scrbook' document organised with a main file and several chapter files. In my preamble I have a large quantity of acronyms defined (around 30 I believe). The problem is that when I add \gls{} entries to the document and then typeset with latexmk (I use TexShop on Mac OSX), it reaches maximum number of runs and returns an error. I wasn't able to recreate the error with a minimal document to post here unfortunately, but I was hoping that somebody could give me a hint on where to look at least, for example with known package conflicts or something like that. This is the last part of the log for the error:

Rule 'bibtex thesis': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'thesis.aux'
------------
Run number 2 of rule 'bibtex thesis'
------------
------------
Running 'bibtex  "thesis"'
------------
Latexmk: applying rule 'bibtex thesis'...
For rule 'bibtex thesis', running '&run_bibtex(  )' ...
This is BibTeX, Version 0.99d (TeX Live 2015)
The top-level auxiliary file: thesis.aux
The style file: ieeetr.bst
Database file #1: thesis_references.bib
Warning--entry type for "goalwebsite" isn't style-file defined
--line 324 of file thesis_references.bib
(There was 1 warning)
Rule 'cusdep acr acn thesis': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'thesis.acr'
------------
Run number 3 of rule 'cusdep acr acn thesis'
------------
Latexmk: applying rule 'cusdep acr acn thesis'...
For rule 'cusdep acr acn thesis', running '&do_cusdep( makeacr2acn )' ...
This is makeindex, version 2.15 [TeX Live 2015] (kpathsea + Thai support).
Scanning style file ./thesis.ist.............................done (29 attributes redefined, 0 ignored).
Scanning input file thesis.acr...done (0 entries accepted, 72 rejected).
Nothing written in thesis.acn.
Transcript written in thesis.alg.
Rule 'cusdep acn acr thesis': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'thesis.acn'
------------
Run number 4 of rule 'cusdep acn acr thesis'
------------
Latexmk: applying rule 'cusdep acn acr thesis'...
For rule 'cusdep acn acr thesis', running '&do_cusdep( makeacn2acr )' ...
This is makeindex, version 2.15 [TeX Live 2015] (kpathsea + Thai support).
Scanning style file ./thesis.ist.............................done (29 attributes redefined, 0 ignored).
Scanning input file thesis.acn...done (0 entries accepted, 0 rejected).
Nothing written in thesis.acr.
Transcript written in thesis.alg.
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'thesis.acr'
      'thesis.aux'
      'thesis.toc'
Latexmk: Maximum runs of pdflatex reached without getting stable files
Latexmk: Did not finish processing file 'thesis.tex':
   'pdflatex' needed too many passes
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs of latex/pdflatex.
Latexmk: applying rule 'pdflatex'...
Latexmk: Errors, so I did not complete making targets

I am sure that glossaries are the problem because the `thesis.acr' file keeps updating with every run, also if I comment out the commands that prints the glossaries it manages to compile just fine. The weird thing is if I trash the aux files it manages to compile just fine as well.

This is what my `latexmkrcedit' file looks like, I see that it does acknowledge problems with the glossaries but I have no idea what to do about it:

# Don't edit this file IF it is named latexmkrcDONTedit directly since
#   TeXShop may replace it on update.
# If you wish to edit or add to this file, copy it to ~/Library/TeXShop/bin
#   and rename it latexmkrcedit if that file doesn't already exist.

# Edit this File to add/change dependencies and rules
# The following is used by ALL the latexmk engine files

# change the empty string ,'', to '--shell-escape' to add the shell-escape default
$TSUserCompileOptions = '';
#$TSUserCompileOptions = '--shell-escape';

# turn recorder option off (no .fls file generated)
#$recorder=0;

# turn off duplicated missing references
$silence_logfile_warnings = 1;

# Custom dependency for glossary/glossaries package
# if you make custom glossaries you may have to add items to the @cus_dep_list and corresponding sub-routines
add_cus_dep('glo', 'gls', 0, 'makeglo2gls');
        sub makeglo2gls {
                system("makeindex -s '$_[0]'.ist -t '$_[0]'.glg -o '$_[0]'.gls '$_[0]'.glo");
        }
# The glossaries package, with the [acronym] option, produces a .acn file when processed with (xe/pdf)latex and
# then makeindex to process the .acn into .acr and finally runs of (xe/pdf)latex to read in the .acr file. Unfortunately
# the glossary package does just the reverse; i.e. (xe/pdf)latex processing produces a .acr files and makeindex then
# is used to convert the .acr file to a .acn file which is then ... . This dependency assumes the glossaries package.
add_cus_dep('acn', 'acr', 0, 'makeacn2acr');
        sub makeacn2acr {
                system("makeindex -s '$_[0]'.ist -t '$_[0]'.alg -o '$_[0]'.acr '$_[0]'.acn");
        }
# for glossary package (Sigh...) --- they can co-exist!     
add_cus_dep('acr', 'acn', 0, 'makeacr2acn');
        sub makeacr2acn {
                system("makeindex -s '$_[0]'.ist -t '$_[0]'.alg -o '$_[0]'.acn '$_[0]'.acr");
        }
# example of an added custom glossary type that is used in some of the glossary/glossaries example files:
# this is for the new glossary type command \newglossary[nlg]{notation}{not}{ntn}{Notation} from the glossaries package
# NOTE: the glossary package uses a very different command: the <in-ext> and <out-ext>
# are reversed in the calling sequence :-(
add_cus_dep('ntn', 'not', 0, 'makentn2not');
        sub makentn2not {
                system("makeindex -s '$_[0]'.ist -t '$_[0]'.nlg -o '$_[0]'.not '$_[0]'.ntn");
        }
# for the   glossary package (Sigh...) --- they can co-exist!
add_cus_dep('not', 'ntn', 0, 'makenot2ntn');
        sub makenot2ntn {
                system("makeindex -s '$_[0]'.ist -t '$_[0]'.nlg -o '$_[0]'.ntn '$_[0]'.not");
        }

# dependencies for custom indexes using the index package
# examples for sample.tex for index package:
 add_cus_dep('adx', 'and', 0, 'makeadx2and');
        sub makeadx2and {
                system("makeindex -o '$_[0]'.and '$_[0]'.adx");
        }
 add_cus_dep('ndx', 'nnd', 0, 'makendx2nnd');
        sub makendx2nnd {
                system("makeindex -o '$_[0]'.nnd '$_[0]'.ndx");
        }
 add_cus_dep('ldx', 'lnd', 0, 'makeldx2lnd');
     sub makeldx2lnd {
             system("makeindex -o '$_[0]'.lnd '$_[0]'.ldx");
     }

# Custom dependency and function for nomencl package
add_cus_dep('nlo', 'nls', 0, 'makenlo2nls');
        sub makenlo2nls {
                system("makeindex -s nomencl.ist -o '$_[0]'.nls '$_[0]'.nlo");
        }

I would really appreciate the help, it's frustrating when you can't concentrate on the content because of random problems. Thanks.

Answer 1

This is partly my fault for muddling the file extensions when I replaced glossary with glossaries. The input file from LaTeX's point of view is the output file from the indexing application's point of view, and the output file from LaTeX's point of view is the input file from the indexing application's point of view. I confused my ins and outs because I was looking at it from the wrong point of view.

If you're using glossaries the file dependency chain is "if there's a .glo file, run makeindex to create the .gls file". This means you need

add_cus_dep('glo', 'gls', 0, 'makeglo2gls');
sub makeglo2gls {
  system("makeindex -s '$_[0]'.ist -t '$_[0]'.glg -o '$_[0]'.gls '$_[0]'.glo");
}

in your latexmkrcedit file.

The reverse process makegls2glo works for the old glossary package. Problems ensue if you have both makeglo2gls and makegls2glo. If you're using glossaries, you need to remove the makegls2glo code from the latexmkrcedit file. This is the cause of your particular issue.

However, there's more of a problem here, not just for latexmk but for other build processes, where the assumption is that you have to run

makeindex -s basename -t basename.glg -o basename.gls basename.glo

where the LaTeX file is called basename.tex. This includes post-processor profiles and user-created batch files.

The glossaries package's flexibility allows it to be used for a wide variety of documents, but this flexibility inhibits a simple file-dependency build.

Let's consider a simple example:

\documentclass{article}

\usepackage{glossaries}

\makeglossaries

\newglossaryentry{sample}{name={sample},
 description={an example}}

\begin{document}
\gls{sample}.

\printglossaries

\end{document}

This fits in with the basic file-dependency model. If the file is called test.tex, then you need to run:

makeindex -s test.ist -t test.glg -o test.gls test.glo

Now let's suppose we have a minor modification:

\documentclass{article}

\usepackage{glossaries}

\setStyleFile{mygloss}

\makeglossaries

\newglossaryentry{sample}{name={sample},
 description={an example}}

\begin{document}
\gls{sample}.

\printglossaries

\end{document}

Now the call to makeindex is:

makeindex -s mygloss.ist -t test.glg -o test.gls test.glo

which doesn't fit the basic model.

Now let's suppose we have:

\documentclass{article}

\usepackage[nomain]{glossaries}

\newglossary[glg]{main}{glo}{gls}{Glossary}%

\makeglossaries

\newglossaryentry{sample}{name={sample},
 description={an example}}

\begin{document}
\gls{sample}.

\printglossaries

\end{document}

This flips the file extensions back to the way they were with the original glossary package, so now the call to makeindex is:

makeindex -s test.ist -t test.glg -o test.glo test.gls

Now let's suppose we have:

\documentclass{article}

\usepackage[nomain]{glossaries}

\newglossary*{unit}{Units}
\newglossary*{sym}{Symbols}

\makeglossaries

\newglossaryentry{mm}{type={unit},
 name={millimetre},
 description={length unit}}

\newglossaryentry{alpha}{type={sym},
 name={\ensuremath{\alpha}},
 sort={alpha},
 description={alpha}}

\begin{document}
\gls{mm}, \gls{alpha}.

\printglossaries

\end{document}

This requires:

makeindex -s test.ist -t test.unit-glg -o test.unit-gls test.unit-glo
makeindex -s test.ist -t test.sym-glg -o test.sym-gls test.sym-glo

The glossary-lipsum-examples.tex sample file that comes with the glossaries package has ten glossaries. Each one needs a separate makeindex call.

Perhaps I want to switch from makeindex's default word order to letter order for a particular document:

\documentclass{article}

\usepackage[order=letter]{glossaries}

\makeglossaries

\newglossaryentry{seal}{name={seal},
 description={aquatic mammal}}

\newglossaryentry{sealion}{name={sealion},
 description={a type of \gls{seal} with ears}}

\begin{document}
\gls{seal}, \gls{sealion}.

\printglossaries

\end{document}

This needs the -l switch:

makeindex -l -s test.ist -t test.glg -o test.gls test.glo

but only for that document.

Perhaps I want to use xindy instead of makeindex:

\documentclass{article}

\usepackage[british]{babel}
\usepackage[order=letter,xindy]{glossaries}

\makeglossaries

\newglossaryentry{seal}{name={seal},
 description={aquatic mammal}}

\newglossaryentry{sealion}{name={sealion},
 description={a type of \gls{seal} with ears}}

\begin{document}
\gls{seal}, \gls{sealion}.

\printglossaries

\end{document}

This now needs:

xindy -M ord/letorder -L english -I xindy -M test -t test.glg -o test.gls test.glo

So these hard-coded rules used by testing the existence of common file extensions in latexmk or simple hard-coded invocations of makeindex in post-processors can't cope with the wide variety of options that may be used with glossaries.

This is why glossaries provides the makeglossaries Perl script. This parses the .aux file to determine what to do.

For that last example, the .aux file includes the lines:

\@xdylanguage{main}{british}
\@gls@codepage{main}{}
\@newglossary{main}{glg}{gls}{glo}
\@istfilename{test.xdy}
\@glsorder{letter}

This provides makeglossaries with all the information it needs, so if I do

makeglossaries test

it knows to use xindy instead of makeindex (from the file extension in \@istfilename), it knows that letter order is required (\@glsorder), it knows the input/output/transcript file extensions (from \@newglossary). If xindy is required, it also knows what language is in use for each glossary (from \@xdylanguage) and the input encoding, if the inputenc package has been used (from \@gls@codepage, empty in this case as the document doesn't load inputenc).

There's a light-weight Lua alternative to makeglossaries which also parses the .aux file, but it's more restrictive than makeglossaries. For example, makeglossaries knows that babel's british language option needs to be passed to xindy with the -L english switch. The Lua version doesn't have these language mappings, so if you use makeglossaries-lite.lua, you need:

\usepackage[order=letter,xindy={language=english,codepage=utf8}]{glossaries}

Here's another example where makeglossaries works better than a direct call to makeindex:

\documentclass{article}

\usepackage{glossaries}

\makeglossaries

\newglossaryentry{seal}{name={seal},
 description={aquatic mammal}}

\newglossaryentry{sealion}{name={sealion},
 description={a type of \gls{seal} with ears}}

\begin{document}
\gls[format=textbf]{seal}, \gls{sealion} and \gls{seal} again.

\printglossaries

\end{document}

If I explicitly do:

makeindex -s test.ist -t test.glg -o test.gls test.glo

I get a warning from makeindex:

## Warning (input = test.glo, line = 1; output = test.gls, line = 5):
   -- Conflicting entries: multiple encaps for the same page under same key.

The location list for the seal entry has page 1 repeated, first in the normal font and then in bold:

seal aquatic mammal. 1, 1

This is something that makeglossaries checks for and tries to fix, so if I do

makeglossaries test

instead, the messages include the following:

Multiple encaps detected. Attempting to remedy.

The resulting test.pdf file now only has the bold page 1 in the location list for the seal entry. (The Lua version makeglossaries-lite.lua doesn't do this.) This doesn't occur with xindy as xindy will discard the textbf location without comment.

Some of the xindy messages can be a bit cryptic, but makeglossaries checks for them and tries to supply a more informative error message. Consider the following example:

\documentclass{article}

\usepackage[xindy]{glossaries}

\makeglossaries

\newglossaryentry{P}{
 name={\P},
 description={paragraph symbol}}

\begin{document}
\gls{P}

\printglossaries

\end{document}

This produces the following error message from xindy:

WARNING: Would replace complete index key by empty string,
         ignoring
         #<ordrule-regexp: '\\[a-zA-Z@]+ *' => '' :again NIL :only-at-start NIL>

What does this mean? If I use makeglossaries, I get a more informative message:

Sort key required for entries only containing command names
Attempting to determine which entries have problem sort keys
Parsing 'test.glo'
1 problematic entry found:

Label: 'P'. Sort value : '\\P '
(Try adding sort={P} to the definition.)

The problem here is that the P entry needs the sort key, so I need to correct my LaTeX code:

\newglossaryentry{P}{
 name={\P},
 sort=P,
 description={paragraph symbol}}

This is why I think that post-processor profiles and other build tools should really use makeglossaries rather than trying to call makeindex explicitly.

Personally I prefer to use arara even though I'm in the habit of using make as a general compile tool, as this allows me to customise the build process on a per-document basis, so I can just do:

% arara: pdflatex
% arara: makeglossaries
% arara: pdflatex
\documentclass{article}

\usepackage{glossaries}

\makeglossaries

\newglossaryentry{seal}{name={seal},
 description={aquatic mammal}}

\newglossaryentry{sealion}{name={sealion},
 description={a type of \gls{seal} with ears}}

\begin{document}
\gls[format=textbf]{seal}, \gls{sealion} and \gls{seal} again.

\printglossaries

\end{document}

or with version 4.0:

% arara: pdflatex
% arara: makeglossaries if found("aux", "@istfilename")
% arara: pdflatex
\documentclass{article}

\usepackage{glossaries}

\makeglossaries

\newglossaryentry{seal}{name={seal},
 description={aquatic mammal}}

\newglossaryentry{sealion}{name={sealion},
 description={a type of \gls{seal} with ears}}

\begin{document}
\gls[format=textbf]{seal}, \gls{sealion} and \gls{seal} again.

\printglossaries

\end{document}

which is more convenient if I can't decide on \makeglossaries or \makenoidxglossaries.