Redefining \include

Question

A puzzling choice made by Leslie Lamport was to define \include in such a way as to issue a \clearpage before reading the file. This has been puzzling me for sometime as it is not a very good choice in terms of the author interface. This question has two parts:

Are there any valid reasons for this decision?
What can possibly break if the command is redefined?

The MWE below shows such a redefinition (I named the command \includex but works just fine even, if it is named \include). My own suspicion that this was introduced early on to resolve problems with twocolumns, but there is nothing in the source2e explaining this decision.

\documentclass{book}
\usepackage{filecontents}
\makeatletter
\def\includex#1{\relax
  \ifnum\@auxout=\@partaux
    \@latex@error{\string\include\space cannot be nested}\@eha
  \else \@includex#1 \fi}

\def\@includex#1 {%
  %\clearpage
  \if@filesw
    \immediate\write\@mainaux{\string\@input{#1.aux}}%
  \fi
  \@tempswatrue
  \if@partsw
    \@tempswafalse
    \edef\reserved@b{#1}%
    \@for\reserved@a:=\@partlist\do
      {\ifx\reserved@a\reserved@b\@tempswatrue\fi}%
  \fi
  \if@tempswa
    \let\@auxout\@partaux
    \if@filesw
      \immediate\openout\@partaux #1.aux
      \immediate\write\@partaux{\relax}%
    \fi
    \@input@{#1.tex}%
    %\clearpage
    \@writeckpt{#1}%
    \if@filesw
      \immediate\closeout\@partaux
    \fi
  \else
    \deadcycles\z@
    \@nameuse{cp@#1}%
  \fi
  \let\@auxout\@mainaux
}
\begin{filecontents}{A.tex}
This is file A
This is the A file
\end{filecontents}
\begin{filecontents}{B.tex}
This is the B file
\end{filecontents}
\begin{filecontents}{C.tex}
This is the C file
\end{filecontents}
\includeonly{A,C}
\begin{document}
\includex{A}
\includex{B}
\includex{C}
\end{document}

You can't ensure correct cross references involving the currently unprocessed files without issuing a \clearpage. — egreg, Apr 12 '13 at 07:09
@egreg Interested to see how it breaks. If you have a look at \clearpage there is nothing, however the inlabel test from newpage could be duplicated in the include definition? — yannisl, Apr 12 '13 at 07:19
If you add \hspace*{...} instead of \clearpage maybe you would be able to get right working variant of \include. The argument of \hspace must be height of text on the page before \include. — Eddy_Em, Apr 12 '13 at 07:23
@Eddy_Em that wouldn't work, it needs to be a page break to interact with \write as egreg said. — David Carlisle, Apr 12 '13 at 08:15
As @egreg says it has to be \clearpage so that \write and float positioning works correctly. As a historical note it wasn't a "team" it has always done this so this is Leslie's doing before we got involved:-) — David Carlisle, Apr 12 '13 at 08:17
@DavidCarlisle Thanks. I understand the \write part and thinking that one should maybe experiment with writing the label part in the fileN.aux. How can one trick the floats? — yannisl, Apr 12 '13 at 08:35
The only way in standard latex (or at all really) of stopping a float floating right past a non-included section is to use \clearpage — David Carlisle, Apr 12 '13 at 08:40

egreg · Answer 1 · 2013-04-12T07:38:38.563

Let's assume you have

\include{fileA}
\include{fileB}

If there is no \clearpage when fileA ends and TeX starts reading fileB, there might be a \write relative to fileA pending and it will get lost: the \write commands relative to \label are performed at shipout, not immediately. When the next shipout occurs, the fileA.aux file will have already been closed.

If you try your example adding a \label command to each one, none of them will get written anywhere (but in the .log file), because the shipout happens when the .aux files have already been closed.

score 14 · Accepted Answer · answered Apr 12 '13 at 13:18

The purpose of the \include mechanism is to allow for partially compiling the document when making a modest amount of changes without the need to recompile the full document and still get cross-references etc correctly resolved (even to parts outside the current part under the knife).

For this to work the part or parts that are included have to be selfcontained in the sense that changes to it do not automatically (and always) render the formatting of other parts not included invalid. To make this possible (at least for smaller changes) the following prerequisites are needed:

The mechanism need to ensure that small changes in text length in the part being compiles doesn't result in formatting changes in other parts not compiled
Floats in the parts compiled need to be placed in the parts compiled

If either of the above points is not valid then using \include would (nearly) always result in invalid documents whereas in the current scheme you can all include parts individually and still arrive and maintain a valid document. (Just for the record, when we produced the first edition of the LaTeX Companion it took ages to compile this book and even the second edition took about 30 minutes in 2004 to compile all examples + all pages and rerun the whole book (I think 5 times) to get all cross-references resolved successfully. So building out things chapter by chapter was essential and even then compiling took long :-)

Of course the first point will be invalid the moment a lot of text is added or removed such as the resulting document get one or more pages longer or shorter.

So using page breaks at the boundaries of the included parts is simply necessary to make the mechanism worth while in the first place and using \clearpage is needed to to ensure that the floats stay within the included part and not move in or out.

@egreg already gave the additional explanation that the mechanism to write aux file data works only on \shipout so that it wouldn't be possible (or at least not easy) to ensure that such things like cross-references or data for table of contents aren't lost. Technically one could think of a possibility to manage this, but using more than one aux file per include, but that wouldn't resolve the point above.

Finally, this is not something the LaTeX Project team invented, it goes back to Leslie Lamport's original design and was in existance since LaTeX 2.08 (at least) so before 1986.

Thoughts for LaTeX3:) Based on \pdffilesize or similar check, if a file from a list has been changed. Compile only sections from a full list that have changed. This would mean multi-auxiliary files for almost everything, including ToC, but then would make it easier to create minitocs per chapter for example. The floats problem can be handled by a revised OR. This has the potential to offer orders of magnitude gains in compilation speed. — yannisl, Apr 12 '13 at 13:56

Redefining \include

2 Answers2

Linked