10

Consider the following MWE:

\documentclass{memoir}

%% Create 123 box registers
\newcount\X
\X=1
\loop
\expandafter\newbox\csname mybox\the\X\endcsname
\advance \X by 1
\ifnum \X<124
\repeat

%% Create a box register and globally store ans empty hbox
\newcommand{\storebox}{%
    \advance \X by 1
    \expandafter\newbox\csname tempbox\the\X\endcsname
    \expandafter\global\expandafter\setbox\csname tempbox\the\X\endcsname\hbox{}
}%

\begin{document}

%% Call \storebox 11 times in a group a total of 5 times
\begingroup
    \storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox
\endgroup

\begingroup
    \storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox
\endgroup

\begingroup
    \storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox
\endgroup

\begingroup
    \storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox
\endgroup

\begingroup
    \storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox
\endgroup

%% Some text so the document isn't empty if it compiles
foo

\end{document}

Compiling with a current TeX Live 2015 with either pdflatex or lualatex yields the following error message:

! Incompatible list can't be unboxed.
\sidecontents ... \z@ {\m@mwhich@margin {\m@msidebar@margin }\ifmemtortm \m@sideb@right \else \m@sideb@left \fi \vtop to0pt{\normalsize \normalfont \sidebarfont \vskip \topskip \vskip -\ht \strutbox \vskip \sidebartopsep \unvbox \sideins
                                                                                                                                                                                                                                              \vss }\hss }

The above example is a simplified version of a much larger (and more useful) document and is extremely sensitive to change. So, for example, calling \newbox a total of 122 or 124 times instead of 123 times in the preamble eliminates the error.

Looking at the log, the box registers behave strangely around 195 as shown below:

\tempbox132=\box192
\tempbox133=\box193
\tempbox134=\box194
\tempbox135=\box256
\tempbox125=\box196
\tempbox126=\box197
\tempbox127=\box198
\tempbox128=\box199
\tempbox129=\box200
\tempbox130=\box201
\tempbox131=\box202
\tempbox132=\box203
\tempbox133=\box204
\tempbox134=\box205
\tempbox135=\box206
! Incompatible list can't be unboxed.
\sidecontents ... \z@ {\m@mwhich@margin {\m@msidebar@margin }\ifmemtortm \m@sideb@right \else \m@sideb@left \fi \vtop to0pt{\normalsize \normalfont \sidebarfont \vskip \topskip \vskip -\ht \strutbox \vskip \sidebartopsep \unvbox \sideins
                                                                                                                                                                                                                                              \vss }\hss }

I first noticed this issue after the recent update to the LaTeX kernel (2015/10/01), and loading \usepackage[2015/01/01]{latexrelease} in the production document did allow the original document to compile. However, the MWE above shows that the kernel update is unrelated as adding the latexrelease package has no effect.

So what is the root cause of this error in this example and how can I properly prevent it from happening in similar circumstances? We can obviously get the error to go away easily by making almost any change to the document, but that is not really an option on the original production document.

brokemongoose
  • 1,499
  • Related? Or not? http://tex.stackexchange.com/questions/275042/etex-package-still-seems-to-be-necessary-in-latex2e-2015-10-01 – cfr Oct 27 '15 at 03:34
  • That question helped me narrow things down to create the example I used but I don't think enough is similar to get any more information from there on this issue – brokemongoose Oct 27 '15 at 04:09
  • Ah I think this is a latex bug:( will trace later today – David Carlisle Oct 27 '15 at 10:48
  • 1
    This is fixed in the sources for LaTeX2e <2015/10/01> patch level 2 (which uses the fix as in the answer below) (that format hasn't gone to ctan yet). – David Carlisle Oct 27 '15 at 15:00

2 Answers2

10

Sorry this is a latex bug, it's bad style to use a \new... in a local group, but the behaviour if you do it is not supposed to be that bad, This adds two \global which get things back on track. I guess we'll push out a patch release.

\documentclass{memoir}

\makeatletter
\gdef\e@ch@ck#1#2#3#4{%
  \ifnum#1<#2\else
    \ifnum#1=#2\relax
      \global#1\@cclvi
      \ifx\count#4\global\advance#1 10 \fi
    \fi
    \ifnum#1<#3\relax
    \else
      \errmessage{No room for a new \string#4}%
    \fi
  \fi}%
\makeatother

%% Create 123 box registers
\newcount\X
\X=1
\loop
\expandafter\newbox\csname mybox\the\X\endcsname
\advance \X by 1
\ifnum \X<124
\repeat

%% Create a box register and globally store ans empty hbox
\newcommand{\storebox}{%
    \advance \X by 1
    \expandafter\newbox\csname tempbox\the\X\endcsname
    \expandafter\global\expandafter\setbox\csname tempbox\the\X\endcsname\hbox{}
}%

\begin{document}

%% Call \storebox 11 times in a group a total of 5 times
\begingroup
    \storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox
\endgroup

\begingroup
    \storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox
\endgroup

\begingroup
    \storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox
\endgroup

\begingroup
    \storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox
\endgroup

\begingroup
    \storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox\storebox
\endgroup

%% Some text so the document isn't empty if it compiles
foo

\end{document}
David Carlisle
  • 757,742
  • I know this next question is vague, but is there a stylistically preferred way to use \new... commands and local groups? In the production document I have some environments that set options and box content. The boxes are printed later in the document based on some measurements handled on the lua end. That is why I have the likely awkward \newbox then \global\setbox combination inside a group, to localize options but print the content later. – brokemongoose Oct 27 '15 at 15:20
  • The normal recommendation is that macros used in the document should not be using \new.. at all plain TeX (and latex2.09) made them \outer to discourage that, took \outer off in latex2e, but generally should not use that feature. Sometimes it is convenient (which is why we got rid of \outer) but hard to say given the description wheter it is needed here) see for example latex float boxes, they are globally allocated once (in the format) and each figure environment uses, and eventually returns one of the boxes, but doesn't use \newbox. – David Carlisle Oct 27 '15 at 16:04
  • But in luatex you have 64 thousand box registers so if you use a \newbox idiom that wastes a few registers due to local group issues, it is hard to get too worried. Things are different in classic TeX which only has 256 box registers. (@grizzilus) – David Carlisle Oct 27 '15 at 16:07
  • All of this was originally in luatex which is why I have so many boxes. I was never trying to save them. I'll find a way to avoid creating the boxes in the document now that I know it is now recommended. – brokemongoose Oct 27 '15 at 16:30
  • @grizzilus as I say the recommendations have formed over decades of use based around 256 available boxes, it could be that the recommended good practice might have to change,. Anyway, sorry for missing out the \global in the 2015/01/01 update, which is the real issue, that's clearly a bug. – David Carlisle Oct 27 '15 at 16:55
4

It appears to be due to an update to Memoir.

I think that the correct way to fix this is probably to use \locbox in place of \newbox, though I'm not entirely sure. If I load elocalloc.sty and use \locbox as in the following preamble:

\documentclass{memoir}
\usepackage{elocalloc}
%% Create 123 box registers
\newcount\X
\X=1
\loop
\expandafter\locbox\csname mybox\the\X\endcsname
\advance \X by 1
\ifnum \X<124
\repeat

%% Create a box register and globally store ans empty hbox
\newcommand{\storebox}{%
    \advance \X by 1
    \expandafter\newbox\csname tempbox\the\X\endcsname
    \expandafter\global\expandafter\setbox\csname tempbox\the\X\endcsname\hbox{}
}%

then everything works fine. This is not much like what Joseph Wright's code, but that answer gave me the idea.

Hopefully, the code you put in the preamble here is actually in a package or class which can be changed without needing to change your original documents.

If not, the following may be useful if you cannot readily alter the preamble of existing documents.

Compilation of Old Code

There are two options.

Option 1

To compile the original document without changing the code at all, you can use an archived copy of Memoir and place the package files somewhere they will be found first by TeX.

I created symbolic links from my working directory to the package files and this is probably the easiest option on GNU/Linux or OS X.

For Windows, you are on your own, but I believe it doesn't understand symbolic links properly. In the worst case, you can just copy the package files to the working directory.

You should have these files/links:

mem10.clo     mem12.clo     mem17.clo     mem25.clo     mem36.clo     mem60.clo     memhfixc.sty  mempatch.sty  
mem11.clo     mem14.clo     mem20.clo     mem30.clo     mem48.clo     mem9.clo      memoir.cls

Strictly speaking, some might not be required, but I would not risk mixing files from different versions of the package unless absolutely necessary. (Right now, only the .cls files differ but an update could, naturally, change that, so I'd stick to all-archived or all-current for compilation of a given document.)

Option 2

If you can add a line to the document (either in the *.tex file or on the command line) then adding

\RequirePackage{etex}

before \documentclass allows the code to compile with current TL and current Memoir.

Explanation

This is due to a change to the kernel.

For reference:

--- ../../2014/texmf-dist/tex/latex/memoir/memoir.cls   2015-03-06 22:27:23.000000000 +0000
+++ tex/latex/memoir/memoir.cls 2015-07-08 23:33:01.000000000 +0100

The relevant change to memoir.cls is probably:

@@ -682,7 +682,9 @@
   \fi

 \ifmem@noetex\relax\else
-  \IfFileExists{etex.sty}{\RequirePackage{etex}}{}
+  \ifx\e@alloc\@undefined
+    \IfFileExists{etex.sty}{\RequirePackage{etex}}{}
+  \fi
 \fi
 \providecommand*{\memoirpostopthook}{}
   \memoirpostopthook

This is a response to changes made by the LaTeX project team which are intended to make etex.sty obsolete. In theory, nothing should now require this package. Packages which used etex needed to be modified to load elocalloc.sty, if required, instead. This loads etex.sty if \e@alloc is undefined. But with TL 2015, it is defined, so etex.sty is not loaded because it is not supposed to be necessary.

I know this because the problem caused an error for me for which I could not create a minimal example. Whatever I commented out fixed the problem, but uncommenting everything led to compilation failure. Nothing made any sense.

Your description sounded familiar - not only were 122 boxes OK, so were 124. I know the problem can manifest or not depending on when additional things are needed - that is, more boxes or whatever than TeX provides. If you need 124 early enough, that might be OK. And if you only need 122 ever, that might be OK. But if you need 123, other things might need more too late in compilation for it to be OK.

However, somebody else will need to provide the gory details as I do not understand them well enough. (Indeed, I'm not even confident about what I've said above, superficial though that explanation is.)

Community
  • 1
cfr
  • 198,882
  • I just checked the TeX Live svn repository. The difference between the current version of memoir (3.7e 2015/07/08) and the previous version of memoir (3.7d 2014/04/23) is the addition the following conditional that, when true, loads the etex package: \ifx\e@alloc\@undefined. I thought it was no longer necessary to load the etex package since the format handles the extended allocations now. So loading etex is a temporary fix, but is there a root cause in memoir or the format? – brokemongoose Oct 27 '15 at 02:18
  • 1
    @grizzilus I was just discovering the same thing. (See edit.) I'm not certain which/who is to blame. Loading etex.sty shouldn't be necessary. It may be that Memoir has not be updated correctly. Or it may be that something in your code ought to be different. I think David will have to say. – cfr Oct 27 '15 at 02:38
  • @grizzilus Please see edit (at the top of the answer because I suspect it is the 'right' solution, though I'm not confident - I'm sure the other solutions are not the right one, though, so it has the advantage of being possibly the right one). – cfr Oct 27 '15 at 03:48
  • Using elocalloc isn't viable for me. The class file I use doesn't look much like the preamble I have in the MWE. (It took me about four days to narrow down what was going on and get that example together.) Loading etex does work and is easier than loading an old memoir, but I am nervous that such a solution may be hiding the problem instead of solving it given the fragile nature of the issue. For example, loading any package that defined a box will allow the document to compile but without solving the root issue. – brokemongoose Oct 27 '15 at 04:02
  • @grizzilus I think loading etex is safe as a workaround until David pushes out a patch. It's to do with the timing of when additional boxes are needed, I think. Loading etex pre-emptively avoids the fragility because the timing no longer matters. It isn't the 'right' solution, but it shouldn't be fragile. And then the patch David is promising should fix it properly. – cfr Oct 27 '15 at 13:44
  • @cfr the main downside of loading etex is that it removes the features in the new allocation scheme notably the \extrafloats command to access more float boxes and a more resilient \newinsert. In a document as here that's probably not an issue (you know if you use those features) but in a package it's a bad idea as you can't know what other features a document uses. – David Carlisle Oct 27 '15 at 14:40
  • @DavidCarlisle I thought it would be safe here because it is about compiling existing documents which worked with earlier versions when etex.sty was loaded by Memoir. So my thought was that if the documents compiled OK with etex then, it should be safe to compile them with etex now. Those documents cannot rely on features from the new scheme because that scheme was not available when they were originally written and compiled. Have I misunderstood? – cfr Oct 27 '15 at 16:40
  • @cfr yes as I say for a specific document as here, it's fine, but there have been packages force-loading etex and that's less good as packages tend to get used on other places. – David Carlisle Oct 27 '15 at 16:52