Word count and 'TeX capacity exceeded'

Question

I use this piece of code in the preamble, obtained from Environment that counts words inside:

\usepackage{xesearch}
\newcounter{words}
\newenvironment{assignment}{%
  \setcounter{words}{0}
  \SearchList!{wordcount}{\stepcounter{words}}
    {a?,b?,c?,d?,e?,f?,g?,h?,i?,j?,k?,l?,m?,
    n?,o?,p?,q?,r?,s?,t?,u?,v?,w?,x?,y?,z?}
  \UndoBoundary{'}
  \SearchOrder{p;}}{%
  \StopSearching
  \marginpar{\arabic{words} words}}

I use this in my main file that includes several chapters. If I include up to the end chapter 4 (33599 words) then the counting environment works fine. If I include the next chapter (~6K words) I get the TeX capacity exceeded error. Having tested and excluded all other options it is definitely down to the assignment environment reaching a buffer(?) limit at some point over 33.599K words. I am not sure how to create a MWE for this particular case without sharing my actual work which for obvious reasons I cannot do.

This may be just too hard for TeX, I don't know. Is using LuaLaTeX an option? If yes, you might want to look at this: https://gist.github.com/phi-gamma/2622252 (didn't test, just googled it). — mbork, Feb 23 '13 at 09:45
You can try whether you get the error with lots of automatically generated contents like this 100k long lipsum text. Then you post and example with a text PUT 100k OF TEXT HERE instead of the long text. — yo', Feb 23 '13 at 10:20
\blindtext also takes an optional repetition argument which could be made large enough to trigger the problem. — Christian, Feb 23 '13 at 10:42
Hi DGarside. A tip: if you copy and paste directly the link of the answer in your question the title will appear. — Claudio Fiandrino, Feb 23 '13 at 11:13

David Carlisle · Accepted Answer · 2015-06-15T22:28:20.370

If I run the document below as posted I get a save stack error:

[30] [31] [32] [33] [34] [35] [36] [37]
! TeX capacity exceeded, sorry [save size=50000].

If I uncomment the two lines marked %%%% then it runs to the end

Output written on xew33.pdf (84 pages).

and reports 51720 words

enter image description here

\documentclass{article}

\usepackage{xesearch}
\newcounter{words}
\makeatletter

\newenvironment{assignment}{%
%%%%     \endgroup
  \setcounter{words}{0}%
\expandafter\ifx\csname wordcount@xs@searchlist\endcsname\relax
  \let\wordcount@xs@searchlist\undefined
  \SearchList!{wordcount}{\stepcounter{words}}
    {a?,b?,c?,d?,e?,f?,g?,h?,i?,j?,k?,l?,m?,
    n?,o?,p?,q?,r?,s?,t?,u?,v?,w?,x?,y?,z?}
  \UndoBoundary{'}%
  \SearchOrder{p;}%
  \else
   \expandafter\StartSearching
  \fi}{%
  \StopSearching
  \marginpar{\arabic{words} words}%
%%%%     \begingroup\expandafter\def\csname @currenvir\endcsname{assignment}%
}




\begin{document}

\def\a{one two three four five six seven eight nine ten. }
\def\b{red yellow green blue. }
\def\c{apple orange pear banana.}
\def\d{\a\a\b\b\c\a\b\c\c\b\b}
\def\e{\d\d\par\a\a\b\d\d\b\b\c\c\a\a\a\a\par}
\def\f{\e\e\a\a\b\b\e\c\d\d\a\c\b\a\e\e}


\begin{assignment}
\f\f\f\f\f\f\f\f\f\f
\f\f\f\f\f\f\f\f\f\f
\f\f\f\f\f\f\f\f\f\f
\end{assignment}

\begin{assignment}
one one two three
\end{assignment}


\begin{assignment}
one two three four five six
\end{assignment}


\end{document}

the document environment in LaTeX plays a similar trick to use the environment syntax but avoid making a group around the whole document.

This does calculate for size > 50000 but I do not seem able to have one assignment environment follow on immediately after another. The second environment registers as zero. Would the way around that be to create assignment_main and assignment_bibliography environments? — DGarside, Feb 28 '13 at 17:12
code updated to re-use the existing search structure seems to get the right answer on a couple of smaller lists at the end, see posted image — David Carlisle, Feb 28 '13 at 18:20
It seems that your code edit removed the %%%% markings that you mention in the body of your answer. Was that intentional? — LSpice, Jun 15 '15 at 22:13

Word count and 'TeX capacity exceeded'

1 Answers1

Linked