12

I've been writing a document for some time now. It is steadily growing and hopefully will continue to do so. Lately I realised that -- although I just recently bought a new MacBook -- when I work on my largest tex-file typing is a little slow in TexShop. To be precise, when I write a sentence, then I am often finished on the keyboard with typing and still see the letters appear one after another on the screen for, say, two or three seconds until TexShop is finally done with the sentence. There is, then, a certain latency.

Now, I am aware that this is a separate issue that should be directed at the developers of TexShop, but it started me thinking whether there is

  1. a maximum size of a tex-file (characters or kilobytes) that one can technically \input or \include into the the main file which contains the preamble and the \begin{document} and \end{document} or, alternatively, whether there is
  2. a maximum file size that is recommended for a tex-file to \input or \include into the the main file.

The question has a bearing on my latency problem, because I could solve the latency problem by subdividing the larger files into smaller ones, but I would prefer not to do so -- actually for no good reason -- unless it was recommended along the lines of question #2.

While I would primarily be interested in answers that pertain to LuaLaTeX, I guess for other users here on the site answers that pertain to LaTeX and XeLaTeX will also be interesting.

ClintEastwood
  • 6,577
  • 2
    TeX does not load your entire file in memory, so there is no limit to the size of the file. There are limits however to the number of registers, macros, etc which the document (or packages) defines. This is unrelated to your question though. The latency in the editor is probably due to some kind of syntax-checking the editor is doing. Try to turn off syntax higligthing for example. Besides that, I find a good idea to split the document in several files and use input. It makes easier navigating the source, for example. – JLDiaz Jun 25 '14 at 08:57
  • 5
    The delay is editor specific, I believe. In any case, splitting a big source into smaller files is recommendable for several reasons, the main one being that losing a small file because of random corruption due to Murphy's law is surely preferable to losing a big file. – egreg Jun 25 '14 at 09:05
  • In some editors (e.g. Vim) syntax highlighting can be very resource intensive, especially with larger files. Have you tried switching it off? There is probably a configuration setting for the number of lines scanned (Vim: :help minlines); decreasing it should reduce the load too. – Philipp Gesang Jun 25 '14 at 12:52
  • I recommend http://tex.stackexchange.com/questions/22431/everyday-latex-and-workflow/22433#22433 for a discussion of ways to organize a large project. A good structure will help you in many ways besides reducing latency and the risk of file corruption. – Ethan Bolker Jun 25 '14 at 20:13

1 Answers1

17

TeX only processes the input line by line, so there is a limit on the number of characters in a line, but not on the total size of the file.

Your editor however probably does need to read the entire file into memory so may have limits on the file size. The editor I use warns if the file is more than 10MB but will open files larger than that, up to the memory available on the machine.

The line buffer is configurable in texmf.cnf, the one with texlive 2014 says

% Buffer size.  TeX uses the buffer to contain input lines, but macro
% expansion works by writing material into the buffer and reparsing the
% line.  As a consequence, certain constructs require the buffer to be
% very large, even though most documents can be handled with a small value.
buf_size = 200000

In luatex this buffer setting is ignored and the buffer is dynamically allocated as needed

luatex manual section 10.2:

The input line buffer and pool size are now also reallocated when needed, and the texmf.cnf settings buf_size and pool_size are silently ignored.

David Carlisle
  • 757,742