7

According to TeXbook TeX does preprocess files of .tex-input line per line (converting all characters of the line to TeX's internal character-representation-scheme (ASCII or Unicode), truncating sequences of space-characters at ends of lines, appending a character specified by \endlinechar).

So questions arise:

  1. What is the maximum amount of characters per line in .tex-input files?
  2. How does TeX behave when encountering at reading-time a line that is too long?

You can use TeX for writing external text files.

  1. How does TeX behave when attempting to write a line to external text file that is too long/that has more characters than TeX can place into a single line of an external text file?

The \write-primitive always writes an entire line to external text-file.

  1. Are there means for
    • just beginning a new line of an external text-file?
    • appending characters to the current line of an external text-file?
Mrs. Marsupial Wallaby
  • 73
  • I'm not sure there is a limit. I certainly have a line of over 1300 characters in one file in my thesis (a \caption, including cross-references, equations, \label and a lot of edits highlighted using the changes package; I tend to write one sentence per line) – Chris H Jun 15 '21 at 09:47
  • Given @David's answer below, I feel like I wasn't even trying – Chris H Jun 15 '21 at 09:49
  • 1
    @ChrisH my test file had 400000 z followed by \bye – David Carlisle Jun 15 '21 at 09:51
  • I am not sure I understand all your \write questions, you can write a newline by writing the \newlinechar (^^J in laTeX) try \typeout{aaa^^Jbbb} – David Carlisle Jun 15 '21 at 10:03
  • While 200000 is the nominal limit, with 199990 characters in a line you still get the error, and running pdflatex on a file starting with 199989 times the letter z, I get an ! Emergency stop error for some reason... – Phelype Oleinik Jun 15 '21 at 10:30
  • @PhelypeOleinik I think with less than 2000000 it reads the line but then you kill its hyphenation pass – David Carlisle Jun 15 '21 at 10:34
  • @DavidCarlisle But it dies with Emergency stop right after the everypar error (add a line with \tracingall before the 199989 zs), so I think it's something else internal (seems like an off-by-one error somewhere) – Phelype Oleinik Jun 15 '21 at 10:54
  • I tried to reproduce but didn't get that (I had also tried adding \tracingall don't all tex files have that?) I was using pdftex cygwin tl2021 with 199989z and \bye on the same line – David Carlisle Jun 15 '21 at 11:01

1 Answers1

7

This is system dependent and with web2c implementations user-settable.

texlive 2021 texmf.cnf has

% Buffer size.  TeX uses the buffer to contain input lines, but macro
% expansion works by writing material into the buffer and reparsing the
% line.  As a consequence, certain constructs require the buffer to be
% very large, even though most documents can be handled with a small value.
buf_size = 200000

If you exceed this you get:

$ pdftex aa408
This is pdfTeX, Version 3.141592653-2.6-1.40.22 (TeX Live 2021) (preloaded format=pdftex)
 restricted \write18 enabled.
entering extended mode
(./aa408.tex! Unable to read an entire line---bufsize=200000.
Please increase buf_size in texmf.cnf.

TeX has no mechanism to append to a file.

user202729
  • 7,143
David Carlisle
  • 757,742
  • Thanks for answering. As to "TeX has no mechanism to append to a file." The question is not about appending to a file but about appending to a line as could, e.g., be done with Pascal's write-procedure which - unlike writeln - does not finish the line. – Mrs. Marsupial Wallaby Jun 15 '21 at 10:25
  • What happens if you have accumulated tokens for writing in an amount where the resulting amount of characters to be written to external file has more characters than TeX's \write can write into a single line? – Mrs. Marsupial Wallaby Jun 15 '21 at 10:28
  • @Mrs.MarsupialWallaby for the log file you can use \message instead of \write for any other files then no it's not possible, but of course you can manage that via macros save up the tokens to be written in a token register then write them once you want to finish a line – David Carlisle Jun 15 '21 at 10:28
  • @Mrs.MarsupialWallaby the tokens to be written are accumulated in the input buffer before\write gets them so I think you will always get the error I showed even on writing. – David Carlisle Jun 15 '21 at 10:29
  • Thank you very much. :-) – Mrs. Marsupial Wallaby Jun 15 '21 at 10:31
  • 1
    Is "macro expansion works by writing material into the buffer and reparsing the line" actually true? Doesn't TeX just start reading from a token list? – texdr.aft Jun 15 '21 at 21:13
  • 1
    @texdr.aft normally yes but I seem to think that comment is right and there are constructs where tex borrows the input buffer, a couple of quick examples failed to show this howerever, and it's a bit late at night to start reading tex.web:-) – David Carlisle Jun 15 '21 at 23:00
  • 2
    @texdr.aft ah \input triggers this if you have \input file followed by z to say column 199980 then if file.tex is just x it all works but make it 20 x then you get the unable to read entire line error even though neither file has a line that long – David Carlisle Jun 15 '21 at 23:21
  • @DavidCarlisle Also, the name manufactured by \csname...\endcsname gets put in the buffer (section 374, comment on buf_size in section 11) – texdr.aft Jun 15 '21 at 23:44