Which TeX procedure removes trailing tabs?

Question

From reading the TeX source, it seems that the input_ln procedure is responsible for removing trailing spaces from lines upon input (before category codes are considered). A very short excerpt from the code there is

if buffer[last-1]<>" " then last_nonblank:=last;
end;

The test appears to target specifically spaces, and not tab characters. However, doing some tests, I see that TeX also removes tabs (regardless of catcodes). For instance, running

echo -e "\\\\catcode9=12\\\\show\\t%" | tex

correctly shows the character <tab>, but omitting the % there is no output.

So... Which part of TeX removes trailing tabs?

Answer 1

Here's how it goes in texlive (and presumably other web2c distros):

1. The changefile tex.ch removes the Knuth definition of input_ln;
2. The underscore gets lost in the conversion of web to c;

3. The resulting C code #includes texmfmp.h which does

   #define inputln(stream, flag) input_line (stream) 
   

4. The function input_line() gets defined in texmfmp.c where the trailing whitespace is cut thusly:

   /* Trim trailing whitespace.  */
   while (last > first && ISBLANK (buffer[last - 1]))
     --last;
   

5. ISBLANK() is defined in c-ctype.h as

   #define ISBLANK(c) (isascii (c) && isblank (c))
   

   where isblank() means (at least for usual locale settings) "is space or tab".

I'd say this behaviour could be a "bug", in that it deviates from the TeXbook: in chapter 8, page 46, Knuth writes:

TeX deletes any <space> characters (number 32) that occur at the right end of an input line. Then it inserts a <return> character (number 13) at the right end of the line, except that it places nothing additional at the end of a line that you inserted with 'I' during error recovery. Note that <return> is considered to be an actual character that is part of the line; you can obtain special effects by changing its catcode.

Elsewhere, we learn that plain tex sets the catcode for <tab> to 10 (space), and makes \<tab> the same as \<space> (and \<return>).

On the other hand, I can imagine the potential for confusion if tabs would not be skipped, here, so it makes some sense to do it.

Answer 2

I think the get_next procedure (starting in Section 332 "Getting the next token") takes care of discarding tabs.

Section 343 "Input from external file ..." loads cur_char from buffer and sets cur_cmd to the catcode for cur_char; it then advances loc, the index into buffer, by 1. In Section 344 "Change state if necessary ...," TeX does nothing if none of the cases for the case statement match. Therefore, when state is skip_blanks, and cur_cmd is 10 (space), the othercases label will match and TeX will jump to the switch label at the beginning of Section 343. loc will be incremented and the tab (or space) will effectively be discarded.

Section 347 "Handle situations involving spaces, ..." puts TeX into the skip_blanks state, and also takes TeX out of skip_blanks when non-space characters are encountered.