8

Consider this plain TeX file:

\newtoks\t
\t={#}
\showthe\t
\bye

If you compile it (with tex of pdftex), then in the output you get

> ##.
l.3 \showthe\t

Why do I get two of the #s?

(The background: I want to build the body of a macro definition in a token list, and the macro takes one argument that I want to put into the token list as #1. The idea is to use \edef\macro#1{\the\t}.)

Hendrik Vogt
  • 37,935

3 Answers3

11

When TeX shows #, it doubles it up. So in your

\newtoks\t
\t={#}
\showthe\t
\bye

example, 

> ##.
l.3 \showthe\t

represents one # inside the token register. You can convince TeX to show this using \string:

\newtoks\t
\t={#}
\expandafter\string\the\t
\bye

(which of course only works because there is a single token in \t!)

For the definition of the macro, you'd need something like

\newtoks\t
\t={You gave '#1'}
\expandafter\def\expandafter\test
  \expandafter#\expandafter1\expandafter{\the\t}
\test{things}
\bye
Phelype Oleinik
  • 70,814
Joseph Wright
  • 259,911
  • 34
  • 706
  • 1,036
  • Unfortunately, \detokenize{#} gives two ##. Is that a bug in pdfTeX? – TH. Jan 04 '11 at 07:25
  • @TH. No, not a bug. \detokenize is meant to work very much like a toks, in the sense that you could do the same as \detokenize achieves by writing a toks to file and reading back with appropriate catcode changes. However, \detokenize is expandable whereas working via a file is not. If you try the file-based route, you'll also get two #. – Joseph Wright Jan 04 '11 at 08:03
  • Yeah. I was actually hoping to use \detokenize to write a single # to the file which is where I ran across it. I forget the exact reason now. Still, I'd assume that a #_6 token would be replaced by a single #_12 token. E.g., I'd expect \scantokens\expandafter{\detokenize{\def\foo#1{#1}}} to work. – TH. Jan 04 '11 at 08:21
  • Thanks for the explanation about \show. But why doesn't \edef\macro#1{\the\t} work? (I knew about the \expandafter workaround, but it's nasty, and I'm puzzled why this does work then.) – Hendrik Vogt Jan 04 '11 at 08:55
3

Joseph answered the reason why # shows up double, as the standard behaviour of show etc.

Since you want to build the body of a macro definition in a token list the only way to incorporate #1 etc. (IMHO) is to put the to put it via a macro.

\newtoks\t
\t={\def\z#1{\bf #1}}
\def\macro#1{\the\t \z{B} \z{#1}}
\macro{test}

\bye

The only difference from what was proposed by the OP, is that to put the body definition in a token list, you need to pass it to the macro by using \the\t and a \helpermacro{#1}.

yannisl
  • 117,160
3

Following what Yiannis wrote, you can do something like this:

% Put the replacement text into \t, using any computation like this doubling loop
\newtoks\t
\t{#1}
\count255 0
\loop\ifnum\count255<10
        \t\expandafter\expandafter\expandafter{\expandafter\the\expandafter\t\the\t}
        \advance\count255 1
\repeat
% Define a helper that when executed will perform the real definition
\edef\helper{\noexpand\def\noexpand\X##1{\the\t}}
\helper
% Now the real macro \X can be used.
\X{[X] }
\bye
TH.
  • 62,639
  • @TH.: That looks a bit clearer than the 5 \expandafter s in Joseph's suggestion, but I'm still puzzled why a simple \edef\macro#1{\the\t} doesn't work. (Fortunately, the initial reason for this post has gone since my \multexpafter got a lot simpler now.) – Hendrik Vogt Jan 04 '11 at 17:25
  • @Hendrik This is designed as such in TeX. Tokens resulting from \the\token variable are not expanded further if the statement occurs inside an \edef macro definition. – yannisl Jan 04 '11 at 19:07
  • @Yiannis: That's not the issue. The issue is that \t{#1}\edef\macro#1{\the\t} is the same as \def\macro#1{##1} which doesn't work: ! You can't use 'macro parameter character #' in horizontal mode. By stuffing it inside another macro, (\helper in my code) causes it to work. – TH. Jan 04 '11 at 20:34
  • @TH.: Can you explain why \t{#1}\edef\macro#1{\the\t} is the same as \def\macro#1{##1}? I can't find this described in the TeXbook. – Hendrik Vogt Jan 05 '11 at 17:24
  • @Hendrik: No. I cannot explain why that is the case. I can merely demonstrate that it is: \newtoks\t \t{#1} \edef\macro#1{\the\t} \show\macro \def\macro#1{##1} \show\macro \end. Best guess is it's a bug in TeX. I can't think of a reason that \ea\def\ea\macro\ea#\ea1\ea{\the\t} should behave differently from the \edef version. – TH. Jan 05 '11 at 20:44
  • @TH.: Thanks for telling me that I'm not alone. However, I don't think it's a bug (although it looks like one): There's one cryptic indication in the TeXbook that it is "status-bydesign", namely on p.216, 6th double-dangerous-bend-paragraph. Maybe you can make something out of what's written there? – Hendrik Vogt Jan 06 '11 at 09:33
  • @Hendrik: I don't think that applies here. It's saying that TeX automatically doubles the # there. E.g., \immediate\write-1{\def\noexpand\foo#1{#1}} writes \def \foo ##1{##1} to the log. Inside an \edef, you'd write \edef\bar{\def\noexpand\foo##1{##1}}. – TH. Jan 06 '11 at 16:40
  • @TH.: Thanks, it seems that I totally misread that paragraph. The mystery remains ... probably one has to have a look at tex.web. – Hendrik Vogt Jan 07 '11 at 14:26
  • @Hendrik: what, you don't have Volume B ;-)? – SamB Jan 28 '11 at 22:35
  • @SamB: Finally (after your recent question) I understand what you're talking about. I thought it was a joke and that you mean a Volume B of the TeXbook. No, I've never seen a copy of "TeX: The Program". – Hendrik Vogt Feb 04 '11 at 12:13
  • @TH. @Hendrik: \ea\def\ea\macro\ea#\ea1\ea{\the\t} expands \the\t before knowing that we are inside a definition. \edef#1{\the\t} expands \the\t while knowing that we are inside a definition, so it doubles hashes. – Bruno Le Floch Feb 07 '11 at 10:48
  • @Bruno: That makes sense, but is that documented anywhere? – TH. Feb 07 '11 at 11:51
  • @Bruno: Great, thanks. I'd also like to know where this is documented, but the mystery starts unraveling. (One of the mysteries of comment-@-replies unraveled: Only the first user is notified. So if you had written "@Hendrik @TH:", then both TH and I would have gotten a notification; TH as the poster of the answer, and I due to the "@Hendrik".) – Hendrik Vogt Feb 07 '11 at 12:56
  • @Hendrik, @TH.: I have no idea if this is documented. I just thought about the explanation this morning after reading that post a few days ago. --- Also, thanks for the @-tip. Would \catcode\@=\active\def@#1: {\notify{#1}}@Hendrik: @TH.: ` work? – Bruno Le Floch Feb 07 '11 at 14:13