3

In answering this question, I noticed some trickiness regarding empty groups {} in an otherwise verbatim context.

Suppose you capture the contents of an environment verbatim (as in that question), except that \, {, and } still have their normal catcodes so that macros will work. Then the contents of the environment is written to a file. Any empty groups {} that are not consumed by macros will be written to file as literal {}. Is there a way to prevent this? Under such circumstances, what can be done to make macros behave as normally as possible?

Under these conditions, macros won't consume following spaces because the spaces are literal, so it would be logical to delimit the end of a macro with a {}. Except that that is resulting in a literal {}.

Minimal example

The example below uses both verbatim and fancyvrb. In the verbatim approach, each line is captured completely verbatim (stored in \verbatim@line), and then retokenized within \verbatim@processline. In practice, it would be more efficient to deal with each line only once, so using fancyvrb would be better. But both produce the same output, and the verbatim example is more hackable for demonstration purposes.

\documentclass{article}

\usepackage{verbatim}
\usepackage{fancyvrb}

\newwrite\outfile
\makeatletter
\newenvironment{verbatimoutexp}[1]%
  {\immediate\openout\outfile=#1%
    \def\verbatim@processline{%
      \begingroup
      % Redefine escapes so they write to file in literal fashion
      \edef\{{\@charlb}%
      \edef\}{\@charrb}%
      \edef\\{\@backslashchar}
      \let\textbackslash\@backslashchar
      % Retokenize with new catcodes
      \everyeof{\noexpand}%
      \endlinechar-1\relax
      \let\do\@makeother\dospecials%
      \catcode`\\=0%
      \catcode`\{=1%
      \catcode`\}=2%
      \xdef\verbatim@line@retok{\expandafter\scantokens\expandafter{\the\verbatim@line}}%
      \endgroup
      \immediate\write\outfile{\verbatim@line@retok}}%
    \@bsphack
    \let\do\@makeother\dospecials
    \catcode`\^^M\active
    \verbatim@start}%
  {\@esphack
    \immediate\closeout\outfile}
\makeatother

\begin{document}

\def\mymacro{MACRO}

\begin{verbatimoutexp}{test1.txt}
\mymacro{}FollowingText
\end{verbatimoutexp}

\begin{VerbatimOut}[commandchars=\\\{\}]{test2.txt}
\mymacro{}FollowingText
\end{VerbatimOut}

\end{document}

Output (same for both files):

MACRO{}FollowingText

Desired output:

MACROFollowingText
Community
  • 1
G. Poore
  • 12,417
  • \def\mymacro#1{MYMACRO} – egreg Jan 13 '15 at 00:15
  • @egreg Yes, that would be a solution for user-defined macros. But if you don't define a particular macro yourself, then you would have to wrap it. Also, in this form, you must have a following {}; otherwise, the following literal character is gobbled. – G. Poore Jan 13 '15 at 00:38

2 Answers2

3

I'd just take \* or any other suitable single-character macro command name and define it to be empty, then you can do:

\documentclass{article}

\usepackage{verbatim}
\usepackage{fancyvrb}

\newwrite\outfile
\makeatletter
\newenvironment{verbatimoutexp}[1]%
  {\immediate\openout\outfile=#1%
    \def\verbatim@processline{%
      \begingroup
      % Redefine escapes so they write to file in literal fashion
      \edef\{{\@charlb}%
      \edef\}{\@charrb}%
      \edef\\{\@backslashchar}
      \let\textbackslash\@backslashchar
      % Retokenize with new catcodes
      \everyeof{\noexpand}%
      \endlinechar-1\relax
      \let\do\@makeother\dospecials%
      \catcode`\\=0%
      \catcode`\{=1%
      \catcode`\}=2%
      \xdef\verbatim@line@retok{\expandafter\scantokens\expandafter{\the\verbatim@line}}%
      \endgroup
      \immediate\write\outfile{\verbatim@line@retok}}%
    \@bsphack
    \let\do\@makeother\dospecials
    \catcode`\^^M\active
    \verbatim@start}%
  {\@esphack
    \immediate\closeout\outfile}
\makeatother

\begin{document}

\def\mymacro{MACRO}
\def\*{}

\begin{verbatimoutexp}{test1.txt}
\mymacro\*FollowingText
\end{verbatimoutexp}

\begin{VerbatimOut}[commandchars=\\\{\}]{test2.txt}
\mymacro\*FollowingText
\end{VerbatimOut}

\end{document}
David Carlisle
  • 757,742
  • That makes sense. I had been hoping that there was some sneaky way to redefine how {} is written to file, but without that, something like this is probably the only option. – G. Poore Jan 12 '15 at 21:22
  • @G.Poore it depends on the context and what macros you want to use, you could make { and } active expanding to nothing so they would write as nothing, but then if the macro takes arguments or needs a grouping effect, {} are not available – David Carlisle Jan 12 '15 at 21:24
1

I was coming at this from the wrong angle. Instead of trying to alter the behavior of the empty group {}, particular when written to file, I should have been looking for ways to remove it altogether. I expect this is the only way to get what I want, but will leave the question open a bit longer in case there are other or better alternatives.

After retokenization in the verbatim example, we have literal text, groups delimited by braces, and possibly unexpandable macros. We need to strip out the braces with catcodes 1 and 2. It might also be nice to issue a warning about any unexpandable macros, since if macros are being used to insert text into otherwise verbatim code, we most likely want only macros that are fully expandable.

The braces may be stripped from a line by iteratively capturing each literal character/brace group/unexpandable macro in the line, and then reassembling the captured elements. Since the braces still have their normal catcodes, they are removed during the capture process. Dealing with nested brace groups is a little more complicated. Essentially, the brace-stripping procedure needs to be repeated until the output no longer differs from the input.

Once the braces are removed, only literal characters and unexpandable macros remain. In a situation in which these macros are undesirable, warnings may be generated by again iterating through piece-by-piece, and checking for macros with \ifcat.

\documentclass{article}

\usepackage{verbatim}
\usepackage{fancyvrb}

\newwrite\outfile
\makeatletter
\newenvironment{verbatimoutexp}[1]%
  {\immediate\openout\outfile=#1%
    \def\verbatim@processline{%
      \begingroup
      % Redefine escapes so they write to file in literal fashion
      \edef\{{\@charlb}%
      \edef\}{\@charrb}%
      \edef\\{\@backslashchar}
      \let\textbackslash\@backslashchar
      % Retokenize with new catcodes
      \everyeof{\noexpand}%
      \endlinechar-1\relax
      \let\do\@makeother\dospecials%
      \catcode`\\=0%
      \catcode`\{=1%
      \catcode`\}=2%
      \xdef\verbatim@line@retok{\expandafter\scantokens\expandafter{\the\verbatim@line}}%
      \endgroup
      \let\verbatim@line@retok@final\empty
      \let\verbatim@line@retok@last\verbatim@line@retok
      \expandafter\gobble@retok@braces\verbatim@line@retok\@nil
      \expandafter\check@retok@unexpandable\verbatim@line@retok@final\@nil
      \immediate\write\outfile{\verbatim@line@retok@final}}%
    \@bsphack
    \let\do\@makeother\dospecials
    \catcode`\^^M\active
    \verbatim@start}%
  {\@esphack
    \immediate\closeout\outfile}
\def\gobble@retok@braces#1#2\@nil{%
  \g@addto@macro{\verbatim@line@retok@final}{#1}%
  \if\relax\detokenize{#2}\relax
    \ifx\verbatim@line@retok@last\verbatim@line@retok@final
    \else
      \let\verbatim@line@retok@last\verbatim@line@retok@final
      \let\verbatim@line@retok@final\empty
      \expandafter\gobble@retok@braces\verbatim@line@retok@last\@nil
    \fi
  \else
    \gobble@retok@braces#2\@nil
  \fi}
\def\check@retok@unexpandable#1#2\@nil{%
  \ifcat#1\relax
    \PackageWarning{}{Verbatim text contained unexpandable macro \string#1}%
  \else
    \if\relax\detokenize{#2}\relax
    \else
      \check@retok@unexpandable#2\@nil
    \fi
  \fi}
\makeatother

\begin{document}

\def\mymacro{MACRO}

\begin{verbatimoutexp}{test.txt}
{}\mymacro{{{}{}{}{{{}}}}}{{FollowingText}} \{ \} \\ \textbackslash
\end{verbatimoutexp}

\VerbatimInput{test.txt}


\begin{verbatimoutexp}{test.txt}
\mymacro{}\relax FollowingText\ignorespaces asfd\ignorespaces{}sadf
\end{verbatimoutexp}

\VerbatimInput{test.txt}

\end{document}
G. Poore
  • 12,417