Macro parameter delimited by more than one delimiter

Question

TeX user can simply create the macro with a parameter delimited by an arbitrary sequence of tokens. For example

\def\macro #1delimiter{...#1...}

Is it possible to create a macro with a parameter which has more than one delimiter? I mean something like this:

\mydef\macro #1[first or second]{...#1...}
\macro text first  % -> ...text ... (delimiter is first)
\macro text second % -> ...text ... (delimiter is second)

The parameter text should be scanned to the first delimiter from a list of delimiters. No more scanning. I know that you can set a fixed delimiter after this (\par for example) and then to treat with this longer parameter and read words "first" or "second" from it. But this approach reads more input data than we need. Unwanted tokenization is done, maybe here is unbalanced text etc.

Answer 1

You can use my macro \seplist exactly for purposes described in your question. The macro \seplist has the following syntax:

\def\yourmacro#1{... macro with normal #1 without separator}
\seplist{list of separators}\yourmacro parameter text

For example:

\def\macro#1{parameter is: "#1"}
\def\separed{\seplist{{sepA}{SepB}{SEPC}}}

\separed\macro text separated by sepA % sepA is separator
\separed\macro text separated by SepB % SepB is separator
\separed\macro text separated by SEPC % SEPC is separator

The actually used separator is stored globaly to the \sepused macro. The macro-programmer can use this.

The input stream is read to the first instance of any of the listed separator, no more. The separator list includes separators in braces. If there are only one-token separators, braces can be omited. Example:

\seplist{0123456789}\macro text to the first decimal digit 7

The separators can consist from any tokens except { } and # (more exactly the categories of these characters play the role). This is the same as when \def primitive with single separator is used. The parameter text can include these characters, but it have to be always balanced. Thus, the separator hidden in braces is ignored. This behavior is similar like in normal separated parameters. Example:

\seplist{0123456789}\macro this text {1234} is separated by five: 5
\seplist{\undefined \defined}\macro this text ends by \undefined or \defined
\seplist{{\undefined\defined}\par}\macro this text ends by \undefined\defined or by \par

If there are more separators, they all match the same text then the longer separator wins. Example:

\def\m#1{\message{param: "#1", separator: "\sepused"}}
\seplist{{BC}{ABC}}\m ahaABC % -> param: "aha", separator: "ABC"

If your \macro is defined as \long then \par can be scanned into parameter. Warning: if your \macro isn't defined as \long then parameter scanning doesn't stop at \par! You have to add the \par to the separator list. Example:

\def\parinmacro{\par}
\def\thismacro{\ifx\sepused\parinmacro \message{something wrong}\fi ...}
\seplist{{ab}{cd}\par}\thismacro This text skips the hidden {ab} and {cd}
                                 and it stops at the end of the paragraph.

You can define the \sepdef macro with the syntax similar to your requirement:

\def\sepdef #1#2[#3]{\def#1{\seplist{#3}{\csname.\string#1\endcsname}}%
  \long\expandafter\def\csname.\string#1\endcsname ##1}

\sepdef\test #1[{first}{second}]{Parameter is: "#1", separator is "\sepused".}

\test text separated by first
\test text separated by second

The implementation of \seplist macro follows. It uses only primitives and basic macros. It should be used in any TeX format.

\long\def\addto#1#2{\expandafter\def\expandafter#1\expandafter{#1#2}}
\newtoks\seplistT

\long\def\seplistD#1{%
   \seplistS##2\seplistE{\def\tmpa{##1}\def\tmpb{##2}\seplistE}%
   \def\tmpb{\tmpa #1}\expandafter\tmpb \tmp\seplistD\seplistE
}
\long\def\seplistE#1{%
   \ifx\tmpa\empty
      \seplistS\seplistD{\def\tmpb{##1}}\expandafter\tmpa\tmpb
      \ifx\tmpb\empty \seplistQ{#1}%
      \else \expandafter\addto\expandafter\seplistLx
                           \expandafter {\expandafter\seplistD\expandafter{\tmpb}{#1}}%
   \fi\fi
}
\def\seplistS{\long\expandafter\def\expandafter\tmpa\expandafter##\expandafter1\tmp}
\long\def\seplistQ#1#2\seplistA{\fi\fi\gdef\sepused{#1}\seplistZ}

\long\def\seplist#1#2{\begingroup
  \toks0={#2}\let\bgroup=\relax \let\egroup=\relax
  \def\seplistL{}\def\seplistLx{}\seplistI#1{}\gdef\sepused{}%
  \ifx\seplistL\empty \expandafter\endgroup \the\toks0\else
  \seplistT={}\expandafter\seplistA\fi
}
\def\seplistA{\futurelet\tmp\seplistB}
\def\seplistB{\let\next=\seplistP
   \expandafter\ifx\space\tmp \let\next=\seplistC \let\nexxt=\seplistM \fi
   \ifx##\tmp \let\next=\seplistC \let\nexxt=\seplistH \fi
   \ifx{\tmp  \let\next=\seplistG \fi
   \ifx}\tmp  \let\next=\seplistC \let\nexxt=\seplistF \fi
   \next
}
\def\seplistC{\afterassignment\nexxt \let\next= }
\long\def\seplistP#1{\seplistX#1\def\tmp{#1}\seplistN}
\def\seplistM{\seplistX{ }\def\tmp{ }\seplistN}
\def\seplistH{\seplistX{##}\def\seplistLx{}\seplistA}
\def\seplistN{\edef\seplistLx{\expandafter}\seplistLx \seplistL \seplistA}
\long\def\seplistG#1{\def\seplistLx{}\seplistX{{#1}}\seplistA}
\def\seplistF{\seplistT\expandafter{\expandafter{\the\seplistT}}\seplistZ}
\long\def\seplistX#1{\seplistT\expandafter{\the\seplistT#1}}
\def\seplistZ{\let\tmp=\sepused 
   \expandafter\seplistS\expandafter{\the\toks0{##1}}%
   \expandafter\expandafter\expandafter\endgroup\expandafter\tmpa\the\seplistT
}
\long\def\seplistI#1{\ifx\seplistI#1\seplistI\else
   \addto\seplistL{\seplistD{#1}{#1}}\expandafter\seplistI \fi
}

Comments to the implementation

We read the parameter token-per-token similarly as the solution of this thread and store these tokens in \seplistT token list. The internal macro \seplistL includes the list of separators in the form:

\seplistD{sepA}{sepA}\seplistD{sepB}{sepB}\seplistD{sepC}{sepC}...

We store the already read token to \tmp and run \seplistL. More exactly: at the start, the temporary \seplistLx is emty. For each read token, we expand \seplistLx and \seplistL to the input stream and before executing it we reset \def\seplistLx{}. Now, the input stream is executed, i.e. the \seplistD macro is processed for each separator. The task of \seplistD{sepA}{sepA} is the following: to test if the \tmp is equal to the first token of its first parameter (s in this example). If it is true, then \seplistD (using \seplistE) adds the text \seplistD{epA}{sepA} (the first token from the first parameter is removed) to the temporary list \seplistLx which will be executed for the next token. If \tmp isn't equal to the first token of the first parameter then \seplistD does nothing.

For example, the next read token in \tmp is e. Then \seplistD{epA}{sepA} saves the \seplistD{pA}{sepA} to the \seplistLx, because the first letter is e. If the next token in \tmp is p, then \seplistD{A}{sepA} is stored to the \seplistLx. And finally, if the next token in \tmp is A, then \seplistD{A}{sepA} does not store \seplistD{}{sepA}, but it decides that separator is found because the first parameter is empty. It defines \sepused as its second parameter (sepA) and it does the end of this game by \seplistQ plus \seplistZ. If the last token in \tmp isn't A then the \seplistD{A}{sepA} does nothing and the chain is broken because the \seplistLx is set to empty in each step. The new chain can be built because \seplistD{sepA}{sepA} is still included in \seplistL which isn't changed during calculation.

Macro-programing in TeX is beautiful but it is different than the classical technique used by "normal" programming. We utilize here the fact that the code can create the code, i.e. data and code isn't strictly separed.