How to get unbalanced opening brace behind macro argument?

Question

Editor's note: this question was significantly revised, it might or might not reflect OP's original intention. OP might want to rollback it and clarify the question/provide a proper MWE if it does not.

This question is for an academical purpose rather for any practical application.

Can you write an expandable macro in traditional TeX so that \macro{⟨argument⟩} yields {⟨argument⟩}{ ?
(The last thing of the replacement-text is to be an unbalanced opening curly brace of category 1.)

For example,

\macro {123{4 5{6}}789}

should, after some expansion steps, expands to

{123{4 5{6}}789}{

Answers should use only expandable operations (in expl3 term, works in f-type expansion); and \expanded is not available in traditional TeX.

Any (non-outer, non-\notexpanded) token might appear in the argument, including some internal tokens such as frozen font selection tokens.

Answer 1

In your question you say:

(The last thing of the replacement-text is to be an unbalanced opening curly brace of category 1.)

When referring to a macro, then the phrase "replacement text" usually denotes the replacement text that belongs to the definition of that macro.

David Carlisle already pointed out in a comment that the replacement text of the definition of a macro cannot contain unbalanced/unmatched braces/explicit character tokens of category 1 or 2.

TeXbook, chapter 20: Definitions (also called Macros), says:

\def⟨control sequence⟩⟨parameter text⟩{⟨replacement text⟩}

TeXbook, chapter 24: Summary of Vertical Mode, presents the grammar of TeX in Backus/Naur-notation. In that chapter you find:

⟨definition⟩ → ⟨def⟩⟨control sequence⟩⟨definition text⟩
⟨def⟩ → \def | \gdef | \edef | \xdef
⟨definition text⟩ → ⟨parameter text⟩⟨left brace⟩⟨balanced text⟩⟨right brace⟩

[...]

All occurrences of ⟨left brace⟩ and ⟨right brace⟩ tokens within the ⟨balanced text⟩ must be properly nested like parentheses.

So in any case the replacement text of the definition of a macro is formed by ⟨balanced text⟩ where braces are balanced/properly nested.

Therefore I suppose the question is not about a macro whose definition's replacement text contains unmatched/unbalanced braces but is about a macro

• whose definition's replacement text does not contain unmatched/unbalanced braces
• which takes an undelimited argument and which can serve the purpose of initiating an expansion-cascade which after a known amount of expansion-steps yields¹ the tokens forming the argument nested in curly braces and trailed by an opening curly brace which is not matched/balanced by a closing curly brace.

Question 1:

How shall braces surrounding \macro's argument be treated if present?

Question 2:

How shall the case of \macro's argument not being surrounded by braces be handled?

If

• the answer to question 1 is "braces surrounding \macro's argument may be stripped off and another pair of braces may be added instead (probably yielding braces of different character code)" and
• the answer to question 2 is "In the result, the argument shall in any case be surrounded by braces" and
• the question was not about traditional TeX²,

then you can extract the gist of Skillmon's answer to the question "Fully robust way to access the first item in a token list (expandably)", i.e., combine \expanded and \unexpanded.

The following \macro delivers the result after triggering two expansion-steps/after two "hits" by \expandafter.

---

¹The answerer of the question's opinion is that the phrase "yields" does not very precisely describe the requirements.

²The phrase "the question was not about traditional TeX" is bold to emphasize that the answerer is aware that this answer does not really answer the question as in this answer it is deviated from the requirements/conditions given in the question.

---

\long\def\macro#1{\expanded{\unexpanded{{#1}}\expandafter}\expandafter{\iffalse}\fi}
\long\def\processtwoargs#1#2{%
  \message{^^J\unexpanded{argument 1: (#1)  argument 2: (#2)}}%
}%
% Tests:
\expandafter\expandafter\expandafter\processtwoargs\macro{first}second}
\expandafter\expandafter\expandafter\processtwoargs\macro{123{4 5{6}}789}second}
\expandafter\expandafter\expandafter\processtwoargs\macro{###}###}
\message{^^JJust make sure there is no confusion about hash-doubling:}
\message{^^J\unexpanded{argument 1: (###)  argument 2: (###)}}
\csname bye\endcsname
\csname stop\endcsname
\endinput

Messages on terminal:

This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdftex)
 restricted \write18 enabled.
entering extended mode
(./test.tex 
argument 1: (first) argument 2: (second)
argument 1: (123{4 5{6}}789) argument 2: (second)
argument 1: (######) argument 2: (######)
Just make sure there is no confusion about hash-doubling:
argument 1: (######) argument 2: (######) )
No pages of output.
Transcript written on test.log.

Answer 2

I'll make the same assumption as Ulrich Diez's answer, except the \expanded/\unexpanded part.

As far as I know the idea this solution can be ported to Knuth TeX without any problem.

I change the plan a bit for simplicity:

• first, "serialize" the token list into a list of N-type tokens with the following principle

principle:
a token X is serialized to two N-type tokens AB such that r-expansion of AB results in X, and A ≠ \relax
actual encoding:
begin-group character X is encoded to \bgroup_handler ⟨X in catcode other⟩,
end-group character X is encoded to \egroup_handler ⟨X in catcode other⟩,
blank space is encoded to \space_handler ⟨any N-type token⟩
anything else is encoded to \exp_end: ⟨that token⟩

• then append the serialization of a single begin-group character,

• then deserialize the whole thing.

That's the idea. I have some code, but not in some directly-runnable-on-Knuth-TeX form. If you insist, keep reading.

---

It requires a working \char_generate:nn function, which can be implemented by preparing all the 256×2 characters for all the char codes.

The actual code builds on Ulrich Diez's code for getting the string representation of { and } https://tex.stackexchange.com/a/628363/250119 , modified a bit to return the space in catcode other instead of space.

The new part's detail is in a file somewhere in my repository https://github.com/user202729/TeXlib/blob/main/test_imperative.tex#L644-L773 . It requires some other files in that repo to be compiled.

Side note, in that code "r-expansion of ⟨token list⟩" means "o-expansion of '\romannumeral ⟨token list⟩'".

For now, if you want to run the code you need to download the whole repo and run the file with LuaLaTeX to "compile" the code. Which is not very easy.

But the principle is there, and it works in my testing. (by serialize/deserialize the token list and observe the output, and also by implementing \weirdfirstarg to extract the first argument and see it works correctly)

The time complexity is unfortunately higher than the promised O(n² log n) where n is the number of tokens, for now.