f-type expansion

Question

I am still struggling with understanding the f-type expansion. What is it all about? The explanation on page 2 in interface3.pdf is not really satisfying.

In the given example

\tl_set:Nn \l_mya_tl { A }
\tl_set:Nn \l_myb_tl { B }
\tl_set:Nf \l_mya_tl { \l_mya_tl \l_myb_tl }

, how can a check that the content of \l_mya_tl is actually A\l_myb_tl?

Does it matter that \l_mya_tl is re-used in order to be set on the third line, and not another, hitherto unused token list variable, say \l_myc_tl?

Why does expansion stop after expanding \l_mya_tl as it is expandable after all?

Is there any thinkable scenario where f-expansion would continue after expanding the first token (\l_mya_tl, here)? How would \l_mya_tl need to be crafted in order to not interrupt further expansion?

Why would someone want to use f-expansion, which stops at some unpredictable place, when the argument is expected to be really fully expanded? (This is what f as in "fully" means to me.)

Answer 1

f-type expansion ends upon finding an unexpandable token; if this token is a space (character code 32, category code 10) it is gobbled.

Your \tl_set:Nf \l_mya_tl { \l_mya_tl\l_myb_tl } will first do recursive expansion of \l_mya_tl, leading to A. This is unexpandable, so the business stops here. The token list to assign is evaluated to A\l_myb_tl and \l_mya_tl is updated to contain this list.

Changing the contents of \l_myb_tl will also change the expansion of \l_mya_tl, because this one contains a pointer to \l_myb_tl, rather than the value this variable had at definition time.

If you want to freeze the value of the updated \l_mya_tl variable to the values of \l_mya_tl and \l_myb_tl you have to use either x-type or e-type expansion.

These last two types lead to the same result, but with a big difference: e-type expansion can appear in expansion contexts, x-type cannot. Not so much of a difference in this case, because you're doing an assignment. Actually, there is no predefined \tl_set:Ne function, because it turns out that \tl_set:Ne would take twice as much time as needed by \tl_set:Nx.

\documentclass{article}
\usepackage{expl3,l3benchmark}

\ExplSyntaxOn

\tl_set:Nn \l_tmpa_tl { A }
\tl_set:Nn \l_tmpb_tl { B }
\tl_new:N \l_tmpc_tl
\cs_generate_variant:Nn \tl_set:Nn { Ne }

\benchmark:n { \tl_set:Nx \l_tmpc_tl { \l_tmpa_tl \l_tmpb_tl } }

\benchmark:n { \tl_set:Ne \l_tmpc_tl { \l_tmpa_tl \l_tmpb_tl } }

\stop

yields, on my machine,

3.16e-7 seconds (1.01 ops)
7.78e-7 seconds (2.39 ops)

In either case, \l_tmpc_tl is assigned AB.

Why would someone want f-expansion? Good question! Until a few months ago, there was no way to do full recursive expansion in expansion contexts. Things changed when the primitive \expanded was added to all engines (it used to be allowed only in LuaTeX), except Knuth TeX, of course.

Answer 2

Compare is with x-expansion:

\documentclass{article}
\usepackage{expl3}

\begin{document}
\ExplSyntaxOn
\tl_set:Nn \l_mya_tl { A }
\tl_set:Nn \l_myb_tl { B }
\tl_set:Nf \l_myc_tl { \l_mya_tl STOP \l_myb_tl }
\tl_show:N \l_myc_tl 

\tl_set:Nx \l_myc_tl { \l_mya_tl STOP \l_myb_tl }
\tl_show:N \l_myc_tl

\ExplSyntaxOff\end{document}

This will give

> \l_myc_tl=ASTOP\l_myb_tl .
<recently read> }

l.207 \tl_show:N \l_myc_tl

? 
> \l_myc_tl=ASTOPB.
<recently read> }

l.210 \tl_show:N \l_myc_tl

Answer 3

As pointed out in the other answers and comments, f-expansion is implemented using \romannumeral which was sometimes needed in expansion contexts before the \expanded primitive was available. This answer also mentions two use cases where it might still be of use, namely expansion without a known end point and lookaheads of the next unexpandable token.

Additionally, I'd like to point out a common use case where it's even wrong to use, as it gives undesired results. This is based on the fact that, while x-expansion continues fully expanding tokens beyond the first unexpanable token, f-expansion is more eager in the case \exp_not:n is used in the token stream.

If we look at the following examples, we see that expansion is the same when \exp:not:N (\noexpand) is used:

\cs_set:Npn \foo { [FOO] }

\tl_set:Nx \l_tmpb_tl { \exp_not:N \foo bar }
\tl_show:N \l_tmpb_tl

\tl_set:Nf \l_tmpb_tl { \exp_not:N \foo bar }
\tl_show:N \l_tmpb_tl

outputs

> \l_tmpb_tl=\foo bar.

> \l_tmpb_tl=\foo bar.

On the other hand, using \exp_not:n (\unexpanded) gives different results:

\tl_set:Nx \l_tmpb_tl { \exp_not:n { \foo } bar }
\tl_show:N \l_tmpb_tl

\tl_set:Nf \l_tmpb_tl { \exp_not:n { \foo } bar }
\tl_show:N \l_tmpb_tl

outputs

> \l_tmpb_tl=\foo bar.

> \l_tmpb_tl=[FOO]bar.

This is especially important when dealing with parts of the contents of token list variables via the \tl_head:, \tl_tail:, \tl_range: etc. functions. All those wrap their result in \exp_not:n. f-expansion may seem appropriate here, but it's actually not:

\tl_set:Nn \l_tmpa_tl { \foo bar }
\tl_set:Nx \l_tmpb_tl { \tl_head:V \l_tmpa_tl }
\tl_show:N \l_tmpb_tl

\tl_set:Nf \l_tmpb_tl { \tl_head:V \l_tmpa_tl }
\tl_show:N \l_tmpb_tl

outputs

> \l_tmpb_tl=\foo .

> \l_tmpb_tl=[FOO].

---

As pointed out by Phelype Oleinik, protected macros behave differently as well:

\cs_new_protected:Npn \protected_foo { \foo }

\tl_set:Nx \l_tmpb_tl { \protected_foo bar }
\tl_show:N \l_tmpb_tl

\tl_set:Nf \l_tmpb_tl { \protected_foo bar }
\tl_show:N \l_tmpb_tl

outputs

> \l_tmpb_tl=\protected_foo bar.

> \l_tmpb_tl=[FOO]bar.