With l3 regular expressions, how can I capture a command and replace it with its name?

Question

I have an input of the form òf a clist containing commands, such as {\my_cmd1,\my_xx_cmd2,\cmd3}. I first iterate through this list and remove commmas, the backslash \ etc.

I save the output in a |tl|. I wanted to then remove underscores "_", as it was failing, I tried using regular expressions. This works well for almost everything and maybe is a better approach. How can I remove the \ using regex and just leave the name of the command?

\documentclass{article}
\usepackage{xcolor}
\begin{document}
\makeatletter
\ExplSyntaxOn
\cs_set:Npn\my_remove_backslash_comma_and_space:n #1
{
\clist_set:Nn\l_tmpa_clist{#1}
    \clist_map_inline:Nn\l_tmpa_clist
      {
        \expandafter@gobble\string##1;
      }
}
\cs_gset:Npn\test:n#1
{
 \tl_gset:Nn\l_tmpb_tl{\my_remove_backslash_comma_and_space:n{#1}}
 \tl_use:N\l_tmpb_tl
}
\ttfamily
%\tl_set:No\l_tmpa_tl{\test:n{\my_var:NTF, \my_othervar:n}}
\tl_set:No\l_tmpa_tl{my_var:NTF}
% Replace underscores
\regex_replace_all:nnN
{ (_)}
{/}\l_tmpa_tl
\par\tl_use:N\l_tmpa_tl
\regex_replace_all:nnN
{(NTF)}
{FALSE}\l_tmpa_tl
\par\tl_use:N\l_tmpa_tl
\ExplSyntaxOff
\makeatother
\end{document}

Answer 1

You have many misconceptions regarding expansion of functions and variables in your code. This will be too much for a comment, so I'm summing them up as part of the answer first, and then come to the real answer how you might remove the underscores.

• \tl_gset:Nn shouldn't be used on \l_tmpb_tl, as that's a local variable, use local assignments such as \tl_set:Nn for local variables instead.

• \tl_(g)set:Nn and other assignments (\clist_set:Nn for instance) aren't expandable, functions using them should be defined protected, so \cs_gset_protected:Npn \test:n for instance.

• it is better to use \cs_new:Npn to define functions, use \cs_set:Npn and friends only if you're changing the definition of a macro you already allocated with a prior new variant, or inside a group for a temporary auxiliary macro. so \cs_new_protected:Npn \test:n and similar.

• if you only need a clist for a single loop, use \clist_map_inline:nn instead of \clist_set:Nn ... \clist_map_inline:Nn.

• \tl_set:No will expand \test:n once, which will not set \l_tmpa_tl to the final result of \my_remove_backslash_comma_and_space:n, but only to a single step of expansion thereof (so it'd start with \clist_set:Nn), but setting to the final result isn't possible at all, since \my_remove_backslash_comma_and_space:n isn't expandable.

---

Now to the reason why you couldn't remove _ as easily: After a \string the underscores will be of category other (so 12), but the _ inside of \ExplSyntaxOn are of category letter (so 11), and those at the document level of category math subscript (so 8). If you want to use tl functions to remove them, you should use the contents of \c_underscore_str as your replacement pattern instead. But you can have an easier life if you use the l3str functions instead of l3tl, since your contents are strings anyway (or at least you want them to become strings).

But things are worse, because when you call your \regex_replace_all:nnN the contents of \l_tmpa_tl don't contain any underscores at all with your \test:n-call (see above, you haven't expanded \string on any of the tokens, as your function isn't expanded, and even if you tried to, it'd be unexpandable and you'd most likely get an error).

To set a single variable to the result of your parsing you have two options, either build a function that is fully expandable and use an e-type expansion on it, or build a function that sets a variable and manipulates it. The following shows both methods:

\documentclass{article}
\usepackage{etl}
\ExplSyntaxOn
\str_new:N \l__yannisl_tmpa_str
\cs_new_protected:Npn \yannisl_unexp_remover:Nn #1#2
  {
    \str_clear:N \l__yannisl_tmpa_str
    \clist_map_inline:nn {#2}
      { \str_put_right:Ne \l__yannisl_tmpa_str { \cs_to_str:N ##1 } }
    \str_replace_all:Nnn \l__yannisl_tmpa_str { _ } {}
    \str_set_eq:NN #1 \l__yannisl_tmpa_str
  }
\cs_generate_variant:Nn \etl_new_replace_all:Nn { NV }
\etl_new_replace_all:NV __yannisl_exp_remover_underscore:nn \c_underscore_str
\cs_generate_variant:Nn __yannisl_exp_remover_underscore:nn { e }
\cs_new:Npn \yannisl_exp_remover:n #1
  { \clist_map_function:nN {#1} __yannisl_exp_remover_aux:n }
\cs_new:Npn __yannisl_exp_remover_aux:n #1
  { __yannisl_exp_remover_underscore:en { \cs_to_str:N #1 } {} }
\ExplSyntaxOff
\begin{document}
\ExplSyntaxOn
\ttfamily
\yannisl_unexp_remover:Nn \l_tmpa_str { \my_var:NTF, \my_othervar:n }
\token_to_meaning:N \l_tmpa_str
\par
\str_set:Ne \l_tmpa_str
  { \yannisl_exp_remover:n { \my_var:NTF, \my_othervar:n } }
\token_to_meaning:N \l_tmpa_str
\ExplSyntaxOff
\end{document}