What is the best way to expand macros inside keyval input in expl3

Question

I think LaTeX packages rarely use macros as key names. And some users expect macros inside the keyval input will be expanded recursively first. But l3keys in expl3 doesn't provide this functionality at this time. So it is a job of package writers. I can think of one way to do this. But it may be a little slow. Therefore I ask this question in hope of finding the best solution.

Note: only macros surrounded by commas are expanded inside keyval input.

\documentclass{article}
\ExplSyntaxOn
\keys_define:nn { test }
  {
    key1 .code:n = \tl_to_str:n {#1},
    key2 .code:n = \tl_to_str:n {#1},
    key3 .code:n = \tl_to_str:n {#1},
    key4 .code:n = \tl_to_str:n {#1},
    key5 .code:n = \tl_to_str:n {#1},
    key6 .code:n = \tl_to_str:n {#1},
  }
\NewDocumentCommand \mycommand {m}
  {
    \keys_set:nn { test } { #1 }
  }
\ExplSyntaxOff
\begin{document}
\mycommand{key1=val1,key2=val2,key3=val3,key4=val4,key5=val5,key6=val6}
\def\mymacroa{key3=val3,key4=val4}
\mycommand{key1=val1,key2=val2,\mymacroa,key5=val5,key6=val6} % error
\def\mymacrob{key2=val2,\mymacroa,key5=val5}
\mycommand{key1=val1,\mymacrob,key6=val6} % error
\end{document}

Answer 1

You can recursively scan the list of options, expanding once an item that's a single control sequence.

\documentclass{article}
\ExplSyntaxOn
\clist_new:N \l__ljr_keysexp_clist
\bool_new:N \l__ljr_keysexp_bool
\cs_new_protected:Nn \ljr_keysexp_set:nn
 {
  \clist_clear:N \l__ljr_keysexp_set_clist
  % scan the options one by one and put them in a clist,
  % expanding once the control sequences that are found
  \bool_set_false:N \l__ljr_keysexp_bool
  \clist_map_inline:nn { #2 }
   {
    \bool_lazy_and:nnTF { \tl_if_single_p:n { ##1 } } { \token_if_cs_p:N ##1 }
     {
      \clist_put_right:No \l__ljr_keysexp_set_clist { ##1 }
      \bool_set_true:N \l__ljr_keysexp_bool
     }
     {
      \clist_put_right:Nn \l__ljr_keysexp_set_clist { ##1 }
     }
   }
  % we found a macro, let's repeat
  \bool_if:NTF \l__ljr_keysexp_bool
   {% there is a macro
    \ljr_keysexp_set:nV { #1 }\l__ljr_keysexp_set_clist
   }
   {% no macro, set the keys
    \keys_set:nV { #1 } \l__ljr_keysexp_set_clist
   }
 }
\cs_generate_variant:Nn \ljr_keysexp_set:nn { nV }
\keys_define:nn { test }
  {
    key1 .code:n = \tl_to_str:n {#1},
    key2 .code:n = \tl_to_str:n {#1},
    key3 .code:n = \tl_to_str:n {#1},
    key4 .code:n = \tl_to_str:n {#1},
    key5 .code:n = \tl_to_str:n {#1},
    key6 .code:n = \tl_to_str:n {#1},
  }
\NewDocumentCommand \mycommand {m}
  {
    \ljr_keysexp_set:nn { test } { #1 }
  }
\ExplSyntaxOff
\begin{document}
\mycommand{key1=val1,key2=val2,key3=val3,key4=val4,key5=val5,key6=val6}
\def\mymacroa{key3=val3,key4=val4}
\mycommand{key1=val1,key2=val2,\mymacroa,key5=val5,key6=val6} % error
\def\mymacrob{key2=val2,\mymacroa,key5=val5}
\mycommand{key1=val1,\mymacrob,key6=val6} % error
\end{document}

But it's not the best approach to key-value systems.

Answer 2

The following remarks do not answer your question, so probably don't upvote this "answer". The following remarks do not fit in a comment. They are intended to draw attention towards questions arising in the context.

---

A list of tokens to be gathered alternately either for keys or for values, which - besides tokens undefined at the time of parsing/splitting the list but defined at the time of further processing the split (value-)components - can contain expandable tokens to be expanded only when not gathering for a value?

For gathering tokens for a key a tail-recursive loop is imaginable, which gathers non-expandable tokens as part of the key and expands expandable tokens instead of gathering them. Gathering a key ends when finding a comma or an equal sign or when all tokens of the list are gathered. If gathering a key ends due to finding a comma, the next key is gathered. If gathering a key ends due to finding an equal sign, the corresponding value is gathered.

How should gathering tokens for a value be done? You could say, just gather tokens unexpanded until either reaching the next comma (not nested in explicit catcode1/2 character tokens) or having gathered all tokens of the list.

But - if it is about arbitrary input by arbitrary users you have to reckon with everything - how do you handle the case that a user gets the idea to have a macro in the value-part, whose expansion contains a leading comma to end the value-part?

Something like this:

\def\MoreKeyVals{, keyc=valc, keyd=vald,}
\mycommand{keya=vala, keyb=valb \MoreKeyVals \keye=vale}

If something like this is also considered, the question arises how an algorithm for gathering tokens belonging to a value should decide whether to expand things or not to expand things?

Besides this exact specifications are needed regarding the removal of curly braces/surrounding spaces. E.g., shall surrounding spaces be removed before expansion or after expansion? That might make a difference in case expansion of expandable tokens yields space tokens.

If at some stage stringification of names of keys plays a rôle, one might ask the edge question of how to handle something like \mycommand{{keya=vala, }keyb=valb}?

Answer 3

\begin{advertisement}

My expkv package has a built in grammar for key expansion, that grammar also supports expanding macros and using their replacement as additional key=value input. This is possible by prefixing the macro with R:␣ (with ␣ being a space). In general the grammar scans for <expansions>:␣ at the start of a key, and executes the <expansions>. For a complete list of defined expansion rules see the package documentation.

Caveat: Keys that contain :␣ in their names become harder to input in \ekvparse (no problem in \ekvset), in which case you need to use \ekvparse{<code1>}{<code2>}{: key: name: with: colons} and the parsed key name will be key: name: with: colons.

To allow this mechanism in l3keys we wrap it by passing the key list once through \ekvparse to apply our expansion grammar, and the result gets forwarded to \keys_set:nn. We need two full expansions, the first for \ekvparse the second for our auxiliary macros to take effect.

\documentclass{article}
\usepackage{expkv}
\ExplSyntaxOn
\cs_new:Npn \ljr_set_keys_expanded:nn #1#2
  {
    \exp_args:Nne \keys_set:ne {#1}
      { \ekvparse __ljr_set_keys_expanded:n __ljr_set_keys_expanded:nn {#2} }
  }
\cs_generate_variant:Nn \keys_set:nn { ne }
\cs_new:Npn __ljr_set_keys_expanded:n #1
  { , { \exp_not:n {#1} } }
\cs_new:Npn __ljr_set_keys_expanded:nn #1#2
  { , { \exp_not:n {#1} } = { \exp_not:n {#2} } }
\keys_define:nn { test }
  {
    key1 .code:n = \tl_to_str:n {#1},
    key2 .code:n = \tl_to_str:n {#1},
    key3 .code:n = \tl_to_str:n {#1},
    key4 .code:n = \tl_to_str:n {#1},
    key5 .code:n = \tl_to_str:n {#1},
    key6 .code:n = \tl_to_str:n {#1},
  }
\NewDocumentCommand \mycommand {m}
  { \ljr_set_keys_expanded:nn { test } {#1} }
\ExplSyntaxOff
\begin{document}
\mycommand{key1=val1,key2=val2,key3=val3,key4=val4,key5=val5,key6=val6}
\def\mymacroa{key3=val3,key4=val4}
\mycommand{key1=val1,key2=val2,R: \mymacroa,key5=val5,key6=val6} % error
\def\mymacrob{key2=val2,R: \mymacroa,key5=val5}
\mycommand{key1=val1,R: \mymacrob,key6=val6} % error
\end{document}

\end{advertisement}

Answer 4

Create a new expanded variant of \keys_set:nx?

\documentclass{article}
\ExplSyntaxOn
\keys_define:nn { test }
  {
    key1 .code:n = \tl_to_str:n {#1},
    key2 .code:n = \tl_to_str:n {#1},
    key3 .code:n = \tl_to_str:n {#1},
    key4 .code:n = \tl_to_str:n {#1},
    key5 .code:n = \tl_to_str:n {#1},
    key6 .code:n = \tl_to_str:n {#1},
  }
\cs_generate_variant:Nn \keys_set:nn { nx }
\NewDocumentCommand \mycommand {m}
  {
    \keys_set:nx { test } { #1 }
  }
\ExplSyntaxOff
\begin{document}
\mycommand{key1=val1,key2=val2,key3=val3,key4=val4,key5=val5,key6=val6}
\def\mymacroa{key3=val3,key4=val4}
\mycommand{key1=val1,key2=val2,\mymacroa,key5=val5,key6=val6}
\def\mymacrob{key2=val2,\mymacroa,key5=val5}
\mycommand{key1=val1,\mymacrob,key6=val6}
\end{document}