What `\cs_new_protected` and `\cs_generate_variant` do?

Question

After my question Undefined control sequence while adding a new parameter to a existing \ExplSyntaxOn code, I trying to fully understand the code behind it. So far, so good, with:

1. The expl3 package and LATEX3 programming
2. https://www.texdev.net/2010/05/23/from-newcommand-to-newdocumentcommand/
3. Using key values in \NewDocumentCommand
4. Always use \NewDocumentCommand instead of \newcommand?

I found a lot of explanations, except one. What are the arguments to this \cs_new_protected:Npn command? A confusing thing is:

...
\cs_new_protected:Npn \user_name_refs:nnnn #1#2#3#4
{
  ...

I understand that \user_name_refs:nnnn #1#2#3#4 refers to my function named \user_name_refs which receives 4 parameters which are nnnn, Unexpanded token or braced token list. But what the Npn are doing in the \cs_new_protected:Npn?

These Npm argument seems to be the same as these from The expl3 package and LATEX3 programming

All functions have a base form with arguments using one of the following argument specifiers:

1. n Unexpanded token or braced token list.

   This is a standard TEX undelimited macro argument.

2. N Single token (unlike n, the argument must not be surrounded by braces).

   A typical example of a command taking an N argument is \cs_set, in which the command being defined must be unbraced.

3. p Primitive TEX parameter specification.

   This can be something simple like #1#2#3, but may use arbitrary delimited argument syntax such as: #1,#2\q_stop#3. This is used when defining functions.

4. T,F These are special cases of n arguments, used for the true and false code in conditional commands.

   ...

Further argument specifiers are available as part of the expansion control system. These are discussed in the next section, Expansion control.

Actually, this The expl3 package and LATEX3 programming, seems quite complete. Is there another one out there? There is no mention of \cs_new_protected on The expl3 package and LATEX3 programming guide. May be it should be added to it?

On these questions, I can find small tips about what \cs_new_protected does:

1. Convert token to string in LaTeX3

   You have to do a loop over the characters to check. Be careful that neither \str_if_in:nnTF nor \peek_after:Nw are expandable, so \cs_new_protected:Nn should be used.

2. Use expl3 inside a LaTeX2e macro

   Instead of \newcommand, it's better to use \cs_new_protected:Npn, if not \NewDocumentCommand of xparse (probably even a better choice).

3. A LaTeX3 new macro/function definition problems

   If the command you want to define has no signature, you must define it with \cs_new:Npn or \cs_new_protected:Npn.

   Beware that you should use \cs_new_protected:Npn or \cs_new_protected:Nn (same rules apply) whenever the code contains non expandable functions (those without a red full or hollow star in the manual). Not the case here, because \int_eval:n is fully expandable.

4. A guide to understanding expandability: when to write protected functions and when not to

   So the 'correct' way to write LaTeX3 code is that if you use anything that is not expandable (i.e. not starred in the documentation) in your code, then you have to use \cs_new_protected:Npn or similar, and not \cs_new:Npn, etc.

5. latex3 complaining about Undefined control sequence. But it is defined!

   Use \cs_new_protected:Npn when the function does unexpandable jobs (such as setting token lists or sequences).

But no explicit explanation about what are these Npm arguments to the \cs_new_protected... Although, LaTeX3: Correct way to define a macro with :o, gives a little more light after quoting the file The LATEX3 interfaces from l3kernel – LATEX3 programming conventions

There's no magic involved. When you say \cs_new_protected:Nn, you're using \def or \gdef in disguise. ... it's undoubtedly better to go step by step:

\cs_new_protected:Nn \__a_one:n
{
    ...
}
\cs_generate_variant:Nn \__a_one:n { o }

Because the \__a_one:n function would have to be defined anyway. So there's no point in setting up a complicated mechanism for this.

To be more specific, after that code, the meaning of \__a_one:o would be

\exp_args:No \__a_one:n

...

There would be no other practical way for a hypothetical \cs_new_protected:Nn \__a_one:o handling the required expansion to basically define \__a_one:n under the hood and then applying \cs_generate_variant:Nn.

So, I have to use this \cs_new_protected with \cs_generate_variant:Nn. But what this \exp_args:No \__a_one:n means? Then, the same question applies to:

1. What \cs_generate_variant:Nn is doing?
2. What \cs_new_protected is? A function? A modifier? A reserved keyword? A function declaration syntax?

---

Related questions

1. Should something like \cs_generate_variant:Nn \foo_my_func:n { N } throw an error?
2. What is a function's signature? [LaTeX3]
3. expl3 function for fixed number of expansion steps
4. Is it possible to separate the declaration of a LaTeX3 variable of type control-sequence from the act of setting its value?
5. What's the purpose of \use_none_delimit_by_q_nil:w and friends in LaTeX3?
6. LaTeX3 iteration and token list comparisons-code improvement
7. Define an expandable function for comparing a token list to a string in LaTeX3
8. f-type expansion
9. Writing a token list conditional with expl3
10. Protected expansion in expl3
11. Passing more arguments to mapping functions in LaTeX3
12. Effiency and speed of expl3 "datatypes"
13. How to test whether a token variable exists?
14. Complete example from LaTeX3 intro
15. Why are there so many underscores in LaTeX3 commands?
16. How to use \cs_generate_variant:Nn to compare token lists?
17. Comparing LaTeX3 token list to string

Answer 1

In expl3, every function name has two parts: First comes the function base name (not sure is that's the actual name for it, but I will call it that in this post to make the distinction clearer), similar to TeX or LaTeX2 macro names but with optional underscores in it, and after that a list of argument specifiers follows, separated with a colon from the base name.

The important part here is, that the argument specifiers are part of the function name! You cannot leave them out, and slight changes may lead to a different function behavior.

What \cs_new_protected is? A function? A modifier? A reserved keyword? A function declaration syntax?

\cs_new_protected is the base name for the set of functions that define new, protected, long functions/macros. You cannot use this base name directly but you have to add the argument specifiers to make it a full function name.

The most common of this set of functions is \cs_new_protected:Npn. As already indicated in your question, the list of specifiers denotes the following:

• N is the full name for the new function to be defined,
• p is a parameter text argument which will form the parameter text for the newly defined function, and
• n is a normal, braced argument which holds the tokens for the replacement text of the newly defined function.

So the call

\cs_new_protected:Npn \myfunc #1#2 { ...#1...#2... }

is equivalent to the TeX defintion

\long\protected\gdef\myfunc#1#2{...#1...#2...}

plus some extra sanity checking. According to the expl3 naming conventions you should prefix your function name with the module name and add the argument specifiers, so the real LaTeX3 name would look something like \user_myfunc:nn.

What \cs_generate_variant:Nn is doing?

As noted, there are often several different functions with the same base name but with different argument specifiers which all represent different functions. The common base name just suggests that their behavior is sufficiently similar.

For example, the above function \cs_new_protected:Npn also exists in a \cs_new_proctected:Npx variant which is basically the same as the Npn variant, but with a fully expanded replacement text, i.e. \xdef instead of \gdef in the TeX equivalent.

With \cs_generate_variant:Nn we can automatically derive new function variants from a base variant by just giving the original, full function name (N) and the argument specifiers for the function to derive (n).

For example, we could derive a variant for \user_myfunc:nn where the first parameter has to be fully expanded and the second parameter should be taken from a variable name before passed as braced argument to the original function by calling

\cs_generate_variant:Nn \user_myfunc:nn { xV }

After that we have a new function \user_myfunc:xV in scope.

There are some rules about which function can be derived from which, e.g. you can only use variants that have N or n parameters to derive new functions from, or you can only derive the x variant from an n base parameter, but the c variant only from the N base, and so on. And there's also a special handling for conditional parameters T and F. You will find the full rules in the l3expan part of the expl3 documentation.