How to test whether a token variable exists?

Question

expl3 provides predicates to test whether variables of various data types exist, e.g. \cs_if_exist:NTF, \tl_if_exist:NTF, \str_if_exist:NTF, \int_if_exist:NTF, etc. However, I couldn't find a similar function for the data type token (Part XV in the Interfaces3 documentation). So How can I test whether a token variable exists (for instance, one that was possibly created using \token_new:Nn)?

Here's a use case. Whenever I write LaTeX3 functions that take parameters, first thing I do inside the function's body is name the parameters, i.e. I initialize local variable with the raw parameters #1, #2. After this point I never use the raw parameters again; only the named variables.

So suppose I wish to write a function, \g, that takes a single argument that should be of type "function variable" that may or may not be defined, and then I wish to test whether this variable is defined. So \g's structure would be something like this:

\ExplSyntaxOn
\cs_new:Npn \g #1
{
    \token_new:Nn \f {#1}
    \token_if_exist:NTF \f {...} {...}
}
\ExplSyntaxOff

Except \token_if_exist:NTF is not available from expl3.

Answer 1

There are not really any 'token variables': the l3token module is a set of low-level functions for 'token stuff', reflecting the fact that TeX works with tokens and so we sometimes need to think at this level.

We cannot create/manipulate for example a letter token in the same way we can a cs token, so it is not really clear what \token_if_exist:N(TF) might do. For example, for a Unicode engine we can have the 'existence' of 1000s of character tokens, but presumably this is not what is to be tested. As such, it is very likely the case that \cs_if_exist:N(TF) will provide here the same useful information.

---

If you wish to store the exact token passed in a variable, you need to do that using a tl: this is irrespective of the nature of the argument. Thus we might have

\tl_new:N \l__my_tmp_tl
\cs_new_protected:Npn \my_demo:N #1
  {
    \tl_set:Nn \l__my_tmp_tl {#1}

Later, if you want to know if the cs given as #1 is defined, and if you don't want to just use #1, you'll need to get the value back from the `tl

\exp_after:wN \cs_if_exist:NTF \l__my_tmp_tl

Note that as \cs_if_exist:NTF needs the cs 'as is', I've had to use \exp_after:Nw to expand the tl. Personally I'd stick to just using #1 but this is ultimately your call!

---

Notably, l3token has not to-date (October 2017) been reviewed as carefully as some other parts of expl3. This review work has taken place as use of expl3 has spread and the ideas have therefore had a good 'airing'. It is likely that the team will look over l3token in the near future, and it is quite possible that some functions will be removed/altered.