What do ExplSyntaxOn and ExplSyntaxOff do?

Question

Related to the question What do \makeatletter and \makeatother do? for LaTeX2e, here related to LaTeX3

What do \ExplSyntaxOn and \ExplSyntaxOff do?

Answer 1

The LaTeX3 coding language, expl3, uses : and _ as 'letters' in function and variable names. This allows use of expl3 code-level material in a document

\cs_new:Npn .... % fails: command \cs followed by characters '_', 'n', 'e', 'w', etc.
\ExplSyntaxOn
\cs_new:Npn .... % works here: command name \cs_new:Npn

The \ExplSyntaxOn command activates this, and also alters the treatment of spaces and line endings so these are ignored, meaning that

\ExplSyntaxOn
\cs_new:Npn \my_function:nn #1#2
  {
    % Code hre
  }

does not require % at the end of each line (compare to 'normal' LaTeX2e code). To allow inclusion of spaces in the output, ~ is made into a 'normal' space by \ExplSyntaxOn. Note that this change in the behaviour of spaces is not essential to access expl3 names, but the two are closely tied together as the combination allows 'clearer' programming in expl3.

\ExplSyntaxOff reverses this, returning :, _ and ~ to their values before \ExplSyntaxOn and making spaces 'important' again.

At a technical level, the idea of category codes in command names is exactly the same as in What do \makeatletter and \makeatother do?, although the \ExplSyntax... switches do more.

---

A couple of points to note about the treatment of ~ within a code block. First, as ~ is a 'space' inside a code block, a ~ at the start of a line is ignored by TeX

\ExplSyntaxOn
~ \cs_new:Npn .... 

is exactly the same as

\ExplSyntaxOn
\cs_new:Npn .... 

as the 'space' is ignored.

Secondly, while ~ is a 'space' inside a code block, \␣ and \~ are still distinct and have the LaTeX2e definitions.

---

There are a few more details to discuss for completeness. \ExplSyntaxOn also sets catcodes appropriately for other characters that implicitly are part of its syntax:

• TAB — ignored (just like spaces)
• " — an ‘other’ char, necessary for hexadecimal input such as \int_eval:n { "F } (= 15)
• & — alignment character for tabular material
• ^ — superscript char for ^^64 style notation
• | — an ‘other’ char, necessary for expl3 boolean expressions

Answer 2

I want to add some information about the combination of LaTeX2e kernel macros and expl3 functions. The command \ExplSyntaxOn doesn't change the catcode of the special token @. So combining of LaTeX2e kernel macros and expl3 functions you have to call \makeatletter, too:

\ExplSyntaxOn
\makeatletter
\cs_set:Npn \@maketitle…