expl3: lowercase a token list

Question

I want to convert to lowercase a token list that is composed by letter, digits and the control sequence \\. I try to use \str_lowercase:n but the control sequence is not covered also to text.

\documentclass{article}

\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentCommand{\lowertl}{m}{%
  \tl_set:Nn \l_tmpa_tl {#1}
  \str_lowercase:f {\tl_to_str:N \l_tmpa_tl}
}
\ExplSyntaxOff


\begin{document}
  \lowertl{This is my\\ Text}
\end{document}

The output running xelatex is:

this is my““ text

Is an easy way to iterate over tokens in a token list and lowercase only the letters?

The reason str_lower_case is failing is because it converts everything inside to catcode letter – apparently the lccode of \ is ". I would've imagined \cs_new:Nn \__lower_it:n { \tl_map_inline:nn {#1} { \token_if_letter:NT {##1} {\tl_lower_case:n} {##1}}} would work, but it gets tripped up by \\ all the same. (This is getting back into about where I stopped understanding TeX guts, so forgive any misconceptions that others will surely correct me on.) — Sean Allred, Sep 01 '16 at 01:19
@SeanAllred There is no \token_if_letter:NT by default. Did you generate a variant? And you are not passing \tl_lower_case:n the argument it expects, I don't think. Also, I think this retains the \\ (or would) but strips the spaces. Am I misunderstanding? — cfr, Sep 01 '16 at 02:29
@cfr It's definitely there – is your installation perhaps outdated? And I was trying to test tl_map_inline here – there are other points which ignore spaces (e.g. tl_head), but the docs are ambiguous for this one. It's possible tl_map won't work at all. (And yes, the way the expansion works out, tl_lower_case is still getting its argument.) — Sean Allred, Sep 01 '16 at 02:32
@SeanAllred Thanks. I don't know. I just made it when I got errors with your code. And yes, I can see that it does get an argument but it seems not in the spirit of L3 to do it that way. And it definitely works. At least, my code works with my version of stuff. I assume it will still work with the updated version, though I don't know yet, obviously. — cfr, Sep 01 '16 at 02:48
@cfr Yep, your solution works with my distribution, but I'd agree it seems a wacky way of going about it :) In the absence of other approaches, though… — Sean Allred, Sep 01 '16 at 02:53
@SeanAllred Actually, I don't think it can have been that which gave me the error. Must have been something else. — cfr, Sep 01 '16 at 02:54
@SeanAllred I'd like a less wacky solution for something else I'm using this in. Although I think it is almost entirely safe in that case, whereas I don't think it is really safe here. It really seems like it should work inline, only I couldn't make it come out right. — cfr, Sep 01 '16 at 02:56
\str_lower_case:n does only case change tokens, but it works on a string basis. Are you trying to make the text lower case, in which case simply use \tl_lower_case:n. (BTW, the reason you see ““ is that without loading the T1 encoding that's the glyph used in the \ slot.) — Joseph Wright, Sep 01 '16 at 06:21

Joseph Wright · Accepted Answer · 2020-01-14T10:31:20.573

6

Update 2020-01-14

The LaTeX kernel now makes \\ robust out-of-the-box. When used with the latest expl3 function \text_lowercase:n, this works with no additional adjustments. The comment in the original answer about strings versus text remains valid: you are case-changing text.

Original answer

The function \str_lowercase:n is for making strings lower case, and is for programmatic data not for text. You want \text_lowercase:n. The only issue is that \\ is not engine robust and the implementation of \text_lowercase:n expects 'text' to be either character tokens, things which expand to character tokens or engine-robust commands. We can solve that by locally making \\ engine-robust:

\documentclass{article}

\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentCommand \LowerCase { m }
  {
    \group_begin:
      \cs_set_protected:Npx \\ { \exp_not:o \\ }
      \text_lowercase:n {#1}
    \group_end:
  }
\ExplSyntaxOff


\begin{document}
  \LowerCase{This is my\\Text}
\end{document}

If you make \\ robust globally then everything stays expandable

\documentclass{article}

\usepackage{etoolbox}
\robustify\\
\usepackage{xparse}

\ExplSyntaxOn
\DeclareExpandableDocumentCommand \LowerCase { m }
  { \text_lowercase:n {#1} }
\ExplSyntaxOff

\begin{document}
  \LowerCase{This is my\\Text}
\end{document}

Note that there is no need to store #1 in a token list variable in either case.

edited Jan 14 '20 at 10:31

answered Sep 01 '16 at 07:26

Joseph Wright

259,911
34
706
1,036

The team are thinking about \\ at the moment! This may yet change :-) – Joseph Wright Sep 01 '16 at 07:35
I have to store #1 because is defined in a function and retrieved later. The token list is used as the text in a tikz multiline node; does rebustifying \\have any side effect? – TeXtnik Sep 01 '16 at 10:35
How would you conserve spaces in user input when you do need to manipulate the token list? (I think I need to do it as a string, though perhaps I'm mistaken about that.) Maybe I should ask a new question. – cfr Sep 01 '16 at 11:18
@JosephWright: I tried both approaches but it seems they don't work inside a \begin{tikzpicture}[overlay, remember picture] environment. – TeXtnik Sep 01 '16 at 13:01
@cfr There's some careful set up inside \tl_lower_case:n that preserves spaces. One can do that a few ways: wipet doubtless will have somewhere posted a short version of the idea or I can provide a longer one – Joseph Wright Sep 01 '16 at 13:07
@TeXtnik Probably a new question is needed: as it stands, my approach should work for the case as outlined in this question. – Joseph Wright Sep 01 '16 at 13:08
@TeXtnik Yes, my @ hackery was also developed for a TikZ context. (Not with overlay, remember picture, but that's probably not crucial.) – cfr Sep 01 '16 at 15:15
@JosephWright If you have time and wouldn't mind, I would appreciate your taking a look at http://tex.stackexchange.com/questions/327619/avoid-hackery-how-can-i-preserve-spaces-in-a-format-specification-when-proces. Manuel has suggested sequences. I think this will work without @-hackery, but it does make things more complicated. I'm guessing it will also be slower. On the other hand, the code is so slow anyway, this may be of little consequence :(. – cfr Sep 01 '16 at 22:31
@JosephWright I opened a new question with the detail of using inside a TikZ node: http://tex.stackexchange.com/q/328522/2483 – TeXtnik Sep 08 '16 at 04:12

score 1 · Answer 2 · edited Jan 14 '20 at 10:35

This is not an expandable version (and I am not sure this is what is really requested)

The basic idea is to split the token at the places where \\ occurs, storing the parts in an \seq variable, change the case of the symbols in the list and glue the parts together, with \\ filled again.

\documentclass{article}

\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentCommand{\lowertl}{m}{
  \seq_set_split:Nnn \l_tmpa_seq {\\} {#1} 
  \seq_map_inline:Nn \l_tmpa_seq {
    \seq_put_right:Nx \l_tmpb_seq {\str_lowercase:n {##1}}
  }
  \seq_clear:N \l_tmpa_seq
  \seq_map_inline:Nn \l_tmpb_seq {
    \int_incr:N \l_tmpa_int
    \int_compare:nNnTF { \l_tmpa_int } < {\seq_count:N \l_tmpb_seq }
    {\seq_put_right:Nn \l_tmpa_seq { ##1 \\ } }
    {\seq_put_right:Nn \l_tmpa_seq { ##1} }% Don't add the \\
  }
  \seq_use:Nn \l_tmpa_seq {}
}
\ExplSyntaxOff


\begin{document}
\noindent\lowertl{This is my\\ Text @\\ FOO OTHER STUFF #}
\end{document}

expl3: lowercase a token list

2 Answers2

Update 2020-01-14

Original answer

Linked