listings: highlight a prefixed keyword starting with a single quote

Question

This is about Rust and their lifetime notation:

\documentclass[crop,varwidth]{standalone}
\usepackage{listings}

\usepackage{xparse}
\usepackage{expl3}
\usepackage[svgnames]{xcolor}

\def\noprint#1{}

\ExplSyntaxOn
\NewDocumentCommand \lifetime { }
{
  \tl_set:No \l_demo_tl {\the\use:c{lst@token}}
  \regex_replace_all:nnN { ([\'\:][a-zA-Z]+[^\'\:]) } { \c{textcolor}\cB{ Violet \cE}\cB{ \1 \cE} } \l_demo_tl
  \tl_use:N \l_demo_tl
  \noprint
}
\ExplSyntaxOff
\lstset{
  basicstyle=\ttfamily,
  alsoletter={':},
  identifierstyle=\lifetime
}

\begin{document}
\begin{lstlisting}
struct Foo<'a, 'b>{} // highlight 'a and 'b -- doesn't work
struct Foo<:a, :b>{} // highlight :a and :b -- works
'a' // not highlighted
:a: // not highlighted
\end{lstlisting}
\end{document}

I tried to adjust the solution from this question [1] but somehow the regex doesn't match the single quote -- if I replace the single quote with a colon it is highlighted as expected.

• keywordsprefix didn't work for me
• moredelim highlights too much

Answer 1

Your version doesn't seem to work because ' isn't just a simple character in listings but a macro called \lst@um' internally. Apparently, this is used to choose between normal and upright quote characters. I don't know how the regex package can be made matching on this, so here's a version that uses the traditional approach.

The idea is basically the same as in the regex version. First we make ' a letter so that it's part of all identifier names. Then our \lifetime macro hooks into the identifier printing and scans the \lst@token token list (which stores the identifier tokens) before they are printed with the appropriate style applied. I won't explain the scanning in detail here, feel free to expand a few examples by hand to convince yourself they work. :)

With that scan we are able to distinguish three cases, where each can be assigned an individual style:

• Identifiers that don't start with ' (\@normalstyle),
• identifiers that start with ' but don't end in ' (\@lifetimestyle), and
• identifier that both start with and end in ' (\@charlitstyle).

Note the \unskip that is used when the style is applied in \@setlststyle to gobble up some spurious glues that would otherwise add unwanted space in the output. Perhaps they need to be removed if other column formats are used.

\documentclass{article}

\usepackage{listings}
\usepackage[svgnames]{xcolor}

\lstset{
    basicstyle=\ttfamily\small,
    alsoletter={'},
    identifierstyle=\lifetime,
    morecomment=[l][\color{gray}]{//},
}

\makeatletter
\def\@normalstyle{\color{blue}}
\def\@lifetimestyle{\color{red}}
\def\@charlitstyle{\color{green}}

\def\@setlststyle#1{%
    \edef\lt@temp{{\unskip\bgroup\noexpand#1}\the\lst@token{\unskip\egroup}}%
    \lst@token=\expandafter{\lt@temp}%
}

\begingroup
\catcode`\'=11

\gdef\lifetime{%
    \expandafter\lifetime@\the\lst@token\lst@um'\@end
}
\gdef\lifetime@#1\lst@um'#2\@end{%
    \if\relax\detokenize{#1}\relax
        \lifetime@'#2\@end
    \else
        \@setlststyle\@normalstyle
    \fi
}
\gdef\lifetime@'#1\lst@um'#2\@end{%
    \if\relax\detokenize{#2}\relax
        \@setlststyle\@lifetimestyle
    \else
        \@setlststyle\@charlitstyle
    \fi
}
\endgroup
\makeatother

\begin{document}
\begin{lstlisting}
 a    abc   // normal identifier
 a'   abc'  // normal identifier
'a   'abc   // timelife identifier
'a'  'abc'  // character literal
struct Foo<'a, 'b>{} // highlight 'a and 'b
\end{lstlisting}

\end{document}

Answer 2

After @siracusa posted his solution I had another look at the problem, and it is indeed as he pointed out that ' is a macro and not expanded into a "normal" symbol that can be used in the regex. Fortunately the regex package allows us to work with macros -- instead of ' or \' the regex uses \c{lst@um'} to match '. The lst@um macro has to be used for all "non or badly printable" characters (_, $, *, -, ...) - see the listings documentation.

\documentclass[crop,varwidth]{standalone}
\usepackage{listings}

\usepackage{xparse}
\usepackage{expl3}
\usepackage[svgnames]{xcolor}

\def\noprint#1{}

\ExplSyntaxOn
\NewDocumentCommand \lifetime { }
{
  \tl_set:No \l_demo_tl {\the\use:c{lst@token}}
  \regex_replace_all:nnN { ([\c{lst@um'}\:][a-zA-Z]+[^\c{lst@um'}\:]) } { \c{textcolor}\cB{ Violet \cE}\cB{ \1 \cE} } \l_demo_tl
  \tl_use:N \l_demo_tl
  \noprint
}
\ExplSyntaxOff
\lstset{
  basicstyle=\ttfamily,
  alsoletter={':},
  identifierstyle=\lifetime
}

\begin{document}
\begin{lstlisting}
struct Foo<'a, 'b>{} // highlight 'a and 'b -- works
struct Foo<:a, :b>{} // highlight :a and :b -- works
'a' // not highlighted
:a: // not highlighted
\end{lstlisting}
\end{document}