regexp capturing issue

Question

The following regexp code gives the result

\documentclass[11pt]{book} % use larger type; default would be 10pt


\usepackage{pgffor}
\usepackage{l3regex,xparse}
\usepackage{etextools}
\begin{document}  

\ExplSyntaxOn
\seq_new:N \l_uiy_result_seq
\NewDocumentCommand {\UiySplit } { m }
  {
    %\regex_extract_all:nnN { \D+ | \d+(?:\.\d*)? } {#1} \l_uiy_result_seq
    \regex_extract_all:nnN {(f)(\d+(?:\.\d*)?)(s)(\d+(?:\.\d*)?)} {#1} \l_uiy_result_seq
    \seq_map_inline:Nn \l_uiy_result_seq { item:~##1\par }
  }
\ExplSyntaxOff

\UiySplit{f234s222}

\end{document}

gives the following output

item: f234s222
item: f
item: 234
item: s
item: 222

Why is it capturing the whole string and outputting it?

Similarly:

\regex_extract_all:nnN {(f|s)(\d+(?:\.\d*)?){1,2}} {#1} \l_uiy_result_seq

is outputing

item: f234
item: f
item: 234
item: s222
item: s
item: 222

the first and fourth shouldn't be there? (well, I don't want them captured)

score 5 · Accepted Answer · edited Apr 13 '17 at 12:35

5

I have the feeling that your task would be best done using a grammar. Unfortunately, there is no such package yet out there (expect one some time next year), so we are stuck with regular expressions only. There may be hope using LPeg if you use LuaTeX, but I know nothing about those developments.

First, why the behavior: I simply followed Perl. The main task of the extract functions is to extract what they match, in your case, the whole string, and a secondary effect is that capturing groups are also, well, captured. But as far as I know, Perl does not provide a way to not capture the whole match, only to avoid capturing individual groups. If I am wrong and there is an accepted syntax for that task, I will most definitely consider adding it to l3regex.

In your situation (and in fact, in most), the various groups that you are capturing have different statuses [hmm... makes me think that perhaps returning a sequence of sequences may make more sense]. So when looping through the sequence, you would anyways have to keep track of the index, checking what its value modulo 4 (or whatever) is, or to remove items from the beginning of the sequence one by one. The fact that the whole match is captured simply means replacing 4 by 5, or discarding one more item from the sequence.

You may be interested in doing a replacement instead of an extraction, for an example applicable here see the code in this other answer of mine.

edited Apr 13 '17 at 12:35

Community

1

answered Mar 17 '12 at 08:15

Bruno Le Floch

44,937

My command strings are simple enough not to need a grammar. Although they could easily be represented in EBNF form it probably would be more complicated to parse them with some latex parser. Not that I wouldn't be interested in trying if some parser existed. With the l3regex parser I can write the regexp strings without any problem but interpreting the results and acting on them is the issue. – Uiy Mar 17 '12 at 18:34
The problem I'll probably have is that I'll end up with an "dimensional madness". If extract is capturing all the matches then as my string becomes longer I'll end up with more unwanted matches. The capture groups I have the more complex it will become to remove things. – Uiy Mar 17 '12 at 18:37
The way I see the regexp string is that if there is no capture group for the string it shouldn't return it in the sequence. If you look at my last example it should make it clear. There is no capture group for f234 yet it is being "captured". If I wanted it to be captured all I would have to do is add extra ( ) around (f|s)(\d+(?:.\d*)?). It seems extract has some "implicit" capturing groups. Possibly create an alternative to only return explicit capture groups? – Uiy Mar 17 '12 at 18:41
@Uiy: I am very wary of adding features that even Perl does not provide: they have had enough time to find out what is useful. I stand by the fact that you can use the replace functions to get \command{<submatch 1>}{<submatch 2>}: then nothing forces you to keep the <submatch 0> (full match, which you are trying to get rid of). – Bruno Le Floch Mar 17 '12 at 22:35
The only drawback of this is if some parts of the string are not in any match: they would remain. To solve that, you can add \G.*?\K to the start of the regular expression. \G anchors the match at the end of the previous match, .*? matches as few characters as possible, and \K resets the beginning of the match. Hence, \G.*?\K matches everything from the end of the previous match to the next "real" match, and thanks to \K it pretends that the starting point is the expected one (so that \0 is still correct). – Bruno Le Floch Mar 17 '12 at 22:36
LPeg is the way to go. It's pretty nice and easy to setup. Was a hassle to learn but worth the effort. – Uiy Mar 21 '12 at 12:31

score 1 · Answer 2 · answered Mar 17 '12 at 07:46

1

The regex maps the result into a sequence.

  \seq_map_inline:Nn \l_uiy_result_seq { item:~##1\par }

You can extract the values of interest from the sequence, by referring to the LaTeX3 manual for sequences.

answered Mar 17 '12 at 07:46

yannisl

117,160

Yes, but how am I suppose to know what to extract? The regexp returns the matches for me cause I don't know them so how would I know which ones to remove? (maybe only on extremely well behaved regexp's or knowing exactly how the l3regexp worked you might be able to do it) – Uiy Mar 17 '12 at 18:43

regexp capturing issue

2 Answers2