Extract substring up to specific character non-inclusive

Question

I was wondering if there is a way with StringCases to start at the beginning of a string and stop one character before a known character.

s = {{"This is small string 2 !"}, {"There is string 5 ! Or is it 2?"},
     {"This is String n !"}};

My problem is that I have a lot of these strings and not all are the same length and some have more characters after the exclamation mark.

I know it involves this code:

StringCases[#, Shortest[StartOfString~~___~~ ????????]]& /@ s

But I don't know how to finish it (where the ??????? are).

I would like my end result to look like this:

final={{"This is small string 2 "},{"There is string 5 "},{"This is String n "}}

I have tried searching for this on the forums and have had no luck.

If you could provide an answer or even a direction to go, that would be very much appreciated!

Thanks!

Also, if this isn't the most efficient way of doing this, or if you have any questions, please let me know.

score 10 · Accepted Answer · edited Apr 13 '17 at 12:55

Use named Pattern to extract a part of matched substring:

StringCases[#, str : Shortest[StartOfString ~~ ___] ~~ "!" :> str] & /@ s

{{{"This is small string 2 "}}, {{"There is string 5 "}}, {{"This is String n "}}}

or

StringCases[#, str : Shortest[StartOfString ~~ ___] ~~ "!" :> str] &@Flatten@ s

{{"This is small string 2 "}, {"There is string 5 "}, {"This is String n "}}

As Martin notes in the comments, another approach is to capture Longest sequence of characters from a negated character class. In this case we don't have to use a named pattern which introduce additional overhead, hence this approach should be more efficient. Since string patterns by default are greedy, Longest can be omitted in this case:

StringCases[#, StartOfString ~~ Except["!"] ...] &

Translating StringExpressions into equivalent regular expressions sometimes gives substantial increase in performance: this is what Mathematica always does under the hood, but not always in the optimal way. Here is verbatim semantic translation:

StringCases[#, RegularExpression["^[^!]*"]] &

(read here on how to find out what a regex Mathematica generates from a StringExpression).

And another regex without capturing group and without use of a negated character class:

StringCases[#, RegularExpression["^.*?(?=!)"]] &

Note that this last regex can't be expressed as a combination of usual Wolfram Language patterns because it uses a positive lookahead zero-length assertion which has no equivalent in the world of Wolfram Language symbolic pattern objects.

Performance comparison

Here is a timing comparison of the all suggested solutions (on a very large number of short strings):

$HistoryLength = 0;
s1 = ConstantArray["There is string 5 ! Or is it 2?", 2*10^5];

{First@AbsoluteTiming[# /@ s1], #} & /@ {
    StringCases[#, str : Shortest[StartOfString ~~ ___] ~~ "!" :> str] &,
    StringCases[#, RegularExpression["^(.*?)!"] -> "$1"] &,
    StringCases[#, StartOfString ~~ Except["!"] ...] &,
    StringCases[#, RegularExpression["^[^!]*"]] &,
    StringCases[#, RegularExpression["^.*?(?=!)"]] &,
    StringDelete[#, "!" ~~ ___] &, StringExtract[#, "!" -> 1] &,
    StringTake[#, StringPosition[#, "!", 1][[1, 1]] - 1] &,
    StringDrop[#, {StringPosition[#, "!", 1][[1, 1]], -1}] &} // 
  Grid[#, Frame -> All, Alignment -> Left, FrameStyle -> Directive[Thin, LightGray]] &

MaxMemoryUsed[]

102886352

Another comparison suggested by Martin (on a string where it takes a while to find the "!"):

$HistoryLength = 0;
s2 = StringRepeat["There is string 5 ", 10^5] <> "! Or is it 2?";
results = {};

{(AppendTo[results, #2]; #1) & @@ AbsoluteTiming[#@s2], #} & /@ {
    StringCases[#, str : Shortest[StartOfString ~~ ___] ~~ "!" :> str][[1]] &,
    StringCases[#, RegularExpression["^(.*?)!"] -> "$1"][[1]] &,
    StringCases[#, StartOfString ~~ Except["!"] ...][[1]] &,
    StringCases[#, RegularExpression["^[^!]*"]][[1]] &,
    StringCases[#, RegularExpression["^.*?(?=!)"]][[1]] &,
    StringDelete[#, "!" ~~ ___] &,
    StringExtract[#, "!" -> 1] &,
    StringTake[#, StringPosition[#, "!", 1][[1, 1]] - 1] &,
    StringDrop[#, {StringPosition[#, "!", 1][[1, 1]], -1}] &} // 
  Grid[#, Frame -> All, Alignment -> Left, FrameStyle -> Directive[Thin, LightGray]] &

SameQ @@ results

MaxMemoryUsed[]

True

82707472

The conclusion: Martin's solution via negated character class with greedy quantifier outperforms others in general: RegularExpression["^[^!]*"]. See this dedicated Q&A about why the equivalent string expression StartOfString ~~ Except["!"] ... is two orders of magnitude slower.

This is exactly what I was looking for. Thank you for your quick response! — newtonjeep, Mar 12 '17 at 14:40
Alternatively: StringCases[#, StartOfString ~~ Except["!"] ...] & — Martin Ender, Mar 21 '17 at 10:17
The very last regex seems like a bad idea. I'm pretty sure that the negative character class and the greedy quantifier would be more efficient, and ungreedy quantifiers should be avoided whenever possible, because they can often lead to unwanted matches in more complicated patterns. — Martin Ender, Mar 21 '17 at 13:39
@MartinEnder I added the regex based on positive lookahead mainly for educational purposes, but as you can see from the timing comparison it is almost as efficient as your suggestion. I also do not agree with your statement that "ungreedy quantifiers should be avoided whenever possible". They do what they are intended to do and work quite reliably if you use them in the right way. — Alexey Popkov, Mar 21 '17 at 14:15
@AlexeyPopkov Try a string where it takes a while to find the !, e.g. StringRepeat["There is string 5 ", 100] <> "! Or is it 2?". The negative character class and greedy quantifier are much faster. The problem with the other solution in terms of efficiency is that ungreedy quantifiers require a lot of backtracking. It tries one character, then fails at the lookahead, then it tries two characters and fails at the lookahead, etc etc. The greedy solution just keeps going as long as possible and never has to backtrack. — Martin Ender, Mar 21 '17 at 14:21
As for using ungreedy quantifiers in general, I'm happy to continue the discussion in chat, but my main problem is that they're poorly understood and often result in less robust code (even though that's not the case here). — Martin Ender, Mar 21 '17 at 14:24
@MartinEnder I added new timing comparison, and it proves you are right about efficiency of greedy quantifiers when it takes a while to find the "!". Surprisingly, StringExtract beats regex in this case. — Alexey Popkov, Mar 21 '17 at 14:52
@AlexeyPopkov Yeah true, but I'm also surprised how insanely slow the pattern-matching equivalent of the fast regex becomes. — Martin Ender, Mar 21 '17 at 14:53

kglr · Answer 2 · 2017-03-21T11:17:23.193

3

StringDelete[#, "!"~~___]& /@ s
StringExtract[#, "!"->1]& /@ s
StringTake[#, StringPosition[## & @@ #, "!", 1][[1, 1]] - 1] & /@ s
StringDrop[#, {StringPosition[## & @@ #, "!", 1][[1, 1]], -1}] & /@ s

all give

{{"This is small string 2 "}, {"There is string 5 "}, {"This is String n "}}

edited Mar 21 '17 at 11:17

answered Mar 21 '17 at 10:46

kglr

394,356
18
477
896

score 0 · Answer 3 · edited Mar 21 '17 at 13:35

0

Here is a regex variant:

{{"This is small string 2 !"}, {"There is string 5 ! Or is it 2?"},
 {"This is String n !"}} /. s_String :>
      First[StringCases[s, RegularExpression["^(.*?)!"] -> "$1"]]

{{"This is small string 2 "}, {"There is string 5 "}, {"This is String n "}}

edited Mar 21 '17 at 13:35

Alexey Popkov

61,809
7
149
368

answered Mar 20 '17 at 03:44

J. M.'s missing motivation

124,525
11
401
574

Since the OP wants "to start at the beginning of a string and stop one character before a known character", the greedy quantifier + isn't the correct choice in the general case. The lazy version +? is more appropriate. – Alexey Popkov Mar 20 '17 at 04:03
Hmm, I see your point; thanks @Alexey. – J. M.'s missing motivation Mar 20 '17 at 04:10
The condition "to start at the beginning of a string" is forgotten too (it is the caret ^ ). – Alexey Popkov Mar 20 '17 at 04:22
Ah, this is what I get when trying to answer without checking with the references. Thank you! – J. M.'s missing motivation Mar 20 '17 at 04:29
2

A better alternative to ungreedy quantifiers is to use a greedy negative characters class, ^[^!]*. (Your regex will also fail if ! is the first character.) – Martin Ender Mar 21 '17 at 10:16
@J.M. I corrected your answer according to Martin's remark. Also I removed redundant capturing group (.*). – Alexey Popkov Mar 21 '17 at 13:36
@Alexey, thanks; I got too busy to address Martin's comment promptly. – J. M.'s missing motivation Mar 21 '17 at 14:13
@Martin, testing confirms it; thank you! – J. M.'s missing motivation Mar 21 '17 at 14:14

Extract substring up to specific character non-inclusive

3 Answers3

Performance comparison

Linked