Position of a particular substring in an imported HTML file

Question

data4 = {{"050318", "111529"}}

Do[
 ID2 = data4[[i, 2]];
 s = Import["http://www.swift.ac.uk/xrt_curves/00" <> ID2, "Text"];
 s = StringSplit[s, "\n"];
 line = Position[s, "Flux Light Curve"],
 {i, 1, Length[data4]}]

The StringSplit works properly, but when I want to identify the position it gives line {} empty I expect instead a number like {{106}} for example

Even when I try with

Do[
 ID2 = data4[[i, 2]];
 s = Import["http://www.swift.ac.uk/xrt_curves/00" <> ID2, "Text"];
 s = StringSplit[s, "\n"];
 line = StringPosition[s, "Flux Light Curve"],
  {i, 1, Length[data4]}]

it gives me not a single line, but many empty ones.

Mr.Wizard · Answer 1 · 2012-02-29T02:03:57.970

8

The string "Flux Light Curve" does not appear as an element of s. If you are looking for a substring you will need a different function.

If you only want to know the positions of strings which contain "Flux Light Curve" as a substring, you might use:

Position[ s, x_ /; StringMatchQ[x, __ ~~ "Flux Light Curve" ~~ __] ]

In this ~~ stands for StringExpression.

The output from this:

StringPosition[s, "Flux Light Curve"]

is a list with many empty lists as elements, because most elements in s do not contain the substring. You can further process the result as needed. For example:

x = StringPosition[s, "Flux Light Curve"];

Position[x, {__Integer}]

DeleteCases[x, {}]

edited Feb 29 '12 at 02:03

answered Feb 29 '12 at 01:57

Mr.Wizard

271,378
34
587
1,371

Was looking at this answer related to a separte Q on querying key patterns. Why does StringMatchQ complain in v11 while giving the correct answer? Position[{"ab", "bc", "ac", "be"}, x_ /; StringMatchQ[x, "b" ~~ __]] >> StringMatchQ::strse: String or list of strings expected at position 1 in StringMatchQ[List,b~~__]. – alancalvitti Jul 21 '17 at 18:53
@alancalvitti By default Position looks at all parts of the expression, including heads, in this case List. See (7692), (67100). Simple solution: only pass expressions that match _String to StringMatchQ: Position[{"ab", "bc", "ac", "be"}, x_String /; StringMatchQ[x, "b" ~~ __]]—this is more general than adding Heads -> False. – Mr.Wizard Jul 22 '17 at 00:18

score 2 · Answer 2 · answered Mar 02 '12 at 10:52

Mr.Wizard's answer correctly identifies why your code wasn't working: you needed StringPosition rather than Position, and then to delete the empty cases.

I wonder if there isn't another way to attack your problem. You are importing an HTML file, so why not use that fact rather than treat it as plain text? You don't actually need all the HTML markup. So instead consider:

ss = Import["http://www.swift.ac.uk/xrt_curves/00111529", "HTML"];

This strips out the extraneous HTML stuff. So the first 100 characters are:

StringTake[ss, 100]

"Skip banner and navigation tools .
Dept. of Physics & Astronomy

  XROA

    US Site | "

Contrast this with

s = Import["http://www.swift.ac.uk/xrt_curves/0000111529", "Text"];
StringTake[s, 100]

(I had to insert this as a picture because inserting an HTML DOCTYPE statement really doesn't seem to work nicely with the StackExchange software, and rightly so!)

As you can see, you can obtain the positions of the desired substring within the whole, without having to split the string first.

StringPosition[ss, "Flux Light Curve"]

{{1797, 1812}, {2435, 2450}, {2465, 2480}}

Of course, you can always StringSplit in the same way and find the substring that contains the sub(sub)string:

sss = StringSplit[ss, "\n"];
StringPosition[sss, "Flux Light Curve"]

{{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, \
{}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {{52, 67}},
   {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {}, \
   {}, {}, {{1, 16}, {31, 46}}, {}, {}, {}, {}, {}, {}}

Position of a particular substring in an imported HTML file

2 Answers2