0

I am trying to use Mathematica to analyze the data given by the Boston Marathon concerning running time, etc. The website http://registration.baa.org/2015/cf/Public/iframe_ResultsSearch.cfm gives a form for requesting data. One can get a lot by just selecting the Gender to be either M or F and requesting 1000 results. However, the data comes in just groups of 25. How can I use Mathematica to get all 1000 results without manually getting the data in groups of 25? Better yet ... how can I get info on all the runners, not just the top 1000?

I know this is not necessarily a Mathematica question, but Mathematica is my favorite tool for this kind of thing and I'm trying to figure out how to use it.

What I've done so far: I can run a search - say, for Males and request 1000 results. When the first 25 results come up, I can view the page source and near the bottom their is a line that begins with <input type="hidden" name="start" value="26"/>. By changing the value from 26 to say, 27, and saving the results as a webpage, I can see the results numbered 27 to 51 when I click on the "Next 25 Records" button. I don't know how to get Mathematica to submit this form for me, nor click through the "next results". This may just be a poor way of approaching the problem, and a more elegant method exists.

J. M.'s missing motivation
  • 124,525
  • 11
  • 401
  • 574
GregH
  • 1,909
  • 12
  • 25
  • 1
    If there's no public API for automated requests, this is basically a question about hacking their form submission interface to scrape the data. Therefore, the main issue is really quite unrelated to Mathematica. Have you tried anything? – Jens Aug 19 '15 at 18:01
  • Yes, I did, though didn't describe anything above. And yes, I just tried to hack their submission form to get more than 25 results at a time to no avail. I did figure a way to change the start value of subsequent pages of results - so instead of seeing 1-25, then 26-50, etc., I can view 27-51 if I desire. But I did that with manual hacking, and haven't yet figured out how to get Mathematica to do that for me. I don't want this to be a case of "Solve my problem for me," but many on this forum seem to know a lot about a lot of things and hoped someone could point me in the right direction. – GregH Aug 19 '15 at 18:10
  • 1
    Perhaps you can add details of what you have already done manually, if you think it can be automated. – Jens Aug 19 '15 at 18:29
  • I've added some details above. Thanks. – GregH Aug 19 '15 at 18:44
  • 1
    This question is about web scraping. – Enrique Pérez Herrero Aug 19 '15 at 18:50
  • You should take a look at this website instead: http://boston.r.mikatiming.de/2015/ . It seems to provide public access to results for more than 1,000 athletes, and presents up to 1,000 results per page in a nicely formatted table. – MarcoB Aug 19 '15 at 20:40

0 Answers0