1

Main Question

I'm currently making a web crawler, but I'll need to deal with login windows sometimes. All simple web crawlers created with Mathematica I've seen before cannot deal with input boxes or login windows.

So I would like to ask how to let Mathematica "input" things on webpages.

Example

Now I would like to know the results on the example website I created when the input is 1,2,..,10.

This is what the website will look like when visiting:

img1

Then we need to input an integer:

img2

Finally, press Submit to get this:

<|"x" -> 5|>

So the final result should be like:

{<|"x" -> 1|>, <|"x" -> 2|>, <|"x" -> 3|>, <|"x" -> 4|>, <|"x" -> 5|>, <|"x" -> 6|>, <|"x" -> 7|>, <|"x" -> 8|>, <|"x" -> |>, <|"x" -> 10|>}

How can I do this using only Mathematica?

Community
  • 1
Wjx
  • 9,558
  • 1
  • 34
  • 70
  • @Kuba er, maybe I just didn't got your point...... I'm trying to let mathematica input things and click buttons on webpages in general, and the link can be replaced by some other links like se's login page, so I don't see much relevance in CreateCloudExpression. – Wjx Aug 12 '16 at 07:40
  • 1
    I think this is a duplicate of Web browsing automation with Mathematica. I would say that crawlers typically don't work like this, but that's another matter. (The forms on websites send post requests to the websites' backends, and that's what a crawler would do, too, if it did it at all. Typical crawlers are not concerned with forms.) – C. E. Aug 12 '16 at 08:44
  • @C.E. I see, I will delete my off topic comments then. – Kuba Aug 12 '16 at 09:09
  • @C.E. I see, maybe I choose the wrong word, I mean to say something that maps over webpages and collect data. And seemingly this is a duplicate, but after so many version updates in these years, I suppose it's still good to open it for a while and see if there's any new methods without involving extra packages. – Wjx Aug 12 '16 at 09:54
  • I agree that this is a duplicate. Anyway, what is your operating system? On the Mac you can call an AppleScript from Mathematica that fills the forms and clicks the buttons. On windows you can use AHK. In both cases that is a last resource as you should try to use URLFetch for sending POST and GET requests instead of controlling the GUI. – Gustavo Delfino Aug 12 '16 at 13:20
  • @GustavoDelfino I'm operating on Windows. mathematica 11.0 – Wjx Aug 12 '16 at 13:24

0 Answers0