3

I'm trying to use cURL to download the contents of webpages that require forms to get to.

In a browser it looks something like this

1.) Login using POST
2.) Pick which page to go to using another POST
3.) Pick another page... using POST
4.) etc.. until I get to the page I want, 
    then download all textfiles linked to on that page.

I am attempting to do this using a bash script and some loops with the values that change for each POST.

My problem is how do I do multiple POSTs with cURL? Must there be cookies involved?

FYI the website is http://metagenomics.nmpdr.org/ (MG-RAST).

Austin Richardson
  • 181

1 Answers1

2

I'm beginning to realize that I only post questions to stack* because I know that as soon as I do, I will figure out the answer.

The answer is indeed cookies:

$ curl -d "who=your_mom&when=last_night" -D mycookies http://outbacksteakhouse.com/date.cgi
$ curl -d "what=took_her_out_for_a_steak" -b mycookies http://outbacksteakhouse.com/date.cgi

-D saves to mycookies
-b loads mycookies

Add another -D to the second command and you might make it to 2nd base.

Austin Richardson
  • 181