curl – Automating Web Requests Using curl

curlweb

I have some "corporative" systems I have to use at work. Some of them are terrible, with frames, pop-ups and such.

I was thinking about automating some stuff using curl or something like that. I need login, click on hyperlinks, set some data, and them do some post.

Is it possible to use curl? Or am I'm going to be better using something like Selenium?

Best Answer

Yes, it's fully possible with curl. Most importantly will be to save and reload cookies between uses of curl with --cookie-jar. You can also post form data as needed. I usually use a Firefox add-on called Live HTTP Headers to capture what happens when I navigate a website. It will record any headers, but also any form posts which is very helpful when trying to figure out what to do with curl. I've written bash scripts automating various curl invocations and use pipes and temporary files (courtesy of mktemp) to do some limited processing on the webpage, but if have to do a lot of webpage processing I usually switch to Perl with LibWWW.

Related Question