Huginn: my problem about getting cookie

[hillerliao]

I want to scrape new articles from this web page http://m.ebrun.com/top/original

it provides restful api, but it requires cookie to submit the get request.

when I requst the url with get method , it generates the cookie and save it locally.

and then the web refresh the request itself with the cookie.

My target is to send a get request with website agent and send the cookie in the reponse header to the next agent. How can I fullfil this ?

thank you.

[dsander]

Hi, you can use this Scenario as an example. It logs into a Huginn instance using a cookie: https://huginnio.herokuapp.com/scenarios/19

[hillerliao]

I do not really get what the example said. My problem is how to get the cookie first. The site do not require me to log in by a post request with username and password. It just requires a get request and it will set cookie . Can i simulate this step by a certain agent? @dsander

[dsander]

You can use a PostAgent and set the method to get, change emit_events to true. You can try run the Agent and it should return the cookie in the headers section of the event. I also had to set disable_redirect_follow:

{
  "post_url": "http://m.ebrun.com/top/original",
  "expected_receive_period_in_days": "1",
  "content_type": "form",
  "method": "get",
  "payload": {},
  "headers": {},
  "emit_events": "true",
  "no_merge": "false",
  "output_mode": "clean",
  "disable_redirect_follow": "true"
}