Scrapy: How to call 3rd party function in parse() callback.

[hanjihun]

What I want to implement is:

def parse(self, response):
    param = {}
    self.send_request(self, param)

def send_request(self, param):
    url = "www.sample.com/auto/"
    yield FormRequest(url, callback=self.parse_auto, formdata=param, method="POST")

def parse_auto(self, response):
    ...

In the above code, the self.send_request(self, param) function does not work. I am on right way?

[kmike]

Hey @hanjihun,

This is how Python works, there is nothing Scrapy-specific: parse method is a generator, and you need to iterate over its results:

def parse(self, response):
    param = {}
    for req in self.send_request(self, param):
        yield req
    # or, in Python 3:
    # yield from self.send_request(self, param)

https://stackoverflow.com can be a better channel for support questions (use tag "scrapy").

[hanjihun]

@kmike Thank you