I am willing to scrape the data from the website: website quotes
I thought of using scrapy framework so that I can scrape it. But I did't how I can work out when the values are changing when the date and time is changing.
I didn't found anything which can help me. Kindly, help me find out how I can scrape the website as I have mentioned above and extract all the quotes so that I can plant my analysis on the data.
How is it possible to scrape such a website with Scrapy? Please help.
The data of the website changes when the text, i.e., date and time frame changes on the website.
Hello @JafferWilson, I'm not sure what do you mean by "when the date and time is changing" but when I change the values in "Date" or "Hour" input boxes the page makes a XHR request like https://static.iqoption.com/api/quotes-history/quotes/2.0?to=1529528519&from=1529528460&active_id=1&_key=1529569800 which returns JSON data. If you want to scrape that data you need to make similar requests with scrapy and process the responses.
@wRAR This is awesome. Can you please guide me where exactly you come to know about this request? As I would love to know so tat in future I can make it out easily.. :)
@JafferWilson Chrome (Firefox works too) developer tools, the Network tab.
Hey! We're trying to use Scrapy issue tracker to track bugs and feature requests, for general help there are other support channels like https://stackoverflow.com (use tag "scrapy"). I'm closing this ticket - anyways, it seems @wRAR already provided some good suggestions.
Most helpful comment
Hello @JafferWilson, I'm not sure what do you mean by "when the date and time is changing" but when I change the values in "Date" or "Hour" input boxes the page makes a XHR request like
https://static.iqoption.com/api/quotes-history/quotes/2.0?to=1529528519&from=1529528460&active_id=1&_key=1529569800which returns JSON data. If you want to scrape that data you need to make similar requests with scrapy and process the responses.