Electricitymap-contrib: Add US-SPP historical Wind & Solar and load forecasts refetch ability

[brunolajoie]

SPP has excellent data sources available here that we are not exploiting to its full potential.

Our current SPP parser could be improved by

• [x] adding ability to fetch wind & solar forecasts, as we did with the ENTSOE parser here
• [x] add ability to fetch load forecasts, as we did with ENTSOE here
• [ ] adding ability to fetch back time generation mix

[brunolajoie]

To be more precise,

• This address https://marketplace.spp.org/file-api/download/generation-mix-historical?path=%2FGenMix_2018.csv provides historical 5-minute generation mix for 2018 (csv here). These files go back until 2011

• This address https://marketplace.spp.org/file-api/download/midterm-resource-forecast?path=%2F2019%2F02%2F25%2FOP-MTRF-201902251200.csv provides the J+7 hourly wind and solar generation forecasts made on the 2019-02-25-12:00 local time (csv here). These files can be fetched back in time up to August 2013

• This address https://marketplace.spp.org/file-api/download/mtlf-vs-actual?path=%2F2019%2F02%2F25%2FOP-MTLF-201902251200.csv provides load forecasts made on the 2019-02-25-1200 local time, for the next 7 days, hourly. (csv here)

cc @systemcatch

[corradio]

@systemcatch did you already work on this? Else I might take a look at this this week.

[systemcatch]

@corradio just about to submit a PR. :wink:

[corradio]

I think all 3 have been done right @brunolajoie @systemcatch ?

[systemcatch]

@corradio the historical generation mix hasn't been done yet, I can start working on that soon. There's one file per year so they end up being kind of huge.

[corradio]

ok great! No rush on this one. I've updated the issue. Again, thank you very much for your very valuable help

[systemcatch]

hey @brunolajoie, for the historical mixes do you want me to create a script that reads each file and outputs a .txt file for each year? Then I can send you the data to be added to your database.

[brunolajoie]

Hi @systemcatch, thanks a lot for this, we've found and integrated historical data ourself already!
I'm closing the issue as everything has been done!
awesome 💯