Electricitymap-contrib: Add French overseas territories (EDF opendata API)

[alixunderplatz]

Hey guys,

the PR for the island of Reunion by @PETILLON-Sebastien made me curious about the other French overseas territories.
Looks like there is historical data for all of the other territories for 2016-2018, but no live data:

Martinique:
https://opendata-martinique.edf.fr/explore/dataset/courbe-de-charge-de-la-production-delectricite-par-filiere/table/?sort=-date_heure

Guadeloupe:
https://opendata-guadeloupe.edf.fr/explore/dataset/courbe-de-charge-de-la-production-delectricite-par-filiere/table/?sort=-date_heure

Guyana:
https://opendata-guyane.edf.fr/explore/dataset/courbe-de-charge-de-la-production-delectricite-par-filiere/table/?sort=-date_heure&refine.date_heure=2018

Mayotte (different URL!) has data for 2018:
https://opendata.electricitedemayotte.com/explore/dataset/courbe-de-charge-2018/table/?sort=date_heure

And even for Corse we have historical production data (2016-2019) including imports from Italy. We could use the historical imports to finally implement & validate the complex interconnector structure between Italy-Corse-Sardinia-Italy from the ENTSOE data:
https://opendata-corse.edf.fr/explore/dataset/production-delectricite-par-filiere/table/?sort=-date_heure

This website seems to have data for all the territories except Mayotte in one place:
https://opendata-corse-outremer.edf.fr/explore/dataset/courbe-de-charge-de-la-production-delectricite-par-filiere/table/?disjunctive.territoire&sort=-date_heure&refine.territoire=Corse

[PETILLON-Sebastien]

Yes, I wanted to do a parser for Corsica in the first place (as I'm spending my vacations here).
But no live data is available. Only France and Reunion (and still, not perfect for reunion).
But I don't know if you use historical data to give prediction about current production ?

As a french guy, I'd like to show France is fine, but our overseas lands have dirty electricity !

[alixunderplatz]

@PETILLON-Sebastien historical data is also collected in some cases, where production data is available only on the next day or even later. you can find some code examples for fetching historical production in some of the existing parsers.
writing the parsers for all these islands should be really easy based on the work you've already done for Reunion. :)

[makatianatir]

Hi,
First of all, great work.

I was interested in Reunion and other french overseas territories.
It seems Reunion has disappeared from the list.
The data have been updated on 20 août 2019 13:14.
(https://opendata-reunion.edf.fr/explore/dataset/prod-electricite-temps-reel/information/)
Maybe the parser needs a few tweaks, i was not able to find the issue.

[PETILLON-Sebastien]

@makatianatir I had a look at the API on EDF openApi.
Looks like they didn't provide any data for several hours. I'm not 100% sure, but I think the parser is not bugged. If you see any problem on the RE parser, we can open an issue on it !
@alixunderplatz Yes, I could give it a try. I was more into finding new real-time data, or improving RE estimation (I asked for data about biomass to a NGO in order to improve biomass/coal repartition estimations, but I still have no response).

[PETILLON-Sebastien]

Hello all, I tried to improve Reunion real time data by searching more information about the bagasse/coal repartition in the dedicated power stations. Spoiler : we won't be able to improve it. But I wanted to share with you this attempt, this way, we won't do it again.
I already had bagasse/coal production data from EDF. I found some real time data about the quantity of canne input in the two power station. I thought maybe we could find some correlation over time (canne input leads to bagasse production leads to electricity production).
I share with you my script, retrieved data and graphs about it.
My conclusion is that the power stations are using bagasse and coal simultaneously and that we can get nothing from this.
reunion_bagasse_coal_graphs.pdf
reunion_bagasse_coal_data.xlsx
reunion_bagasse_coal_script.py.txt

[PETILLON-Sebastien]

Hey, I'd like to work on the french overseas territories for historical data.
I have some questions :

• with only historical data, do I have to implement fetch_production only when target_time is defined and raise an NotImplementedError when it's not ?
• I found this API https://opendata-corse-outremer.edf.fr/explore/dataset/courbe-de-charge-de-la-production-delectricite-par-filiere/table/?disjunctive.territoire&sort=-date_heure which is managing all overseas territories historical data, I'd like to build a single parser. How do I play with the zone_key ?
• how is retrieved historical data ? Is it done once only, is it manual ?

Thanks a lot for your help.
PS: Reunion's is off today, I checked and EDF's API seems to be down (https://opendata-reunion.edf.fr/explore/dataset/prod-electricite-temps-reel/table/?sort=date)

[corradio]

Hi!
For the first point, you're absolutely right.
For the second, you can build a single parser. Note that the fetch_production actually receives as parameter the zone_key, so you can use a single parser to fetch different zones.
For the last point, we manually retrieve historical data for now.

[PETILLON-Sebastien]

Thanks a lot for this fast response.
Hope I'll have some time this week-end to start working on it.
Maybe I'll ask some help on slack if I have a doubt on something about the second point.

[corradio]

Closing due to inactivity. Thank you all for your inputs (and well done Sebastien)