Electricitymap-contrib: Swiss (CH) ENTSOE production data wrong

[brunolajoie]

ENTSOE API for Actual Generation Per Production Type gives inaccurate data for Hydro, Hydro Storage, and Gas.

• Hydro & Hydro storage: On average, two third of the total annual output is missing (13.5TWh vs. 36.3 TWh actual cumulated in 2016)
• non-nuclear thermal: 100% is missing. (0 vs. 3.1TWh actual in 2016)

Would there be another live data stream available that we could use?
We should provide feedback to ENSTOE

Official 2016 data: http://www.bfe.admin.ch/themen/00526/00541/00542/00630/index.html?lang=fr&dossier_id=00765

[toboz]

Hey Bruno,

Where do you have the gas data from? There is no gas generation in Switzerland AFAIK, maybe just co-firing for certain waste incineration streams.

[brunolajoie]

You are right, it's not Gas that is missing, its non-nuclear thermal generation. I've updated my initial comment.

[Dalius-ENTSO-E]

I asked swissgrid to check this. The problem is not on swissgrid side, because they simply resend what they receive. Bad data quality is not acceptable and we will work on this.

[toboz]

Also, where do you have the 8.4 TWh cumulated data for hydro from? I'll need to reference this.

[brunolajoie]

Hello @toboz, 8.4 was without storage, with storage its 8.4 + 5.1 = 13.5TWh cumulated for the year 2016, according to the CH production data I have from ENTSOE API

[Dalius-ENTSO-E]

Strange data for hydro production type are reported by EEX. Request to check data delivered, waiting for feedback. Indeed, hydro generation should be ~36,3TWh. TP shows ~13.6TWh. Installed capacities for hydro (reported by EEX) are also not matching with installed hydro installed capacities published on EEX portal. I hope that EEX will solve this issue soon. If not, we will ask help from Swiss NRA.

[brunolajoie]

Thanks a lot Dalius.
If I get this right, the issue is in the hand of EEX? Any news from them? If there is anything we can do, let me know!

[brunolajoie]

Hello @Dalius-ENTSO-E, any news on the subject?
I'm tempted to remove Switzerland form the map, so just wanted to check with you before doing so.

[toboz]

I've pursued this topic for a while now. SwissGrid and ElCom are both of
the opinion that the EU regulations on electricity generation transparency
don't apply in Switzerland and therefore are not willing to invest any
significant effort in resolving this issue. This is despite the political
mainstream lip service, for example by the energy minister and the director
of the federal energy office (BFE), that Switzerland needs an electricity
agreement with the EU (Stromabkommen). Obviously there is a discrepancy
between the target state and the actual state. Since profuse sweet-talking
didn't work, the only other option is to change the laws in Switzerland.
I'm on it, but it will take a while to write a letter to a selection of MPs
and explain the situation to them in sufficient detail...

On 10 January 2018 at 09:49, Bruno Lajoie [e-mail] wrote:

Hello @Dalius-ENTSO-E https://github.com/dalius-entso-e, any news on
the subject?
I'm tempted to remove Switzerland form the map, so just wanted to check
with you before doing so.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tmrowco/electricitymap/issues/892#issuecomment-356538763,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AbsG5TIjW5unakuoqUwdwoCaDXIGwNTAks5tJHmxgaJpZM4Q19qP
.

[brunolajoie]

Does anyone know whether historical data at hourly granularity with correct hydro data is available somewhere ?

[toboz]

@brunolajoie The problem is that there are a lot of small hydro plants in Switzerland with a complicated ownership structure, so I'd be surprised if a centralised data source already existed.

Could we dutchify (https://github.com/tmrowco/electricitymap/pull/1303) Switzerland in the meantime, till I find the right opportunity to change the laws? Assume 60% hydro, 30% biomass, 8% solar and 2% wind for the 'Unknown' part.

Formula E in Zurich needs to know their footprint ;)

[brunolajoie]

How pressing / real is this formula E needs? :)

[toboz]

They just finished and so the event disappeared from the media. So it's not pressing. It is unbelievably real though, the Greens were criticizing this event because it is not sustainable enough: no one should be driving above 200 km/h according to them.

[corradio]

We were made aware of following message from swissgrid:

The readings collected from Swissgrid (eg border exchange, balancing energy) will be published in quarter-hour resolution in the energy overview Switzerland.
You can find this information here:
https://www.swissgrid.ch/de/home/operation/grid-data.html
The data on the composition of the stream are not covered by Swiss grid. This data is owned by the power plant operators and directly to the power exchange (Swissix) forwarded. From there you will be forwarded to the Association of European Transmission System Operators ENTSO-E, who on the transparency portal published freely accessible. Why no values are displayed on Electricitymap.org for Switzerland in part, I can not answer unfortunately. Unfortunately, this has nothing to do with Swissgrid.

Maybe ENTSOE improved their signal?

[PaulCornelissen]

I think something needs to be done about Switzerland.

I was looking at north Italy just now:

They are importing about 12% of their CO2 emissions from Switzerland currently, because Switzerland is unknown and it is assumed to have the same CO2 intensity as northern Italy, which is probably to high. I estimate that this is causing northern Italy CO2 intensity to be ~10% to high.

I think there's two options that are better than we have now apart from actually fixing Switzerland:

• Calculate a static CO2 intensity for Switzerland and use that for exports.
• Calculate the CO2 intensity with the data that we have, knowing that we're mostly missing hydro, this should be a upper limit.

Or we could try to "Fix" Switzerland by calculating the unknown part and making an estimate for that?

[corradio]

What we should definitely do to fix the IT exchanger is to update this file:
https://github.com/tmrowco/electricitymap-contrib/blob/master/config/co2eq_parameters.json#L694
with a statically computed CO2 intensity. I think we could do that by looking at the yearly mix given by an official source? We just need to make sure it takes into account imports.

[andreasamsler]

Hi there, do these data sources help any further ?

• https://opendata.swiss/fr/dataset/wochenstatistik-elektrizitatsbilanz-erzeugung-und-abgabe-elektrischer-energie-in-der-schweiz
• https://opendata.swiss/fr/dataset/schweizerische-statistik-der-erneuerbaren-energien
• https://opendata.swiss/fr/dataset/schweizerische-gesamtenergiestatistik
• https://opendata.swiss/fr/dataset/schweizerische-elektrizitatsstatistik-aussenhandel-der-schweiz-mit-elektrizitat-nach-landern
• https://opendata.swiss/fr/dataset/gesamte-erzeugung-und-abgabe-elektrischer-energie-in-der-schweiz

If the data is only available as PDFs, I can help address that to the respective data stewards.

[seba1337]

Why Switzerland isn't coloured, like there's missing data, but if you click on it, the graph of production works?

[corradio]

the graph is incomplete as the data reported is incomplete and therefore we
don't color it to avoid misleading users :)

On Sat, Dec 21, 2019 at 8:41 PM Sebastian Pleško [e-mail]
wrote:

Why Switzerland isn't coloured, like there's missing data, but if you
click on it, the graph of production works?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tmrowco/electricitymap-contrib/issues/892?email_source=notifications&email_token=AAMUIKF33SJ2SGEMXOI74OLQZZWOFA5CNFSM4EGX3KH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHPCIHQ#issuecomment-568206366,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAMUIKCNJWVWSRROTLZH5YTQZZWOFANCNFSM4EGX3KHQ
.

--

Olivier Corradi

Founder, CEO

https://www.linkedin.com/in/oliviercorradi https://twitter.com/corradio

tmrow.com

[Tketa]

Hi @andreasamsler !

I'm interested in building a parser for Switzerland :)

After asking around on Slack I was told that the focus is on realtime data (updated ~hourly)

Do you know if it would be possible to get data updated more frequently?

[Tketa]

Updated comment as indeed the problem is the data source, hence this issue 🤦‍♂

[toboz]

The thermal is waste derived. No good way to approximate solar though.

man. 23. des. 2019, 23:49 skrev Jérémy Gotteland [e-mail]:

@corradio https://github.com/corradio Looking at the map and data from
Entsoe, we have actually have realtime data for nuclear and hydro
generation, which based on this data
https://www.bfe.admin.ch/bfe/de/home/news-und-medien/open-government-data.exturl.html/aHR0cHM6Ly9wdWJkYi5iZmUuYWRtaW4uY2gvZGUvc3VjaGU_a2/V5d29yZHM9NDAy.html
represented ~91.5% of the electricity production of the country in 2018

Given that solar + wind is 4% and thermal ~4.5%, unsure what the next
steps are here. Should we look for data for those sources even though they
contribute so few? Or could we use a fixed approximation instead?

Appreciate I'm new around here so if the reasoning is wrong please let me
know 😄 Interested in learning more about the domain.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tmrowco/electricitymap-contrib/issues/892?email_source=notifications&email_token=AG5QNZJIOASQCWSWFD2RKQTQ2E57BA5CNFSM4EGX3KH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHSCF2Q#issuecomment-568599274,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AG5QNZJC7TJHJLJC5HN6BPLQ2E57BANCNFSM4EGX3KHQ
.

[Juma7C9]

I see that on the ENTSO-E site, hourly data for Switzerland total load is available ([1]).
As total load, if I understand correctly, is load = production + import - export, and we know partial production, the missing amount should be (missing) production = load - import + export - (known) production.
Its carbon intensity should then be estimable comparing historical data from the database with the known totals as reported by opendata.swiss (in particular [2], table T24).

This would also resolve the incorrect carbon intensity estimates in regions where Switzerland is usually exporting, as already noted by @PaulCornelissen.

[andreasamsler]

Hi @Jum7C9, @tketa, @toboz
I'd like to suggest to bring this issue to the attention of the Swiss Federal Office of Energy, respectively to their OGD unit: [email protected]
I suspect they share our needs to have this data available for re-use. Would you mind if I connect them with you?

[toboz]

Grüeziwohl @andreasamsler yes that seems easier than putting a parliamentary majority behind this or launching an initiative to change the laws ^^

[toboz]

I brought it to their attention via e-mail incl. all the previous communication with ElCom and SwissGrid.

[toboz]

The leader of the geoinformation service wrote that he can't add anything to the negative answer of the ElCom. He can recommend trying matthias.galus at bfe.admin.ch, leader of the digital innovation office. It is feeling kafkaesque now so I suggest someone with better political contacts in Switzerland takes over @andreasamsler :)

[andreasamsler]

Thanks for your efforts, @toboz et al. / I studied German Literature, hence my positive feelings regarding 'kafkaesk' situation. Seriously, am following up on this with my Opendata.ch hat and my board colleagues. Keep you posted with results.

[toboz]

Lovely! Let me know if you would like a copy of the communication with ElCom et al. where the specific regulations of the EU and the perceived interests of different stakeholders in Switzerland are also described.

[andreasamsler]

I handed the issue over to the EnergyData working group of Opendata.ch, where some of these stakeholders are present, too. The group's slack-channel is called "wg-energydata" (on the opendatach_workspace): https://opendata-ch.slack.com/join/shared_invite/enQtODY0NjgwMTI5NzYxLWJjNDhmODJkMDY3MzM0ZjViN2ZiNjVkMTQyOWZmNjRmMzBkYzg1YzY5Yzc2NTM1YTBhYmVjYzE3NGZhNmE5NWI

[corradio]

Fixed by https://github.com/tmrowco/electricitymap-contrib/pull/2898