Electricitymap-contrib: Philippines

[jarek]

http://www.wesm.ph/ has hourly generation breakdown for Luzon and Visayas. These are two of Philippines' three major island groups, covering about 75% of the population.

According to news reports, the third island group, Mindanao, was to join or has joined WESM in June or July 2017, but there seems to be no data yet. Maybe they're still integrating it.

I would like to work on this next week :)

[jarek]

I'm still digging through all the data and trying to get reliable real-time info for everything. Data so far:

Major data sources

• http://wesm.ph/ has fuel breakdown and demand numbers for the current day
• http://wesm.ph/inner.php/downloads/ has past information and some future forecasts
• https://www.ngcp.ph/power-situation.asp seems to be updated a couple of times a day, including on weekends, but not really regularly, and doesn't have fuel mix breakdown of generation

Regions and interchanges

Philippines electricity is split into three grids, roughly corresponding to Luzon, Visayas, and Mindanao (the three major island groups). These are not official "regions" or "provinces" and don't have their own ISO 3166-2 codes. I would probably suggest making codes up, e.g. PH-L, PH-V, and PH-M, to avoid conflicting with existing ISO 3166-2 codes. I looked into getting GeoJSON for them and it looks feasible from the Natural Earth dataset we use, with small changes to the shell script, I have it figured out.

There is currently no interchange between Visayas and Mindanao, see e.g. https://sg.news.yahoo.com/interconnecting-power-supply-mindanao-visayas-132845854.html

There is a connection between Luzon and Visayas, but its use is poorly reported (and not real-time). See schedules in https://www.ngcp.ph/weekly-outlook-vis.asp?location=LEYTE_SAMAR and reports in previous-day summary (PDS).

On 2017-09-01, next-day summary (NDS) indicated that on 2017-09-02 (Saturday), Visayas would import a peak of 333 MW from Luzon on top of its own 1521 MW local generation, so a non-trivial amount.

However, when checked on 2017-09-03, the PDS for Visayas for 2017-09-02 indicated HVDC of 0, with Local Generation being equal to Energy Demand = ex-post peak 1719 MW at 14:00. Similarly PDS for 2017-09-02 for Luzon indicated HVDC of 0 with Local Generation (ex-post peak 8383 MW at 14:00) also being equal to Energy Demand.

(Note that you can choose past dates for Previous-Day Summary and Next-Day Summary views.)

http://wesm.ph/inner.php/downloads/hvdc_schedule claims to contain past interchange numbers, however I have not seen it return anything other than 0 MW for the few recent dates I've tried, both ex-ante and ex-post.

Mindanao

This is the third major island group. There is currently no real-time or historical data for the region on WESM website, though there is some non-real-time data on NGCP website. They are currently being added to WESM, see progress reports at http://wesm.ph/inner.php/downloads/wesm_mindanao_top_weekly_report. Hopefully they'll be added to the other WESM data once complete.

Demand

Daily demand peaks are on the order of 8400 MW for Luzon and 1850 MW for Visayas.

Demand figures on the front page http://wesm.ph/ are loaded from http://wesm.ph/system_lwap_graphs.php from the *DemandVal variables - fairly easy to parse. These figures appear fairly reliable and accurate.

Forecasted demand: Luzon NDS, Visayas NDS

http://wesm.ph/inner.php/downloads/significant_variations_report claims forecast performance for Visayas is within 5% for 2017-08-25, for Luzon within 3%.

For further verification, see e.g.:

Per https://www.facebook.com/NGCPph/posts/1533064120083426, on 2017-08-25 Visayas forecast peak demand around 18:00-21:00 was 1775 MW, available capacity 1746 MW. Scraping http://wesm.ph/system_lwap_graphs.php on that day gave actual peak demand 1737 MW at 14:00, between 18 and 21 the peak was 1676 MW at 19:00 (but possibly the announcement lowered the demand).

http://wesm.ph/inner.php/downloads/market_prices_&_schedules for Real Time Ex-Post, Generation, Visayas for 2017-08-25 gave 1733 MW at 14:00, and 1722 MW at 19:00.

Visayas PDS set for 2017-08-25 gave RTD_DEMAND of 1737 and RTX_DEMAND of 1703 MW at 14:00 (RTD being ex-ante, RTX being ex-post); RTD_DEMAND of 1676 and RTX_DEMAND of 1652 MW at 19:00. (Get the CSV export for a slightly better view of the data).

The data seems wonky though, or I don't fully understand it. Values in NDS and PDS don't equal, and the value shown on home page (see Fuel Mix below) is different still. It looks like OFFER in NDS is incomplete, or more companies offer once it's clear that more power will be needed (or once the price goes up). I also have no idea what is going on with the HVDC interchange.

Production and fuel mix

This is trickier, particularly for Visayas.

The only reliable real-time fuel assignment I've found so far is on the http://wesm.ph/ home page. All other sources are either forecasts or past-day updates that only update once a day after midnight.

There are three part-pages that can be loaded in with AJAX:
http://www.wesm.ph/system_fuel_participants.php
http://www.wesm.ph/luzon_fuel_participants.php
http://www.wesm.ph/visayas_fuel_participants.php

However, the fuel pie chart data is the same in "system" and "luzon" files. It appears that that is the Luzon generation, and Visayas is in the correctly-named file.

Past fuel mix for the same day only can be retrieved by appending ?hour= URL param.

However, the real-time info reported on the home page doesn't line up. The sum of generation is different than the demand figure shown on the same page. In case of Luzon, generation is 2-4% lower than reported demand. However, in case of Visayas, generation is about 22-28% lower!

Timestamps are also a little confusing. Throughout the WESM site, zero-indexed hours and 1-indexed hours (with last datapoint of the day at 23 or 24, respectively) seem to be mixed fairly freely, and confusingly.

Requesting http://wesm.ph/visayas_fuel_participants.php?hour=1 gives a file with

var currHour = 2;
$('.data-hour-fp').html("@0100");

Requesting ?hour=0 gives

var currHour = 1;
$('.data-hour-fp').html("@0000");

Fuel mix data updates around 20-30 minutes after the hour, so that at around 02:20 we get data with "0100" and currHour = 2.

Past: http://wesm.ph/inner.php/downloads/market_prices_&_schedules has data for previous day only. E.g. at 2017-09-03 22:04 Philippine time it had data up to end of day 2017-09-02. On 2017-09-02 02:22 Philippine time it had data up to end of 2017-09-01. Data consists of production in MW by plant ID.

Forecast: http://wesm.ph/inner.php/downloads/day-ahead_projection_generation has forecast by plant ID.

Plant ID looks fairly feasible to match to fuel type: Market Network Model reports may be used for mapping plant IDs like 1MARVEL_G02 to more descriptive names and fuel types. http://www.kuryente.org.ph/power-plants also has a list of plants by name and fuel type, but doesn't appear to be exhaustive. In a quick attempt, on Luzon about 95% of projected production (7444 of 7781 MW) for 2017-09-02 22:00 was done by 24 plants, and I was able to fairly easily map 22 of them using the latest Market Network Model document and a search engine.

I will try to do a mapping of the plants to fuel type to see if it makes the reported past values line up with either values reported in fuel_participants.php or total values reported in PDS.

But if they don't, I'm a bit unsure how to proceed.

Simplest would be to take the pie graphs on home page (from fuel_participants.php) at face value and assume no interchange. A little bit better would be to calculate difference between generation and demand and add that in as "unknown". Will that be enough for our purposes?

Pragmatically: Luzon is a lot bigger than Visayas, and its data seems better. So maybe the 28% "unknown" for Visayas wouldn't matter so much.

[systemcatch]

@jarek
You weren't kidding about the novel, I think you could get a trilogy and film deal out of it if you keep going!

Being serious this is great work. I think with all the uncertainties involved in trying to extrapolate from the other data sources it's better to stick with simply taking the generation from the pie charts and allowing for some unknown amount like you say.

With Mindanao hopefully being added to the site soon we may find that the data set is improved substantially re: interconnections/reliability. Rather than trying to build a more complex parser I think a fairly simple one will better for adapting to changes in the near future.

If you want I can help out with the Plant ID mapping?

[jarek]

Sure :) If you want to play detective, having these plants mapped will be useful for later reference e.g. comparing reported fuel mix with reported production by plant.

These are sorted in descending production order based on production last week (based on CSV exported from http://wesm.ph/inner.php/downloads/market_prices_&_schedules). The first ~20 will get us most of the way there, the last ~20 are usually zero production.

Plants with same prefix ('3ILIJAN_G02' and '3ILIJAN_G01') are usually the same type, but not always, e.g. 1BURGOS_G01 is wind while 1BURGOS_G02 and _G03 are solar.

Luzon:

['1SUAL_G01', # coal
'3ILIJAN_G02', # gas
'3ILIJAN_G01', # gas
'3QPPL_G01', # coal
'1MARVEL_G02', # coal
'3PAGBIL_G01', # coal
'3CALACA_G02', # coal
'3CALACA_G01', # coal
'1MSINLO_G02', # coal
'1MSINLO_G01', # coal
'3STA-RI_G05', # gas
'3STA-RI_G06', # gas
'3STA-RI_G03', # gas
'3STA-RI_G04', # gas
'3STA-RI_G01', # gas
'3STA-RI_G02', # gas
'3SNGAB_G01',
'3SLPGC_G02',
'1CASECN_G01', # hydro
'3SLTEC_G02',
'3SLTEC_G01',
'3BACMAN_G01', # geothermal
'1SMC_G01', # coal
'3TIWI_C', # geothermal
'1MARVEL_G01', # coal
'3PAGBIL_G03', # coal
'1SMC_G02', # coal
'3MKBN_A', # geothermal
'1BAKUN_G01', # hydro
'1MAGAT_U03', # hydro
'1ANDA_G01', # coal
'1MAGAT_U04', # hydro
'1MAGAT_U02',
'3MKBN_B', # geothermal
'3KAL_G02',
'1BURGOS_G01', # wind
'3KAL_G01',
'1SROQUE_U02',
'1MAGAT_U01',
'3MALAYA_G02',
'3MKBN_D', # geothermal
'1HEDCOR_G01', # hydro
'3TIWI_A',
'3SLPGC_G01',
'1SROQUE_U03',
'1AMBUK_U01',
'3MKBN_E',
'1BINGA_U02',
'1BINGA_U01', # hydro
'1BINGA_U03',
'3BACMAN_G02',
'1BINGA_U04', # hydro
'3MGPP_G01',
'1CAPRIS_G01',
'1AMBUK_U03',
'1IBEC_G01',
'1AMBUK_U02',
'1ANGAT_M',
'1SROQUE_U01', # hydro
'1APEC_G01',
'3AVION_U01',
'3CALIRY_G01', # hydro
'1ANGAT_A',
'3AWOC_G01',
'3AVION_U02',
'1GIFT_G01',
'1BT2020_G01',
'1IPOWER_G01',
'1PNTBNG_U01', # hydro
'1PETSOL_G01',
'1SABANG_G01',
'1PNTBNG_U02', # hydro
'3CALSOL_G01',
'1LIMAY_B',
'1BAUANG_G01',
'2TMO_G03',
'3MEC_G01',
'2TMO_G01',
'1S_ENRO_G01',
'1SUBSOL_G01',
'1NWIND_G02',
'1MASIWA_G01',
'2TMO_G02',
'1NIABAL_G01',
'1NWIND_G01',
'3BBEC_G01',
'1MAEC_G01',
'2TMO_G04',
'1CLASOL_G01',
'1NMHC_G01',
'1MARSOL_G01',
'1PETRON_G01',
'3BOTOCA_G01',
'1BULSOL_G01',
'1RASLAG_G02',
'1SLANGN_G01',
'1RASLAG_G01',
'1LIMAY_A',
'3ORMAT_G01',
'1CABSOL_G01',
'1YHGRN_G01',
'1CIP2_G01',
'2VALSOL_G01',
'2MMPP_G01',
'3KAL_G03',
'1NMHC_G03',
'1SMBELL_G01',
'1ARMSOL_G01',
'2PNGEA_G01',
'1DALSOL_G01',
'1ZAMSOL_G01',
'1BURGOS_G02',
'1SPABUL_G01',
'1BURGOS_G03',
'3ADISOL_G01',
'1BTNSOL_G01',
'3RCBMI_G01',
'3RCBMI_G02',
'3PAGBIL_G02',
'3LIAN_G01',
'1BOSUNG_G01',
'2SMNRTH_G01',
'1ACNPC_G01',
'1GFII_G01',
'1IPOWER_G02',
'3MKBN_C',
'3MALAYA_G01',
'3BART_G01',
'3SLPGC_G03',
'1MSNLO_BATG',
'3HDEPOT_G01',
'1SMC_G03',
'3TIWI_B',
'3SLPGC_G04',
'1SUAL_G02',
'1T_ASIA_G01',
'1UPPC_G01',
'3KAL_G04']

Visayas:

['4LEYTE_A',
'5TPC_G02',
'8PEDC_U03',
'6PAL1A_G01',
'5KSPC_G02',
'5KSPC_G01',
'8PALM_G01',
'5CEDC_U03',
'5CEDC_U01',
'5CEDC_U02',
'8PEDC_U02',
'8PEDC_U01',
'6NASULO_G01',
'4LGPP_G01',
'6HELIOS_G01',
'8PDPP3_G01',
'5CPPC_G01',
'6PAL2A_U01',
'6PAL2A_U02',
'8PWIND_G01',
'5TOLSOL_G01',
'6PAL2A_U03',
'6MANSOL_G01',
'4SEPSOL_G01',
'6CARSOL_G01',
'6SACASL_G02',
'8SUWECO_G01',
'6SACASL_G01',
'5CDPPI_G02',
'5EAUC_G01',
'4PHSOL_G01',
'8PDPP_G01',
'6SACSUN_G01',
'6SLYSOL_G01',
'8SLWIND_G01',
'7BDPP_G01',
'6MNTSOL_G01',
'5TPC_G01',
'5CDPPI_G01',
'7JANOPO_G01',
'6FFHC_G01',
'8GLOBAL_G01',
'7LOBOC_G01',
'8COSMO_G01',
'8STBAR_PB',
'6AMLA_G01',
'8STBAR_PB2',
'6SCBE_G01',
'5PHNPB3_G01',
'8PDPP3_G',
'8CASA_G01',
'8AVON_G01',
'6VMC_G01',
'6URC_G01',
'8PPC_G01',
'6PAL2A_U04',
'8PDPP3_H',
'8PDPP3_D',
'5LBGT_G01',
'6HPCO_G01',
'8GUIM_G01',
'6PAL2A_G01',
'8PDPP3_F',
'8PDPP3_C',
'6CENPRI_U01',
'6CENPRI_U02',
'6CENPRI_U03',
'6CENPRI_U04',
'8PDPP3_E',
'5LBGT_G02']

[systemcatch]

@jarek
Mapped everything I could using your sources and google.
https://gist.github.com/systemcatch/13a1b301471861458bef8a105a6f30a9

[jarek]

Thanks, that's fantastic. I will try to put a parser-and-comparison-engine together. Will report back.

[jarek]

So, this is still on my todo list, but unfortunately I don't yet anything ready to share. I have been collecting live-reported data on and off, but have not processed it or compared to post-reported data yet.

[alixunderplatz]

@jarek here is some more support :)

on https://www.doe.gov.ph/list-existing-power-plants you'll find a list with the installed capacities and also a list with the operating power plants (in case anything is not yet mapped) as of June 2017:

LUZON:

"capacity": {
      "biomass": 87,
      "coal": 5444,
      "gas": 3430,
      "geothermal": 833,
      "hydro": 2538,
      "oil": 2133,
      "solar": 325,
      "wind": 337
    },

VISAYAS:

"capacity": {
      "biomass": 101,
      "coal": 1054,
      "gas": 1,
      "geothermal": 965,
      "hydro": 20,
      "oil": 663,
      "solar": 459,
      "wind": 90
    },

MINDANAO:

"capacity": {
      "biomass": 36,
      "coal": 1070,
      "gas": 0,
      "geothermal": 108,
      "hydro": 1080,
      "oil": 788,
      "solar": 59,
      "wind": 0
    },

PHILIPPINES TOTAL:

"capacity": {
      "biomass": 224,
      "coal": 7569,
      "gas": 3431,
      "geothermal": 1906,
      "hydro": 3637,
      "oil": 3584,
      "solar": 843,
      "wind": 427
    },

[jarek]

Hello,

I dug into this a little bit and unfortunately have no good news.

At least for Visayas on 2018-03-02, data reported in real-time in the pie charts on WESM front page was substantially different from data reported afterwards in WESM's "downloads"/reporting section.

Sources of information

• scraped demand: http://wesm.ph/system_lwap_graphs.php as indicated above - saved on the day of (at 2018-03-02T15:17 CET, to be exact)
• scraped generation aka "pie chart" data aka "chart" data - saved on the day of from http://wesm.ph/visayas_fuel_participants.php?hour=1 (modifying the hour)
• MPS RTX aka "reported actual" - exported today via http://wesm.ph/inner.php/downloads/market_prices_&_schedules for "Real Time Ex-Post", resource "Generation", region "Visayas". Similarly MPS RTD aka "reported scheduled", exported for "Real Time Ex-Ante". These were then mapped to fuels using @systemcatch's mapping provided above
• HVDC aka the interlink between Luzon and Visayas - exported today via http://wesm.ph/inner.php/downloads/hvdc_schedule, "Real Time Ex-Post" - contrary to my experiences before, day-after reported data is now available, and matches reports in PDS (but not really NDS forecasts)

For interested, data in CSV: https://gist.github.com/jarek/e8b2c15321da87f754530c39921badbf - data I collected is near "scraped: Vis demand" and "generation total", and then "chart biomass", "chart coal", etc

The graphs go up to 21st hour because I didn't get later "chart" data (also because data for 23rd hour is very hard to get because of how the data rolls over).

Biggest problems

1) generation figures don't really line up with anything;
2) coal and geothermal are missing generation in "chart" data vs MPS.

Supply vs demand

• MPS RTX and MPS RTD total generation line up decently well
• scraped demand is sometimes significantly lower than MPS generation figures, but in the evening it's much closer
• "sum chart": sum of generation from different fuels as scraped on the day of is significantly lower than what was subsequently reported in MPS, and also different from demand
• "sum chart + HVDC" to account for imports - still different from demand

Reported generation vs MPS RTX and RTD

Coal and geothermal, the two biggest fuels, differ significantly in what was reported on the hour vs what was reported later. Solar is good and oil tracks fairly well as well.

Here is how much coal and geothermal are missing. It's unfortunately not static over the day, either...

Bad news summary

I can't find a way to make sense of the Visayas 2018-03-02 data. I will try Luzon and maybe another day to see if that looks better. If not, I might try emailing them asking what's up...

[corradio]

Thanks for investigating @jarek. Maybe the mismatch will be fixed in the future.
Worth keeping an eye out for changes.

[alixunderplatz]

I created the "historical data" label for zones with available generation data for the past. 🏷

Isn't it possible to use the real-time mix and fill the gap between the generation per type and the demand with "unknown" (like for Kyushu/Japan and for NL)?
This unknown share today makes up between 2-2.5 GW of a total load between 8-10 GW today, so it's always around 25%. In my eyes that's tolerable.
The seperate views for Luzon and Visayas don't seem to work at the moment, but the total system view does.

In a second step you could fill up or overwrite it with the ex-post data of the per-unit generation. Is this an option?

[jarek]

I have to admit I haven't checked on this in a while and I never contacted WESM about the inconsistencies. I will try to look again and see if the data matches up now.

[jarek]

Per http://wesm.ph/inner.php/news/news_and_events/574, WESM and Philippine Electricity Market Corporation might have created or activated a new body called Independent Electricity Market Operator of the Philippines with website http://www.iemop.ph/ which might possibly have with more or different data. They seem to have generation mix for Mindanao now, for one thing. I have not yet checked if the values sum up to the reported demand, or match post-reported values.

[q--]

The data on WESM.ph seems to be no longer available, but IEMOP.ph looks very promising. In fact, it looks like IEMOP is the old WESM site; you can replace http://wesm.ph with http://iemop.ph to get the links working again, though the data is no longer available

http://www.iemop.ph/ also has hourly demand per region at the top of the page -

RTD | Date: Sat 11 Jan 2020 Hour: [ 17:00 H ]

so that's really old and not useful.

However, via here I found http://180.232.127.40/views/IEMOP_SITE_SUMM1/STORY1?iframeSizedToWindow=true&:embed=y&:showAppBanner=false&:display_count=no&:showVizHome=no, which has hourly supply/demand data (where supply means 'available capacity'), as well as generation mix (button at the top of the page). This page seems to be updated hourly as well. (I checked at two different times on the same day and the values had changed in the mean time.)

Note: given that the generation mix page reports 2.5% solar despite it being night there, I think this is the daily generation mix so far - but if electricitymap does fetch this data each hour, because demand data is also available, it should be possible to do the math to figure out the generation mix per hour.

[martincollignon]

@q-- at this point I expect you to find real-time generation data for China in the coming days 😂Amazing detective work!

[willbeaufoy]

The source @q-- mentioned also has hourly data for generation mix in theory, though right now the smallest time period I can get actual data for is 6 hours. I will check again tomorrow in case this is a temporary failure for hourly data.

[willbeaufoy]

The source @q-- mentioned also has hourly data for generation mix in theory, though right now the smallest time period I can get actual data for is 6 hours. I will check again tomorrow in case this is a temporary failure for hourly data.

Nope still at 6 hours minimum unfortunately.

[q--]

Hmm, I don't get any results for generation mix unless I set the "DELIVERY_DATE" picker to "last 23 hours" or more.

Are you using the same control I am?

[willbeaufoy]

Hmm, I don't get any results for generation mix unless I set the "DELIVERY_DATE" picker to "last 23 hours" or more.

Are you using the same control I am?

It looks like it varies, as now I don't get anything even for 23 hours.