Electricitymap-contrib: Data for Israel
Another one from me this weekend :)
Israel has data for the "system load", which is updated every few minutes. This is also equivalent to "system production", because there are no imports (only exports are shown on IEA's statistics)
This url https://www.iec.co.il/_layouts/iec/applicationpages/lackmanagment.aspx leads to this data table:
I thought from the html of the page, it would be easy to get the desired "statusVal":
<span id="lblActualSupply"><div class="IecContentTitle">爪专讬讻转 讛讞砖诪诇 讻注转 讘诪讙讜讜"讟</div><span class="statusVal">8,067</span></span>
But here is a part of what I get back as response instead:
<Response [200]>
<html>
<head>
<meta charset="utf-8"/>
<script>
window.rbzid="F1MPV4aQjOIsxeqssncm3guiYZhNzK1pbkx9JHjc36L9ok7CIWB5gwQE01ukWP8TcKv8MwNyWRltsisX49d+Ty1Znd7yQzLgKv2NkuVd6ew=";
u82222.O=function(x){return x;};u82222.E=function (){return typeof u82222.t.u==='function'?
...
...
...
E(21)]){}else{location[u82222.f(u82222.r(11)+u82222.x(6)+u82222.e(17)+u82222.N(0),u82222.e(2))]();}
</script>
</head>
<body>
</body>
</html>
So looks like it actually returns some sort of script.
I need some help from you folks to create the parser ;)
Could somebody please tell me how to get the desired part from the html of the website using BeautifulSoup or sth similar? Thanks! :)
alixunderplatz
All 2 comments
It looks like the first time we make the request, you get the weird obfuscated thing, but also a cookie. When you redo the request with the cookie, you get plain HTML.
>>> import requests
>>> first = requests.get('https://www.iec.co.il/_layouts/iec/applicationpages/lackmanagment.aspx')
>>> first.text[:200]
'<html><head><meta charset="utf-8"><script>\n window.rbzid="J36FVmfMyCuT4whUvFe57EfnnF/zvwbiIetiFbCG8ciKfDDVxa7AjytlugMJTM7CP9DP0OffLsOjblPYcwaJ11t4AmdUyCK48muUVE00o2k=";\n u82222.O=functio'
>>> first.cookies
<RequestsCookieJar[Cookie(version=0, name='rbzid', value='J36FVmfMyCuT4whUvFe57EfnnF/zvwbiIetiFbCG8ciKfDDVxa7AjytlugMJTM7CP9DP0OffLsOjblPYcwaJ11t4AmdUyCK48muUVE00o2k=', port=None, port_specified=False, domain='.iec.co.il', domain_specified=True, domain_initial_dot=True, path='/', path_specified=True, secure=False, expires=None, discard=True, comment=None, comment_url=None, rest={'HttpOnly': None}, rfc2109=False), Cookie(version=0, name='GCLB', value='CPuawNfln7q5Gg', port=None, port_specified=False, domain='www.iec.co.il', domain_specified=False, domain_initial_dot=False, path='/', path_specified=True, secure=False, expires=None, discard=True, comment=None, comment_url=None, rest={'HttpOnly': None}, rfc2109=False)]>
>>> second = requests.get('https://www.iec.co.il/_layouts/iec/applicationpages/lackmanagment.aspx', cookies=first.cookies)
>>> second.text
'\r\n\r\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> \r\n<html xmlns="http://www.w3.org/1999/xhtml">\r\n<head><title>\r\n\t讞讘专转 讛讞砖诪诇 诇讬砖专讗诇\r\n</title><link id="cssLink" rel="stylesheet" type="text/css" href="/Style Library/Iec/he-IL/LackManagment.css" />\r\n <script type="text/javascript">\r\n </script>\r\n</head>\r\n<body style="margin:0;" aria-owns="miflas-heading IecStatus">\r\n <dl id="IecStatus" class="iecStatus" role="list">\r\n\t\t\t<dt id="miflas-heading" role="heading" aria-level="4">\r\n 诪驻诇住 讛讞砖诪诇\r\n </dt>\r\n\t\t\t<dd class="current" role="listitem">\r\n\t\t\t\t<h6 role="presentation"> \r\n <span id="lblActualSupply"><div class=\'IecContentTitle\'>爪专讬讻转 讛讞砖诪诇 讻注转 讘诪讙讜讜"讟</div><span class=\'statusVal\'>8,290</span></span> \r\n </h6>\r\n\t\t </dd>\r\n\t\t\t<dd class="pic" role="listitem">\r\n\t\t\t\t<h6 role="presentation">\r\n <span id="lblHours"><div class=\'IecContentTitle\'>砖注转 讛砖讬讗</div><span class=\'statusVal\'>18:00</span></span>\r\n\t\t\t\t</h6>\r\n\t\t\t</dd>\r\n\t\t\t<dd class="resCurrent" role="listitem"> \r\n <img id="imgBattery" class="resCurrentPic" src="/Style%20Library/Iec/he-il/Images/LackManagment/BatteryLevel_1000.png" border="0" />\r\n\t\t\t\t<h6 role="presentation"> \r\n <span id="lblCurrentSupplyTitle">专讝专讘转 讛讞砖诪诇 讻注转 讘诪讙讜讜"讟</span>\r\n <span id="lblCurrentSupply"><span class=\'statusVal\'>2,314</span></span>\r\n </h6>\r\n\t\t\t</dd>\r\n\t\t\t<dd class="picRes" role="listitem">\r\n\t\t\t\t<h6 role="presentation"> \r\n <span id="lblSupply"><div class=\'IecContentTitle\'>专讝专讘转 讛讞砖诪诇 讘砖讬讗 讘诪讙讜讜"讟</div><span class=\'statusVal\'>1,611</span></span>\r\n </h6>\r\n\t\t\t</dd>\r\n\t\t</dl>\r\n \r\n\r\n <span id="miflas">2</span> \r\n \r\n <script type="text/javascript" >\r\n setTimeout("location.reload(true);", 120000);\r\n </script>\r\n \r\n </body>\r\n</html>\r\n'
jarek
on 24 Nov 2019
Looks like the breakdown of capacity (https://en.wikipedia.org/wiki/Energy_in_Israel#Electricity) is heavily fossil fuel based with little renewable energy. Since there won't be any exports/imports we can estimate the CO2 intensity.
systemcatch
on 12 Dec 2019
Related issues
pascalheraud
路
4Comments
corradio
路
5Comments
systemcatch
路
4Comments
wojciej
路
3Comments
consideRatio
路
4Comments
Most helpful comment
It looks like the first time we make the request, you get the weird obfuscated thing, but also a cookie. When you redo the request with the cookie, you get plain HTML.