Electricitymap-contrib: US-SVERI data portal is timing out

[systemcatch]

This has been going on for at least a week now. The link times out in my browser as well, I think the site is offline rather than us being blocked. I've sent an email to Arizona University and will keep chasing them until I get a response.

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 141, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 83, in create_connection
    raise err
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 284, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 150, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f672a4a55f8>: Failed to establish a new connection: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='sveri.energy.arizona.edu', port=443): Max retries exceeded with url: /api?ids=1,2,4,5,6,7,8,16&saveData=true&startDate=2018-07-25&endDate=2018-07-26 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f672a4a55f8>: Failed to establish a new connection: [Errno 110] Connection timed out',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/feeder/lib/fetch_data.py", line 131, in launch_parsers
    **parser_kwargs)
  File "/home/contrib/parsers/US_SVERI.py", line 121, in fetch_production
    raw_data = query_api(limits, session=session)
  File "/home/contrib/parsers/US_SVERI.py", line 39, in query_api
    data_req = s.get(url)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 508, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='sveri.energy.arizona.edu', port=443): Max retries exceeded with url: /api?ids=1,2,4,5,6,7,8,16&saveData=true&startDate=2018-07-25&endDate=2018-07-26 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f672a4a55f8>: Failed to establish a new connection: [Errno 110] Connection timed out',))

[systemcatch]

@corradio
Received a response. Very disappointing news, looks like the SVERI initiative is shutdown.

Hi Chris,

The SVERI project has not been funded for the last couple of years and we’ve run out of resources to keep the site operating. Many of the data feeds from the utilities had gone bad (missing or erroneous data), so the overall data quality was poor. It’s possible that it will resurrected in the future.

Regards,

--
William Holmgren
Assistant Research Professor
Department of Hydrology and Atmospheric Sciences
Institute for Energy Solutions
University of Arizona
Environment and Natural Resources 2, Room 514N
520 626 2911
https://forecasting.energy.arizona.edu

[corradio]

I'm thinking we could raise awareness on our Twitter account about this issue.
What do you think @ovbm ?

[jarek]

I will close this issue as it appears that SVERI is not coming back :(