Pvlib-python: NREL TMY server rejecting python requests

Created on 25 Jun 2018 · 6Comments · Source: pvlib/pvlib-python

Describe the bug
NREL's TMY file server recently started rejecting requests originating from urlopen. The response was:

"<html><head><title>Request Rejected</title></head><body>An error has occurred. Please consult with your administrator.<br><br>Your support ID is: 12437308730316847769<br><br><a href='javascript:history.back();'>[Go Back]</a></body></html>"

However, it's still possible to download a file from the same url with a web browser. Mostly likely they, intentionally or not, started blocking any requests that don't come with a browser-like HTTP User-Agent header.

I have a fix mostly ready to go.

To Reproduce

pvlib/test/test_tmy.py F
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

    @network
    def test_readtmy3_remote():
        url = 'http://rredc.nrel.gov/solar/old_data/nsrdb/1991-2005/data/tmy3/703165TYA.CSV'
>       tmy.readtmy3(url)

pvlib/test/test_tmy.py:19:

        # read in file metadata
        meta = dict(zip(head, csvdata.readline().rstrip('\n').split(",")))

        # convert metadata strings to numeric types
>       meta['altitude'] = float(meta['altitude'])
E       KeyError: 'altitude'

pvlib/tmy.py:178: KeyError

Expected behavior
Should work like it used to!

Versions:
all

bug

Source

wholmgren

All 6 comments

@wholmgren and @cwhanse unfortunately it looks like NREL has permanently retired the old rredc websites, however I think you can download a single CSV from MTS2: