Pandas-datareader: Unable to fetch Google finance data

Created on 16 Sep 2017 · 13Comments · Source: pydata/pandas-datareader

Google has changed their URL from "http://www.google.com/finance/info" to "https://finance.google.com/finance"

Source

nasirali1

Most helpful comment

Temporary and very ugly solution for testing purposes:

from pandas_datareader.google.daily import GoogleDailyReader

@property
def url(self):
    return 'http://finance.google.com/finance/historical'

GoogleDailyReader.url = url

# get data

import pandas_datareader as pdr
from datetime import datetime

start = datetime(2010,1,1)
end = datetime(2014,1,1)
ret = pdr.get_data_google(['AAPL'], start, end)

minimalgeek on 18 Sep 2017

👍5

All 13 comments

OK, please tell us if and where this breaks the API.

What are the error messages with respect to pandas-datareader?

[ ] Follow-up in near-duplicate issue #391

rsvp on 16 Sep 2017

Hi rsvp,
Thanks for a prompt reply. Please find below the python code and error trace:

CODE:
def finance_date():
"""
Stock indicies for
.INX = S&P 500
.DJI = Dow Jones
.IXIC = NASDAQ
"""
nds = web.get_quote_google(['.IXIC', '.DJI', '.INX'])

ERROR TRACE:
Traceback (most recent call last):
File "/home/ali/IdeaProjects/Environmental-Data-Collection/core/finance/nds_stock.py", line 55, in
finance_date()
File "/home/ali/IdeaProjects/Environmental-Data-Collection/core/finance/nds_stock.py", line 40, in finance_date
nds = web.get_quote_google(['.IXIC', '.DJI', '.INX'])
File "/home/ali/IdeaProjects/Environmental-Data-Collection/venv/lib/python3.5/site-packages/pandas_datareader/data.py", line 56, in get_quote_google
return GoogleQuotesReader(args, *kwargs).read()
File "/home/ali/IdeaProjects/Environmental-Data-Collection/venv/lib/python3.5/site-packages/pandas_datareader/base.py", line 72, in read
return self._read_one_data(self.url, self.params)
File "/home/ali/IdeaProjects/Environmental-Data-Collection/venv/lib/python3.5/site-packages/pandas_datareader/base.py", line 79, in _read_one_data
out = self._read_url_as_StringIO(url, params=params)
File "/home/ali/IdeaProjects/Environmental-Data-Collection/venv/lib/python3.5/site-packages/pandas_datareader/base.py", line 90, in _read_url_as_StringIO
response = self._get_response(url, params=params)
File "/home/ali/IdeaProjects/Environmental-Data-Collection/venv/lib/python3.5/site-packages/pandas_datareader/base.py", line 139, in _get_response
raise RemoteDataError('Unable to read URL: {0}'.format(url))
pandas_datareader._utils.RemoteDataError: Unable to read URL: http://www.google.com/finance/info?q=.IXIC%2C.DJI%2C.INX

nasirali1 on 16 Sep 2017

👍1

Maybe it makes sense to contact Google to ask for a correction, but I do not think that the issue affects only a few.

paintdog on 16 Sep 2017

paindog, Google has changed/restructured their finance urls. It's not a mistake that we can ask Google to fix.

nasirali1 on 16 Sep 2017

They reduced the available data to the period of about one year. That's a problem they could fix...

paintdog on 16 Sep 2017

paintdog, I think, this is a simple URL update in pandas-datareader API. Current URL used in api is "http://www.google.com/finance/info". This URL should be updated to new Google finance URL: "https://finance.google.com/finance".

API code snippet (data.py)
class GoogleQuotesReader(_BaseReader):

"""Get current google quote"""

@property
def url(self):
    return 'http://www.google.com/finance/info'

nasirali1 on 16 Sep 2017

👍2

Temporary and very ugly solution for testing purposes:

from pandas_datareader.google.daily import GoogleDailyReader

@property
def url(self):
    return 'http://finance.google.com/finance/historical'

GoogleDailyReader.url = url

# get data

import pandas_datareader as pdr
from datetime import datetime

start = datetime(2010,1,1)
end = datetime(2014,1,1)
ret = pdr.get_data_google(['AAPL'], start, end)

minimalgeek on 18 Sep 2017

👍5

Changing the URL works, but the 302 redirect that www.google.com/finance/historical returns will currently strip the startdate= and enddate= params, which is why people are seeing only the past year's data being returned.

Unintentional bug in the www.google.com redirect rule at play?

sof on 19 Sep 2017

👍1

the commit added by soon fixed the historical collection, however the delayed quote is still not functional, I've tried http://finance.google.com/finance with no success

TyrelCB on 26 Sep 2017

👍2

@TyrelCB The latest PR by @soon has failed Travis build: https://github.com/pydata/pandas-datareader/pull/402

rsvp on 27 Sep 2017

Google Finance under renovation, portfolios to be deprecated mid-November 2017 : https://github.com/rsvp/fecon235/issues/7#issuecomment-332572738

rsvp on 27 Sep 2017

This fix may be a waste of effort. It looks like there are big holes in the underlying datasets on Google Finance, eg Bank of America (BAC) or JP Morgan (JPM) have nothing or garbage prices (as of the date I am posting).