Google-cloud-python: BigQuery: to_dataframe function not working

[VikramTiwari]

python: 3.6.3
google-cloud: 0.32.0

when I am trying to run a code based on this test (https://github.com/GoogleCloudPlatform/google-cloud-python/blob/017714cf91a310caafc60181c044d7812c7226b7/bigquery/tests/system.py#L1259) on my side, I am getting the error:

'HTTPIterator' object has no attribute 'to_dataframe'

function call that I am using

df = client.query(query).result().to_dataframe()

[chemelnucfin]

Hello,
I tried running that code and I get these objects. It seems to me google cloud gives the latest google-cloud-bigquery, but what is your version of google-cloud-bigquery?

(Pdb) p Config.CLIENT.query(QUERY).result()
<google.cloud.bigquery.table.RowIterator object at 0x7f81c41869e8>
(Pdb) p Config.CLIENT.query(QUERY)
<google.cloud.bigquery.job.QueryJob object at 0x7f81cab50c88>
(Pdb) p Config.CLIENT
<google.cloud.bigquery.client.Client object at 0x7f81c4186710>
(Pdb) p Config
<class 'tests.system.Config'>

[chemelnucfin]

From looking through the code, it does seem like that each of those return those objects and it should return a RowIterator. I believe that you might be running an older version of BigQuery.

[VikramTiwari]

My sample code:

def get_result_dataframe(query):
    try:
        df = client.query(query).result().to_dataframe()
        return df
    except Exception as e:
        print('Error:', e)

print(get_result_dataframe('SELECT sum(attendance), startTime  FROM `bigquery-public-data.baseball.schedules` group by startTime'))

Run log:

Error: 'HTTPIterator' object has no attribute 'to_dataframe'
None

Package info:

$ pip show google-cloud
Name: google-cloud
Version: 0.32.0
Summary: API Client library for Google Cloud
Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
Author: Google Cloud Platform
Author-email: [email protected]
License: Apache 2.0
Location: /Users/Vikram/anaconda3/lib/python3.6/site-packages
Requires: google-cloud-spanner, google-cloud-translate, google-cloud-logging, google-api-core, google-cloud-monitoring, google-cloud-datastore, google-cloud-pubsub, google-cloud-vision, google-cloud-dns, google-cloud-runtimeconfig, google-cloud-firestore, google-cloud-error-reporting, google-cloud-bigquery, google-cloud-bigtable, google-cloud-language, google-cloud-container, google-cloud-videointelligence, google-cloud-trace, google-cloud-storage, google-cloud-speech, google-cloud-resource-manager, google-cloud-bigquery-datatransfer, google-cloud-core

$ pip show google-cloud-bigquery
Name: google-cloud-bigquery
Version: 0.28.0
Summary: Python Client for Google BigQuery
Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
Author: Google Cloud Platform
Author-email: [email protected]
License: Apache 2.0
Location: /Users/Vikram/anaconda3/lib/python3.6/site-packages
Requires: google-cloud-core, google-auth, google-resumable-media, google-api-core, requests

OS: OSX 10.13.1

[alixhami]

Hi @VikramTiwari sorry for the confusion, but the to_dataframe function has not been released yet in the current latest version.

[VikramTiwari]

Thanks @alixhami ! Will wait for the new release 👍

[alixhami]

Hi @VikramTiwari - google.cloud.bigquery version 0.29.0 includes the to_dataframe function. Let me know if need any more help with your code after upgrading!

[VikramTiwari]

Hey, @alixhami This is working great in 0.29.0 Thanks for all the work! 👍

PS: Didn't wanna open a new issue. Is there an example somewhere about how to create a date-partitioned table?

[VikramTiwari]

Never mind. Got it working. Closing the issue. 🥇

[alixhami]

Awesome, I'm glad it works!