Google-cloud-python: Listing scheduled queries in bigquery_datatransfer 0.2.0

[spil-josko]

With bigquery_datatransfer 0.1.1, the previous version, my script was successfully accessing our google bigquery project and downloading the scheduled query from it. (I scheduled SELECT 'test' for this test).

The only problem was that I could not use other service accounts for our other projects. With 0.2.0 I can use the from_service_account_json factory method to use the other service accounts. But in version 0.2.0 I can no longer download any scheduled queries:

    logger.info("Getting client information for {project_name} from secret {secret}".format(
        project_name=project_name,
        secret=service_account_json_path)) 

    client = datatransfer.DataTransferServiceClient.from_service_account_json(
            service_account_json_path) 

    parent = client.project_path(project_name) 
    transfers = client.list_transfer_configs(parent)

    logger.info("Found {num} scheduled queries for {project}".format(
        num=transfers.num_results, project=parent))

The project_path rather than location_path call was one change I had to make. The above code does not generate any errors, it just outputs that there are 0 queries found.

Am I doing something wrong? Are scheduled queries no longer supported?

[tseaver]

@spil-josko Thanks for the report! It looks to me as though the BigQuery Transfer API can store transfer configurations either by project or by project and location.

google-cloud-bigquery-datatransfer 0.1.1 seems to have expected the second mode (the docstrings all use client.location_path to compute the parent), while google-cloud-bigquery-datatransfer 0.2 seems to expect only the first: it has even dropped the location-specific path helpers.

From my reading of the API docs, I believe both modes should be supported. @tswast, @shollyman can you please confirm? Checking. Hmm, it looks as though the location-specific variants are only present in the REST version of the API. The fact that @spil-josko can still use the 0.1.1 version of the library seems to indicate that location-specific bits are actually supported.

@spil-josko In the meanwhile, you might be able to work around the issue by computing the location path yourself, e.g.:

    parent = '{}/locations/{}'.format(
        client.project_path(project_name), location)
    transfers = client.list_transfer_configs(parent)

[tswast]

When I first wrote the samples for the data transfer API both with location and without location were supported. I believe with location should be prefered (0.1.1 behavior).

[tseaver]

@tswast I had tagged this as a "fixit" docs issue, but don't believe it should be so: I don't know what intended usage we should be documenting.

[sgreenberg]

Hi @spil-josko, I'm marking this as closed since it is an old issue and since the library is now on v 0.5. Please reopen if this is still causing you trouble.