Google-cloud-python: Resource Manager: 403 creating new project with service-account

[adamdavis40208]

Environment details

1. OSx 10.14.4
2. 3.7.3
3. Name: google-cloud-resource-manager
   Version: 0.28.3

Steps to reproduce

This code 403s (using a service account):

    resource_manager_client = resource_manager.Client()
    new_project = resource_manager_client.new_project(project_id, name=project_name)
    new_project.create()

Error: The caller does not have permission

Code using googleapiclient works, if I pass parent:

    credentials = GoogleCredentials.get_application_default()
    service = googleapiclient.discovery.build('cloudresourcemanager', 'v1', credentials=credentials)
    project_body = {
        'name': project_name,
        'projectId': project_id,
        'parent': {"type": "organization", "id": org_id}
    }
    request = service.projects().create(body=project_body)
    request.execute()

One interesting note: when I leave parent off of googleapiclient, I also get a 403, but I also get a note saying that I must specify parent if I'm a service account.

The default account is my service account, and I'm able to create projects via the gcloud cli:

gcloud projects create x234-delete-me \ 
  --organization {redacted}
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/x234-delete-me].
Waiting for [operations/cp.{redacted}] to finish...done.

[tseaver]

@adamdavis40208 Thanks for the report! Duplicates #6039.

[adamdavis40208]

The issue has been open for 7 months -- I'd recommend dropping a note somewhere on the readme, as that issue doesn't scream "stuff doesn't work right now!" to someone new to the project.

Are there any other areas of the resource manager code that won't work until #6039 is fixed?

[rsimac]

While not completely sure if it belongs here, but rather than opening new and potentially dupe issue, here goes...

It may be beneficial to update readme.rst and reword the reasoning behind the inability to create projects using service accounts for it may mislead developers on underlying (rest) api limitations.

It appears the service accounts are very much capable of listing, updating, deleting and creating the projects and are by no means limited in that aspect.

Bit of code to demonstrate the perfectly functional project create using service account client, without any gcloud command line authentication. Pure python code.

def test_resource_manager_2():
    '''
    per    https://github.com/googleapis/google-cloud-python/issues/2539

    IT WORKS
    '''

    from googleapiclient import discovery

    logger.info("Logging in as service account using json credentials from file")
    credentials = service_account.Credentials.from_service_account_file(filename=keyfile)


    #using latest and greatest google rest apis
    service = discovery.build('cloudresourcemanager', 'v1', credentials=credentials)

    now = datetime.datetime.now()

    projectId =  "simacr-proto-"+now.strftime('%Y%m%d%H%M%S')

    #specifying project name, id and parent folder
    project_body = {
             'name': projectId,
             'projectId': projectId,
             'parent': {'type':'folder', 'id':'1071056422573'}        
         }

    try:
        #creating the request
        request = service.projects().create(body=project_body)

        #executing the request
        request.execute()

        #checking on new project
        arewethereyet = False
        while not arewethereyet:
            request = service.projects().get(projectId=projectId)
            try:
                newproj = request.execute()
                logger.info("New project created: {}".format(newproj))
                arewethereyet = True
            except Exception as e:
                logger.warning("Google cloud replying error but actually not yet created the project... False error: {}".format(e))
                time.sleep(2)


    except:
        logger.exception("Failed creating the project")




    try:
        logger.info("Deleting new project")
        request = service.projects().delete(projectId=projectId)
        request.execute()

        #check the project status
        request = service.projects().get(projectId=projectId)
        newproj = request.execute()

        logger.info("New project in deleting status: {}".format(newproj))

    except:
        logger.exception("Failed deleting the project")



    return

Log snippet reflecting above code (do please ignore initial warning for it is irrelevant for this case). Also ignore the interesting error returned from google just after the project was 'created' but before the google actually got fully 'aware' of it...

INFO:root:Logging in as service account using json credentials from file
WARNING:googleapiclient.discovery_cache:file_cache is unavailable when using oauth2client >= 4.0.0
Traceback (most recent call last):
  File "C:\mytools\python27\lib\site-packages\googleapiclient\discovery_cache\__init__.py", line 41, in autodetect
    from . import file_cache
  File "C:\mytools\python27\lib\site-packages\googleapiclient\discovery_cache\file_cache.py", line 41, in <module>
    'file_cache is unavailable when using oauth2client >= 4.0.0')
ImportError: file_cache is unavailable when using oauth2client >= 4.0.0
INFO:googleapiclient.discovery:URL being requested: GET https://www.googleapis.com/discovery/v1/apis/cloudresourcemanager/v1/rest
INFO:googleapiclient.discovery:URL being requested: POST https://cloudresourcemanager.googleapis.com/v1/projects?alt=json
DEBUG:google_auth_httplib2:Making request: POST https://oauth2.googleapis.com/token
INFO:googleapiclient.discovery:URL being requested: GET https://cloudresourcemanager.googleapis.com/v1/projects/simacr-proto-20190624165923?alt=json
WARNING:googleapiclient.http:Encountered 403 Forbidden with reason "forbidden"
WARNING:root:Google cloud replying error but actually not yet created the project... False error: <HttpError 403 when requesting https://cloudresourcemanager.googleapis.com/v1/projects/simacr-proto-20190624165923?alt=json returned "The caller does not have permission">
INFO:googleapiclient.discovery:URL being requested: GET https://cloudresourcemanager.googleapis.com/v1/projects/simacr-proto-20190624165923?alt=json
WARNING:root:Google cloud replying error but actually not yet created the project... False error: <HttpError 404 when requesting https://cloudresourcemanager.googleapis.com/v1/projects/simacr-proto-20190624165923?alt=json returned "Requested entity was not found.">
INFO:googleapiclient.discovery:URL being requested: GET https://cloudresourcemanager.googleapis.com/v1/projects/simacr-proto-20190624165923?alt=json
INFO:root:New project created: {u'name': u'simacr-proto-20190624165923', u'parent': {u'type': u'folder', u'id': u'1071056422573'}, u'projectId': u'simacr-proto-20190624165923', u'projectNumber': u'448915697971', u'lifecycleState': u'ACTIVE', u'createTime': u'2019-06-24T20:59:21.525Z'}
INFO:root:Deleting new project
INFO:googleapiclient.discovery:URL being requested: DELETE https://cloudresourcemanager.googleapis.com/v1/projects/simacr-proto-20190624165923?alt=json
INFO:googleapiclient.discovery:URL being requested: GET https://cloudresourcemanager.googleapis.com/v1/projects/simacr-proto-20190624165923?alt=json
INFO:root:New project in deleting status: {u'name': u'simacr-proto-20190624165923', u'parent': {u'type': u'folder', u'id': u'1071056422573'}, u'projectId': u'simacr-proto-20190624165923', u'projectNumber': u'448915697971', u'lifecycleState': u'DELETE_REQUESTED', u'createTime': u'2019-06-24T20:59:21.525Z'}
Done prototyping