Terraform-provider-google: Unable to create StackDriver workspace for project

[aeneasr]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment
• If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.

Terraform Version

v0.11.10

Affected Resource(s)

• google_monitoring_alert_policy

Terraform Configuration Files

resource "google_project_services" "project" {
  depends_on = ["google_project.project"]

  project = "${google_project.project.project_id}"

  services = [
    // ...
    "logging.googleapis.com",
    "monitoring.googleapis.com",
    "stackdriver.googleapis.com",
  ]
}

resource "google_monitoring_alert_policy" "cluster" {
  project = "${var.google_project_id}"

  display_name = "ORY GKE Cluster Alert Policy"
  combiner = "OR"
  enabled = true
  conditions = [
    {
      display_name = "test condition"
      condition_threshold {
        filter = "resource.type=\"gke_cluster\" AND severety>=\"WARNING\""
        duration = "60s"
        comparison = "COMPARISON_GT"
        threshold_value = 0
      }
    }
  ]
}

Debug Output

Error: Error applying plan:

1 error(s) occurred:

* module.backoffice.google_monitoring_alert_policy.cluster: 1 error(s) occurred:
* google_monitoring_alert_policy.cluster: Error creating AlertPolicy: googleapi: Error 400: 'projects/<REDACTED>' is not a Stackdriver account project.

Expected Behavior

It should be possible to link the project to a StackDriver account or use google_monitoring_alert_policy right away.

Actual Behavior

I was unable to find any resources on how to link the project with the stackdriver account. It should either be documented how to achieve that (happy to do so after I know how to do it :) ) or added as a resource(?) or setting.

Steps to Reproduce

I have not tested it yet, but a minimal reproduction case is probably to create a new project and try to set up a monitoring alert right away.

[danawillow]

Hi @aeneasr, can you try having the alert policy depend on the project services resource? The current config looks like it'll run the two of them in parallel. Let me know how that goes! If it still doesn't work, can you post a gist of debug logs? https://www.terraform.io/docs/internals/debugging.html. Thanks!!

[aeneasr]

Hi @danawillow , thank you for the response! The dependence should be implicit because the project and project_services is being created in a different module (module gcp) than google_monitoring_alert_policy (module some_kubernetes_workspace). gcp declares the project ID as output and some_kubernetes_workspace uses the project ID as input to - among others - link the alert policy to the project. I'll set up a log gist now.

[aeneasr]

@danawillow would it be possible to send you the logs privately? I think that all sensitive info (API Keys, OAuth2 Client Secrets, ...) is locked away in k8s secrets but some things popped up in the trace/logs so I feel uncomfortable with sharing that publicly.

[danawillow]

Sure, that's fine. first name last name at google dot com. In the meantime, @chrisst is there anything that needs to be done to enable stackdriver besides enabling the API?

[chrisst]

I believe that in order to use stack driver for the first time you will need to enable a stack driver "workspace" which is loosely associated with a GCP project. I was hoping to deliver that but it looks like the api to create the workspace is still in development right now :(
So for now there is still a manual step of going to https://app.google.stackdriver.com/accounts and adding your workspace.

When they release the api that supports this I'll get it added in!

[aeneasr]

Oh I see, that's what I kinda feared. Thank you for the update. @danawillow I'll refrain from sharing the trace with you then.

[jaceq]

@chrisst Is there any separate github issue for building this API ? I wonder what kind of ETA are we talking about here... ?

[chrisst]

@jaceq Unfortunately the stack driver codebase and apis are closed source and so there isn't a good place for the community to see the feature development. I've been told by the team that they are working on it but I also don't have insight as to when it will be available.

[jaceq]

@chrisst Thank you for clarification. That's a bit of shame to be hanging in the dark.

[spmsaj]

I believe that in order to use stack driver for the first time you will need to enable a stack driver "workspace" which is loosely associated with a GCP project. I was hoping to deliver that but it looks like the api to create the workspace is still in development right now :(
So for now there is still a manual step of going to https://app.google.stackdriver.com/accounts and adding your workspace.

When they release the api that supports this I'll get it added in!

Hi, does this mean the api to create workspace is still not there from a GCP standpoint? And Terraform integration is waiting on that?

[rsalmond]

@spmsaj that is what I understand to be the case yes.

[ocervell]

API is now in alpha, but projects have to be whitelisted. We can work on the stackdriver workspace resource once it goes beta (undetermined ETA).

[jfotopoulos]

Hi.. any updates on this issue?

[jaceq]

@ocervell given it's almost half a year later... is this in beta finally?

[JessicaGreben]

I'm digging around the GCP monitoring API docs and I still don't see API support for creating stackdriver workspaces. Sad. I'm also asking in the GCP slack stackdriver channel about a timeline, but no word yet.

[jamesmoriarty]

@danawillow any updates from your side?

[danawillow]

No updates on a date. If you're part of the alpha, you can ask your GCP rep to contact our team on your behalf for support.

[ghost]

For the record, I'm also having the same issue. Had to create the workspace manually in StackDriver to be able to create the alert.

[emmaLP]

Is there a workaround for this in the meantime while this is is being worked on?

This is a blocker for a fully automated project set up.

[JessicaGreben]

@emmaLP the only workaround that I could come up with is to get early access to the alpha for Stackdriver Workspace API. I was told to talk with our Google Cloud Support point of contact to ask to be added to that. However you would still need to write the code to use the API to create stackdriver stuff, since terraform still doesn't have support for it.

[chrisst]

I have added alpha support for stackdriver workspaces to our closed build of the google provider. Since this is a closed alpha you will need to contact your account manager to get access to the stackdriver API and also the alpha Google provider builds.
Both the API and the Google provider are in alpha so are probably not good candidates for integrating into production at this point. Unfortunately I don't know the timeline for the stackdriver API's so I can't provide an update for when it will be available publicly.

[tibers]

Similar verbiage for TF 12:
Error: Error creating AlertPolicy: googleapi: Error 400: 'projects/<PROJECT ID>' is not a workspace.

[lawliet89]

Hey @chrisst It's been more than half a year since your comment. Is that API still in alpha?

[tibers]

@lawliet89 yeah all the gcloud commands are still gcloud alpha ... for working with stackdriver.

example: https://cloud.google.com/sdk/gcloud/reference/alpha/monitoring/policies/create

[DanielJonesEB]

I know we're not supposed to leave +1 style messages, but given that there appears to be no other way to provide feedback or get visibility on StackDriver stuff, I'd like to say that I find it somewhat astonishing that in 2020 I can't fully automate the provisioning of an IAAS environment.