Terraform-provider-google: AI Hub Support in Terraform

[dipanjannag]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

Google cloud added the the AI Hub in Beta. Currently in Terraform we can create a Ai Hub model but we can not create Ai Hub jobs, notebook resources.

New or Affected Resource(s)

google_ai_hub

• google_ai_hub
• google ml_engine

References

• google_ai_hub
  

[ndmckinley]

It's not immediately clear that this would be doable with our standard REST approach - it seems like we'd need to call out to Python, from the docs. That might be hard to package. If this acquires a lot of :+1: we'll consider that - otherwise, we most likely want to stick with REST-accessible resources.

[dipanjannag]

@ndmckinley Thanks! However I'm not very sure what I'm missing But I feel we might be able to manage it via Rest API (i.e. via service: ml.googleapis.com ).
Resource Jobs: Rest API ref
Resource Model: Rest API ref
Resource Model.version: Rest API ref

[ndmckinley]

We could definitely support the REST objects, but to upload the scikit models or tensorflow models, we'd need to call out to python, it seems. I'm talking about the operations described here - those are the ones that would be hard to support. Would the REST objects be useful to you if we didn't have those operations?

[dipanjannag]

Yes It'd be definitely helpful in general sense. The way I understand it,

• ML jobs creates a ML model (e.g. tensorflow_model.pbor simple_model.pkl). Now any custom python module used in the process is generally not part of infra and should not be a responsibility of Terraform.
  To give an example: For Dataflow Job anything pointed via template_gcs_path is user code (maybe a python module).
• In the same way when creating models or models.version any user code (e.g. python modules) i.e. code pointed via packageUris[] in projects.models.versions is separate and should not be a part of Infra itself.

[noamt]

I'd be happy to help with this feature

[ndmckinley]

Awesome, all yours! Let us know if we can be of any support here. :)

[noamt]

It looks like magic modules already has the definition for models and model versions, which means it's technically possible to publish new versions.
So what's missing? Is it only a question of generating the terraform provider?

[dipanjannag]

We can always create new model+version. We can not create Jobs, Notebooks etc.

manage it via Rest API (i.e. via service: ml.googleapis.com ).
Resource Jobs: Rest API ref
Resource Model: Rest API ref
Resource Model.version: Rest API ref

[noamt]

Ah, OK.
It also looks like jobs are on the way:
https://github.com/GoogleCloudPlatform/magic-modules/pull/2565