Pipelines: Add endpoint to the API server to provide information about the deployment

[NikeNano]

Add endpoint to KFP API server that exposes cluster specific information: e.g. what is expected mode, single/multi? What is ui_host? etc.

This would be beneficial for the CLI to use better default values, see further discussion: https://github.com/kubeflow/pipelines/pull/4293

[Bobgy]

/cc @Ark-kun

Thank you for creating the issue!
I think many agree with this idea, next step would be to clearly define what exact info should be exposed and how to configure it.

[NikeNano]

Some things that would be great are:

• namespace which Kubeflow is deployed in
• endpoint
• deployment type
• deployment version

[Bobgy]

These info probably make sense in the existing /healthz endpoint: https://github.com/kubeflow/pipelines/blob/2676b56151ba57f17fd43c5e0734297632724785/backend/src/apiserver/main.go#L142

[NikeNano]

Ok, sounds good, do you see any more information that would be relevant to add @Bobgy?

[Bobgy]

For deployment type, I imagine it could be

• full kubeflow
• standalone
• gcp ai platform pipelines

Is this what you are talking about?

We could also include metadata which cloud platform it was deployed to, maybe not immediately useful, but I think we might get to that point.

[Bobgy]

Or maybe you meant single/multi user mode?
I guess we could include both

[DavidSpek]

I'm currently looking into this as well for my PR https://github.com/kubeflow/pipelines/pull/4638.

[DavidSpek]

@NikeNano @Bobgy I have added the single/multi user mode to the /apis/v1beta1/healthz to solve the issue of the KFP namespace also being automatically set in single user mode in my PR https://github.com/kubeflow/pipelines/pull/4638, after which it passed all tests. I have now moved the API call to /apis/v1beta1/healthz to its own function and (hopefully) fixed the output so that multi_user is a boolean. Currently waiting for the new tests to finish but I don't expect the last changes to cause new issues.

[NikeNano]

For deployment type, I imagine it could be

• full kubeflow

• standalone

• gcp ai platform pipelines

I think both the type in terms of full/standalone/ai platform and single/multi would be great to include.

[NikeNano]

@NikeNano @Bobgy I have added the single/multi user mode to the /apis/v1beta1/healthz to solve the issue of the KFP namespace also being automatically set in single user mode in my PR https://github.com/kubeflow/pipelines/pull/4638, after which it passed all tests. I have now moved the API call to /apis/v1beta1/healthz to its own function and (hopefully) fixed the output so that multi_user is a boolean. Currently waiting for the new tests to finish but I don't expect the last changes to cause new issues.

Sounds great!

[DavidSpek]

@NikeNano Just to keep this issue up to date. I have attempted to perform a get request to the healthz API from the python client in a large variety of ways with them all failing during e-2-e testing due to authorization issues for the KFP api endpoint in GCP. My latest attempt is to add the healthz endpoint to the kfp-server-api pypi package, which is still busy with tests. I hope I can get everything working properly soon so more information can be added to the API endpoint as discussed above.

[DavidSpek]

The PR has just passed all the tests. I cleaned up the code a bit and it is running tests again now. As (hopefully) the PR can be merged soon, I think it would be useful to further discuss what information should be added to the healthz API. Seeing as I have been busy with this for a while now I'm willing to take a shot at adding the info to the API server if I can get some guidance where the wanted constants are found in the code.