Kibana: [Ingest Manager] Refactor Datasource create/edit forms to recognize and support the package's `solution` field to drive displaying a custom UI for configuration

[paul-tavares]

The Package definition seems to now support defining a flag named solution on a Datasource whose purpose is to instruct the Kibana UI that a custom UI should be displayed. See Endpoint definition here:

https://github.com/elastic/package-storage/blob/master/packages/endpoint/0.1.0/manifest.yml#L23

datasources:
  - name: endpoint
    title: Endpoint data source
    description: Interact with the endpoint.

    # This tells the UI that for configuration, it must link to a specific solution
    # Only solution packages can contain this field.
    solution: endpoint

The Kibana Ingest Manager creation and editing of Datasources should be refactored to now process based on this flag being set to a know value.

(see this PR comment for reference)

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[paul-tavares]

We should verify that flag is in fact valid - I also did not realize it existed until recently when i looked at this file.

/cc: @jen-huang , @ruflin

[ph]

@paul-tavares I've looked and searched through the code I don't know where this field is coming from, I thought we initially wanted to hardcoded the special UI based on the package type but if we can make it work without too much effort with a cleaner field lets do it.

@ruflin I having the "solution" at the datasource level in the package would work?

[paul-tavares]

@ph that field is on the package in the package-storage and does not yet
make it all the way across to ingest types (I assume we strip it out). Part
of this change would be to add it to ingest Datasource types and then use
it to drive the UI.

--

---

*Paul Tavares *| Sr. Software Engineer | Elastic / www.elastic.co | Durham,
NC, USA

[ruflin]

The field was added having Uptime, APM in mind. But TBH it potentially makes more sense to hardcode it instead in Kibana. The reason is that for example lets say the of Endpoint link changes in 7.10 but the old packages are still around. As the Endpoint UI and Ingest Management UI are always on the same version, it might be simpler to just hardcode it there and remove it again from the package info.

[jen-huang]

As the Endpoint UI and Ingest Management UI are always on the same version, it might be simpler to just hardcode it there and remove it again from the package info.

I think that is still doable while still utilizing this solution field. The value of solution will just act as a flag to know whether a certain package is a custom solution. The implementation details of what the custom solution displays/links to is separate from the value.

[jen-huang]

I will work on this after #67234 is merged.

[ruflin]

@jen-huang Does this mean the flag should more become a bool?

[jen-huang]

@jen-huang Does this mean the flag should more become a bool?

I don't feel strongly about this. If we leave it as a string, then we can potentially point multiple packages to the same solution, but I'm not sure if that's a realistic use case. Thoughts?

[ruflin]

Taking a step back: How does a "custom" solution hooking into our configuration building? If I remember our conversation correctly, there is a place the solution can register itself and request a callback. Lets assume we have a package foo. So far we built the UI based on the manifest. But in the next Kibana release there is no a UI for the package foo. What would happen, is that this UI now registers in the Ingest Manager to be the one called for the package foo. The package itself did not have to know, that it has now a dedicated UI. If the above is correct, I'm wondering if we even need a bool?

[jen-huang]

If the above is correct, I'm wondering if we even need a bool?

The logic you describe correct. If we feel that solutions registering themselves in the UI is good enough without the additional safeguard/check on the package manifest, then we can do away with the solution field entirely.

[ruflin]

This is my preference at the moment and we introduce the solution field if we need safeguards later on.

Do we rely somehow on the solution field today or can it be just removed? @jen-huang @paul-tavares

[ruflin]

I opened https://github.com/elastic/package-registry/pull/504 on the registry side.

[paul-tavares]

It can just be removed. We don’t needed in endpoint.

---

*Paul Tavares *| Sr. Software Engineer | Elastic / www.elastic.co | Durham,
NC, USA

[jen-huang]

Closing this as we don't need to do anything on Kibana side now.