Pipelines: output_artifact_paths defaults aren't actually set in dsl.ContainerOp

[schmidt-jake]

What happened:
No pipeline output artifacts were being shown in the Kubeflow UI even though I properly wrote metadata to /mlpipeline-ui-metadata.json.

What did you expect to happen:
I expected a Facets Overview UI to appear in the Run Output / Artifacts of Kubeflow UI.

What steps did you take:
I ensured I was writing the html string to /mlpipeline-ui-metadata.json properly, following the documentation at https://www.kubeflow.org/docs/pipelines/sdk/output-viewer/#introduction.

Anything else you would like to add:
The documentation for the output_artifact_paths kwarg for dsl.ContainerOp says:

output_artifact_paths: Maps output artifact labels to local artifact file paths.
      It has the following default artifact paths during compile time.
      {'mlpipeline-ui-metadata': '/mlpipeline-ui-metadata.json',
       'mlpipeline-metrics': '/mlpipeline-metrics.json'}

but this apparently is not true. I manually set:

dsl.ContainerOp(..., output_artifact_paths={'mlpipeline-ui-metadata': '/mlpipeline-ui-metadata.json', 'mlpipeline-metrics': '/mlpipeline-metrics.json'})

and everything worked, the UI successfully displayed. Can someone please make sure these defaults are actually set?

[andrewsmartin]

This looks as though it was an intentional change. https://github.com/kubeflow/pipelines/pull/2046

It would be great to get more clear info about breaking changes in the release notes.

[Ark-kun]

Hello, @JakeTheWise, @andreyvelich and @daikeshi
We're sorry for making this breaking change. We were not aware of many people creating custom components that used this functionality.

This looks as though it was an intentional change.

The change was intentional. Recent versions of Argo fail the workflow is the artifact file is missing. The auto-added artifacts and their special treatment introduced many problems in the system.

It would be great to get more clear info about breaking changes in the release notes.

This is a good suggestion. We'll try to add this information to the release notes.

[Ark-kun]

P.S. In near future we're going to deprecate dsl.ContainerOp(..., output_artifact_paths=...) in favor of the older dsl.ContainerOp(..., file_outputs=...) which now supports the same functionality (producing artifacts).
We'll put deprecation warnings at least 3 releases before removing the feature.

[Ark-kun]

@JakeTheWise, @andreyvelich, @daikeshi Can you describe your use case for the mlpipeline-ui-metadata and mlpipeline-metrics artifacts in custom components? And can you also tell a bit more about the components you're creating. Which language do they use? What kind of data they work with? How big is the data? (There are new features that make big data passing much easier for all components - Lightweight python components, component.yaml definitions and ContainerOp)

[schmidt-jake]

All python. I'm basically recreating/customizing TFX pipelines. The datasets can get to be maybe a few gigs. The data is IOT timeseries from industrial equipment.

[andrewsmartin]

Same use case here as @JakeTheWise. We're integrating TFX components with KFP for more customizability. TFX comes with some visualizations out of the box (for example, StatisticsGen and ModelEvaluator) that we display in the KFP UI.

[schmidt-jake]

@andrewsmartin how'd you get those visualizations to work (I assume you're talking about the tfdv and tfma visualizations)? I couldn't get them to display in the KF UI.

[andrewsmartin]

Hi @JakeTheWise, we have some in-house code to, at the end of the corresponding TFX component run, grab the html file paths from GCS and drop the metadata into the `/mlpipeline-ui-metadata.json' file. There is some instructions for doing this here. Happy to help out if you run into any issues.

[schmidt-jake]

@andrewsmartin Yeah thanks! I was able to write my TFMA viz html to GCS and dump it into the component metadata file but there are still issues with rendering the html/widgets: Uncaught (in promise) TypeError: Cannot read property 'SlicingMetricsModel' of undefined at embed-amd.js:201132. Could you share your html template? Or at least point me to the right dependencies for mine?

_STATIC_HTML_TEMPLATE = """
<html>
  <head>
    <title>TFMA Slicing Metrics</title>
    <!-- Load RequireJS, used by the IPywidgets for dependency management -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.4/require.min.js"
            integrity="sha256-Ae2Vz/4ePdIu6ZyI/5ZGsYnb+m0JlOmKPjt6XZ9JJkA="
            crossorigin="anonymous">
    </script>
    <!-- Load IPywidgets bundle for embedding. -->
    <script src="https://unpkg.com/@jupyter-widgets/html-manager@^0.12.0/dist/embed-amd.js"
            crossorigin="anonymous">
    </script>
    <!-- Load IPywidgets bundle for embedding. -->
    <script>
      require.config({{
        paths: {{
          "tfma_widget_js": "https://raw.githubusercontent.com/tensorflow/model-analysis/master/tensorflow_model_analysis/static/index"
        }}
      }});
    </script>
    <link rel="import" href="https://cdn.rawgit.com/tensorflow/model-analysis/v0.6.0/tensorflow_model_analysis/static/vulcanized_template.html">
    <!-- The state of all the widget models on the page -->
    <script type="application/vnd.jupyter.widget-state+json">
      {manager_state}
    </script>
  </head>
  <body>
    {widget_views}
  </body>
</html>
"""


_SINGLE_WIDGET_TEMPLATE = """
    <div id="slicing-metrics-widget-{0}">
      <script type="application/vnd.jupyter.widget-view+json">
        {1}
      </script>
    </div>
"""

[Ark-kun]

"https://raw.githubusercontent.com/tensorflow/model-analysis/master/tensorflow_model_analysis/static/index"

You should link to exact versions, otherwise the files might just disappear one day.

[schmidt-jake]

Good point, thanks. I changed that to https://cdn.rawgit.com/tensorflow/model-analysis/v0.6.0/tensorflow_model_analysis/static/index and it started working, ha.

[Ark-kun]

Good point, thanks. I changed that to https://cdn.rawgit.com/tensorflow/model-analysis/v0.6.0/tensorflow_model_analysis/static/index and it started working, ha.

You just have to use commit hash instead of "master": https://raw.githubusercontent.com/tensorflow/model-analysis/5aeb800/tensorflow_model_analysis/static/index.js

[axeltidemann]

Hi @JakeTheWise, we have some in-house code to, at the end of the corresponding TFX component run, grab the html file paths from GCS and drop the metadata into the `/mlpipeline-ui-metadata.json' file. There is some instructions for doing this here. Happy to help out if you run into any issues.

@andrewsmartin Could you share code on how you did that? I am trying something similar, which I have described in this issue: https://github.com/tensorflow/tfx/issues/3094