Pipelines: Frontend UI cannot display large workflows

[radcheb]

What steps did you take:

I started a big pipeline. The pipeline runs successfully but the pipeline graph is not displayed.

What happened:

The UI says "No graph to show" and no graph is show:

What did you expect to happen:

Expected the UI to display the pipeline graph.

Environment:

Kubeflow 1.0
Pipeline compiled with kfp 0.5.1

KFP version: 1.0 commit 743746b

KFP SDK version: 0.5.1

Anything else you would like to add:

[Miscellaneous information that will assist in solving the issue.]
It appears that the api return a workflow with compressedNodes field instead of nodes, in workflow.status.
The compressedNodes is actually a base64-encoded gzip-compressed json string of nodes.
I discovered this while analysing the api data.

Here is python code used to generate an example big pipeline:

from kfp import Client
from kfp.compiler import Compiler
from kfp.components import func_to_container_op
from kfp.dsl import pipeline

def some_func(i: int) -> str:
    msg = f"""{i}
    This is a huge function, with a lot of code.
    Lorem Ipsum is simply dummy text of the printing and typesetting industry. 
    Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, 
    when an unknown printer took a galley of type and scrambled it to make a type specimen book. 
    It has survived not only five centuries, but also the leap into electronic typesetting, 
    remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets 
    containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker 
    including versions of Lorem Ipsum.
    """
    return msg

@pipeline(name="huge pipeline")
def test_pipeline():

    component = func_to_container_op(func=some_func, base_image="library/python:3.6")

    for i in range(400):
        op = component(i=i)
        previous = op

run = Client().create_run_from_pipeline_func(
    pipeline_func=test_pipeline,
    arguments={},
    experiment_name="kf-tests-experiment"
)

manifest = json.loads(Client().get_run(run.run_id).pipeline_runtime.workflow_manifest)
assert "nodes" not in manifest["status"]
assert "compressedNodes" in manifest["status"]

print(manifest["status"]["compressedNodes"])

/kind bug
/area frontend
/area backend

[radcheb]

I suggest a fix in this PR:
https://github.com/kubeflow/pipelines/pull/4180

It's already tested locally.

[Bobgy]

Thanks! I'll take a look

[yoni-taranis]

@radcheb - Thanks for the PR, I'm running with your fix and indeed we now have no issues with large workflows.

I do have to say that I had an error in the decodeCompressedNodes function - "TextDecoder is not a constructor" - not sure why, for now I changed the decoding to use the Buffer class (Buffer.from(result.buffer).toString();)

[radcheb]

@yoni-taranis Thanks a lot for the hint about TextDecoder. I used your solution (with native Buffer) and it works.
https://github.com/kubeflow/pipelines/pull/4180/files#diff-93870a1522c3892c5495182d183e2252R371

The PR is now ready https://github.com/kubeflow/pipelines/pull/4180 if you could test it locally it will be great.