Tfx: AttributeError: 'list' object has no attribute 'type' [while running 'GenerateStatistics[train]

[pingsutw]

https://colab.research.google.com/github/tensorflow/tfx/blob/master/docs/tutorials/tfx/components.ipynb
When running TFX example on google colab, got following errors

statistics_gen = StatisticsGen(
    examples=example_gen.outputs['examples'])
context.run(statistics_gen)

Error messages:

INFO:absl:Running driver for StatisticsGen
INFO:absl:MetadataStore with DB connection initialized
INFO:absl:Running executor for StatisticsGen
INFO:absl:Using 1 process(es) for Beam pipeline execution.
INFO:absl:Generating statistics for split train
INFO:absl:Statistics for split train written to /tmp/tfx-interactive-2020-06-28T05_35_31.021774-wrfcf4sp/StatisticsGen/statistics/2/train.
INFO:absl:Generating statistics for split eval
INFO:absl:Statistics for split eval written to /tmp/tfx-interactive-2020-06-28T05_35_31.021774-wrfcf4sp/StatisticsGen/statistics/2/eval.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/apache_beam/runners/common.cpython-36m-x86_64-linux-gnu.so in apache_beam.runners.common.DoFnRunner.process()

98 frames
AttributeError: 'list' object has no attribute 'type'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/tensorflow_data_validation/arrow/arrow_util.py in flatten_nested(array, return_parent_indices)
    192   parent_indices = None
    193 
--> 194   while is_list_like(array.type):
    195     if return_parent_indices:
    196       cur_parent_indices = array_util.GetFlattenedArrayParentIndices(

AttributeError: 'list' object has no attribute 'type' [while running 'GenerateStatistics[train]/RunStatsGenerators/GenerateSlicedStatisticsImpl/TopKUniquesStatsGenerator/ToTopKTuples']

TensorFlow version: 2.2.0
TFX version: 0.22.0
tensorflow-data-validation: 0.22.0

[Chaddhaji]

I have the same error. Any solutions?

[rmothukuru]

Could reproduce the error. Please find the Github Gist. Thanks!

[sansrc]

I have the same error. Any solutions?

Pinning tensorflow-data-validation to version 0.22.0 seems to fix this issue for now:

!pip install tfx==0.22.0 tensorflow-data-validation==0.22.0

[Chaddhaji]

I have the same error. Any solutions?

Pinning tensorflow-data-validation to version 0.22.0 seems to fix this issue for now:

!pip install tfx==0.22.0 tensorflow-data-validation==0.22.0

Yes, but then it throws error in Evaluator Component.

[sansrc]

I have the same error. Any solutions?

Pinning tensorflow-data-validation to version 0.22.0 seems to fix this issue for now:
!pip install tfx==0.22.0 tensorflow-data-validation==0.22.0

Yes, but then it throws error in Evaluator Component.

Cannot reproduce that. At least for me, the notebook completes without an error, both locally and on google-collab.

[pingsutw]

clicked the restart button after running below commands, it seems to work.

!pip install tfx==0.22.0

[Chaddhaji]

clicked the restart button after running below commands, it seems to work.

!pip install tfx==0.22.0

Hi pingsutw!
I just noticed "!pip install tfx==0.22.0" was already present in the colab. But it was not working yesterday. Is it some kind of bug?
Because I didn't do anything, and now the colab is working.

Thanks pingsutw.

[sansrc]

I just noticed "!pip install tfx==0.22.0" was already present in the colab. But it was not working yesterday. Is it some kind of bug?
Because I didn't do anything, and now the colab is working.

It's because __tensorflow-data-validation==0.22.2__ was released in the meantime. tfx==0.22.0 specified the tensorflow-data-validation dependency like this >=0.22.0,<0.23.0. You pip-installed before the release and got tensorflow-data-validation==0.22.1. After the release you got tensorflow-data-validation==0.22.2, which fixed the issue.

tfx==0.22.0 & tensorflow-data-validation==0.22.0 ok
tfx==0.22.0 & tensorflow-data-validation==0.22.1 breaks
tfx==0.22.0 & tensorflow-data-validation==0.22.2 ok

See Bug Fixes in TFDV 0.22.2 Release: https://github.com/tensorflow/data-validation/releases/tag/v0.22.2

@zhitaoli I guess, this issue can be closed.

[zhitaoli]

Yes let's close this. I think we should upgrade our dependency range to >=0.22.2,<0.23.0 for upcoming patch release, but AFAICT you don't need to make a change on our side.