Tvm: [Relay] Support for Tensorflow 2.0

[ic]

TF2 has been released this week, and becomes the stable version. Some APIs are breaking or not supported in Relay.

[Note: I may have been wrong on the resize op thing. Same problem with TF1.14]
For example, the half_pixel_centers attribute in the resize op has been introduced in TF2 (separate issue #4101), and deprecated namespaces like tf.gfile, tf.graph_util, tf.GraphDef, etc.

Walking through the TF tutorial, here are some elements to support TF2. Definitely insufficient, so just for getting started on the support.

[srkreddy1238]

@ic that was good start to upgrade TF.
You could raise a formal PR and invite for review to take it to completion.

We could use this issue to track TF upgrade.

[ic]

I'll be working on #4104, usually on Fridays.

[srkreddy1238]

How about the backward compatibility to 1.12 or 1.14 while upgrading to 2.0 ?

One easy solution is to use TF upgrade script and convert to 2.0 and use it, but it may not be more native to 2.0 features.

• Do we drop old version and go to 2.0 ?
• Maintain both
  1. Move old frontend to compatibility (relay.frontend.compatibility.from_tensorflow)? What would be CI default version ? Validating both test cases here may be a challenge !
  2. Selectively change behaviour based on tf.__version__

Any ideas @tqchen @tmoreau89 @yongwww @soiferj ?

[yongwww]

IMHO, both 1.x and 2.0 should be maintained for a while, most of the working tf models were built on top of 1.x. Another option is to maintain the same from_tensorflow for both 1.x and 2.0, it is possible to maintain it for both since savedmodel is the universal format, not sure how much work is needed. For TVM CI, maybe we can build different container for this kind of backward compatability testing. @tqchen

[ic]

Google seems to push for using TF2.x, perhaps to "catch up" on popularity against PyTorch and the like on developer happiness? Still I support @yongwww's point: Existing work outside Google is on TF 1.x (and 0.x :-)), and they will linger for some time. So this PR, and the sub-PR about fixing the tutorial target full backward compatibility, using the tf.compat.v1 APIs for now. Having said and done so, a couple comments:

• The new annotations like @tf.function really change a lot of syntax, and maintaining a single script for both 1.x and 2.x would be a pain for developers and users. We can detect at runtime what TF version is packaged, and switch script accordingly (e.g. a facade from_tensorflow module that loads the right implementation) (well, same as @srkreddy1238's point (2)).
• Does it matter anyway? TVM users might be looking ahead, as the community is growing and penetrating systems in production. It may be bold but saner to concentrate all strengths on what is clearly the close future: TF2.x.

[srkreddy1238]

Thanks for the comments @yongwww & @ic

As @yongwww said, front end implementation may not get affected much as the savedmodel is universal except the operator specific API changes in 2.0. TF 2.0 has major modification in terms of the way we build the graph and execute (session, eager execution, tf.function ..etc.). Test cases will demand more effort here.

I would suggest dual compatibility by

• Selectively switching at frontend implementation.
• Individual test cases to 1.x and 2.0.
• Necessary documentation or tutorial to promote TF 2.0 in coming days.

[tqchen]

It seems that we have reached consensus for dual compatibility for now. In terms of implementation details, TF2 diverges too much, we can also create a separate code path and do switch at the highest level

[yongwww]

@ic @srkreddy1238 if you guys are working on this support, would you mind sharing the status?

[cchung100m]

Hi @ic @srkreddy1238

Would you mind sharing the latest status?

[ic]

Sorry for the delay. The intention is to support TF1 and TF2 separately, trying to see if partial migrations can help. No progress since last update, although I hope to have some time soon to get back on it.

[FrozenGene]

Maybe we could boost the progress. I am told by many folks they are interested in Mobilenet V3 model. However, HardSwish is not defined in the r1.13 version of tensorflow so that we can not run Mobilenet V3 end2end test.

[tqchen]

NOTE: the Mainline CI now uses tensorflow 2.0

[srkreddy1238]

Good to see CI moving to 2.0.

On this context.

• I have upgraded TF test cases to use pure 2.0 API (with out compat v1, tf.graph, tf.session ...etc.).
• Trying to simplify the frontend with support to use model signatures unlike explicit arguments of shapes and out info (In the interest of SavedModel & TF Serving signatures being officially used export formats for TF).

Most of this work is completed (except few operators failing with 2.0 API).
Hoping to upstream soon once I am cleared with contribution policy approval.

[yongwww]

Good to see CI moving to 2.0.

On this context.

• I have upgraded TF test cases to use pure 2.0 API (with out compat v1, tf.graph, tf.session ...etc.).
• Trying to simplify the frontend with support to use model signatures unlike explicit arguments of shapes and out info (In the interest of SavedModel & TF Serving signatures being officially used export formats for TF).

Most of this work is completed (except few operators failing with 2.0 API).
Hoping to upstream soon once I am cleared with contribution policy approval.

Hi @srkreddy1238 do you have any update? it would be super cool to see a pr.

[yongwww]

We have looked into the models like image classification, object detection, segmentation, etc. Seems we need to enable the support for TensorList, control-flow, function, and some missing operations. We would like to work on this project from scratch. As we discussed above, we might create a new tf frontend for tf2.x (and I believe the parser for tf1.x will be deprecated in the future as tf community stops the support for 1.x).

Please comment here or ping me/us if anyone is interested in working with us on this. I synced with siva @srkreddy1238 in the past few weeks, we will keep him and the community updated if we work together on this.

[srkreddy1238]

TF 2.x differs at the front facing API level but the end graph doesn't change a lot here.
Doing a fresh parser will duplicate most of the layers we have till today. I think we may end up with a lot of redundant work.

My understanding and advice:

• TF2.x official format is saved model.
• TF2.x parser in my understanding should support inputs as
  (saved_model, signature name, input_spec) : Saved model input
  (concrete_function, input_spec) : Manually built TF Function.
  (TF Hub, input_spec) : A TFHub URL directly
• Parsing any of these input options finally goes through grappler and results in a grapph_def.
• There is no difference in this graph_def here and the existing parser does the job here.
• The additional layers (TensorList, VarHandles, etc.) could follow the existing implementation at node level.

TF2.x brings in standardization without the need of additional arguments like inputs, outputs, shapes(few cases) ..etc to be given by the user manually like we earlier asked for.
I think TF2.x is an enhancement or amendment over 1.x parser than a replacement.

Ref. https://github.com/srkreddy1238/tvm/tree/tf2.0 has the implementation.

• This branch is outdated and I need to rebase it.
• The 2.0 parser is part of the test script here.
• I will need to clean it bit and will send a PR soon as a baseline.

[yongwww]

@srkreddy1238 yeah, I agree that GraphDef protocol buffer definition just changed a little, creating a new parser will end up with some redundant work (copy from the existing parser). I tried to run the test cases in your shared branch, I found the existing parser works for most of the TF 2.x ops (previously I assume we have to modify most of the mapping in the parser). we can just update the existing parser for TF2.x, and remove unnecessary part in the future once we stop supporting 1.x support.

Please rebase your work, it would be good to have it work on top of TF 2.4(latest version is 2.4.0rc4). We can work on variable and tensorlist related ops at the same time.