Tvm: [RFC] Relay C++ Frontend

[antinucleon]

In production environment, usually it is pure C++ environment. We are working out a pure C++ relay frontend. I want to know whether there is public interest so we can contribute back.

Pros: Less dependency, more friendly to production
Cons: We need to maintain two frontends, doubled work when there is update, or we slowly move to C++ frontend and only use Py as wrapper.

[Ravenwater]

+1

We try to avoid the Python overhead and are always looking at the C++ interfaces. All of our research is centered around new number systems and hardware acceleration and Python is a net negative for any of these use cases.

When customers do ask for a Python interface, we tend to wrap a C++ API of some kind.

So count us in for a pure C++ interface that removes the Python dependency for both development and production.

[junrushao1994]

It is possible to write an IR converter which consumes IR built in Python, and outputs C++ source code containing IR building functions in C++, or vise versa. In this way, one could benefit both C++ and Python users at the same time.

[antinucleon]

@junrushao1994 It depends on how the production pipeline is built. Currently we are moving relay.build_module, relay.frontend.caffe2 and some TOPI to C++ and it should work for Facebook pipeline. If there is an IR convert to convert TOPI directly to C++ it will simplify the problem a lot.

[srkreddy1238]

+1

[tqchen]

I think having C++ development flow is great and I support the proposal.

In the meanwhile, we will keep the "python first" principle for ease of prototyping and development, so developers will be able to develop in both C++ and python. Note that having python component in compiler does not harm the fact that we can have a clean multi-language deployment.

[jroesch]

In general we have been developing initially in Python and then moving core APIs to C++. I think moving more and more code into C++ and re-exporting via Python over time is generally good, and allows other languages to also expose core APIs.

I echo what Tianqi wrote we should focus also focus on providing a flexible set of APIs in Python as well so users can easily experiment and prototype without having to directly modify TVM's core. I think this is a huge advantage of the TVM stack currently.

[Ravenwater]

Since C++ is so much more expressive than Python particularly for numerical software, having a stand-alone C++ environment is much more productive for deep research in tensor processing, scheduling, and new arithmetic. The latter in particular is unpleasant with a Python interface as Python is lacking template support.

Once you have the core data structures and operators defined in the core, then scripting DL models via a Python interface is attractive, particularly with the integration with Numpy and Scipy libraries, and other integration libraries to deal with images, audio, etc.

But having a clean C++ core with all the trimmings is a more powerful experimental environment for DL and compiler algorithms.

[kevinthesun]

Currently a lot of topi related optimization modules are in python, such as autotvm. We might want to gradually add corresponding C++ part, while these module are stable enough.

[tvm-archiver]

I think we should consider it, I think having the tuner sit in Python is
okay the more important bit being the schedules and other compiler pieces
in C++ for integrating the
compiler. I talked with some PyTorch people today and they suggested a
Python free version of the compiler would be important if they use it in
JITing.

• Jared

On Tue, Apr 2, 2019 at 4:28 AM Lianmin Zheng [e-mail]
wrote:

Should we open an RFC to discuss how to port autotvm and topi to c++?

--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/dmlc/tvm/issues/2685#issuecomment-478954829

[tqchen]

3082