Glow: What's the Roadmap of Glow

[QiJune]

I am very interested in this project. The design of NodeIR and InstructionIR is awesome. Lots of optimization from compiler are introduced.

I have interests in several modules and also have some questions:

• Optimizations for the different backends. In the CodeGen stage, we still need to implement each instruction in C code for CPU backend and/or OpenCL kernel code for OpenCL backend. I am not sure if we need some another IR to express the optimizations, such as loop transformation for different kinds of hardware. For example, TVM uses Halide IR, while TensorComprehensions uses both Halide IR and Polyhedral IR.

• Support dynamic RNN with control flow related Node, such as WhileNode. Glow now implement RNN by unrolling each step. It will make the model size larger and maybe less flexible.

• Support for specialized neural network accelerators. There are emerging lots of AI chips now. I am not sure if Glow has such plan to support some accelerators based on FPGA or ASIC.

A roadmap will be really great for the community participation. Thanks!

[nadavrot]

@QiJune Thanks you for the kind words. I love your contribution to the project.

I agree with your comment. I think that we should come up with a better way to discuss our plans publicly. This is especially important as more contributors join the project. I wonder if it would be a good idea to open a mailing list. What do other project on Github do? Maybe we can use the Wiki or Project tabs on github?

Here are my opinions to your questions.

1. OpenCL is an experimental project. @opti-mix and I developed it to start experimenting with host-device partitioning and with a non-cpu backend. This gave us the ability to work on a platform that did things differently, for example, it required non NHWC convolutions. This helped to stress different parts of the compiler. However, the design of the OpenCL is not performance. It simply calls a sequence of convolutions. It's slightly better than Caffe, because it schedules and recycles memory, but it does not optimize the kernels themselves. For people who care about GPU performance I would suggest one of two possible designs. Either go down the XLA path and launch optimized GPU kernels, and auto-generate fused data-parallel kernels. Or, come up with a new (possibly Halide-like) approach for generating kernels.

2. We currently unroll RNN networks. This works pretty well because the compiler can schedule the network memory efficiently. The downside is that it's not easy to implement early termination. @qcolombet is working on a proposal for predication, that will allow efficient early termination. The Glow paper discusses this approach.

I think that IfNode is the wrong approach. XLA implemented it because they had to support the TensorFlow nodes. Internally they must convert these nodes to some other representation. I think that when we are ready to implement control flow in Glow we should use the CFG approach (basic blocks with gotos, like LLVM).

1. I can't provide any information here, but I'll say that Glow has some secret backends. There are people who develop out-of-tree backends in Glow.

Please let me know what you think.

[QiJune]

@nadavrot I think that we can have a wiki to describe Glow's long-term roadmap.
For a common Github project, a minor version will be released after a few months' developments. The minor version is actually the project's short-term roadmap/goal. As for the short-term roadmap, we can discuss and list it via Github issues. It can be deatiled task description to attract community developers' particiption easier.

And thanks for your detailed answers, it really helps.

I think that when we are ready to implement control flow in Glow we should use the CFG approach (basic blocks with gotos, like LLVM).

Got it! It's a more concise solution.

[wolf1981]

@nadavrot I am very interested in this project too. Would glow be planed to support CUDA backend in the near future?

[nadavrot]

@wolf1981 We don't have plans to support CUDA in the near future.

[bertmaher]

We now have https://github.com/pytorch/glow/wiki/Glow-Roadmap on our GitHub wiki to provide a roadmap for external contributers/users. Thanks for suggesting this improvement!