Bazel: Support C++20 modules

[abergmeier]

Description of the problem / feature request / question:

Starting to look into C++ Modules. While it will earliest officially arrive in 2020, would be good to have support for that soonish.
Heard there is some Module support inside of Google. Can/Will that be opensourced?

[hlopko]

Hi Andreas,
everything regarding modules (only for clang) is open sourced already. The only missing piece is a CROSSTOOL that makes use of them and provides all necessary features. But I think it would be quite a challenge to write such CROSSTOOL without understanding the implementation. If I don't find a good movie on my flight to bazel conference I might take a look and hack something together. No promises though, since this would be a free time exercise, not something that is a priority for bazel right now.

[abergmeier]

Interesting.
@mhlopko Can you perhaps elaborate a bit, what parts of CROSSTOOL would interact with modules? Personally I thought it would be more in the realm of cc-providers and additional actions for module validity checks.

[hlopko]

Yeah you are right, the implementation does deal with providers and actions, but all that logic is already in bazel and ready to be used. What is missing is a bunch of features (and maybe action_configs, I'm not sure) in the CROSSTOOL to pass the right flags for the right actions. Names of the features are hardcoded in bazel, so they need to match.

[hlopko]

So in other words, it's all there, just not very well documented.

[abergmeier]

just not very well documented

Like all else about CC toolchains you mean ;)

Thanks. I am trying to get a custom hermetic Clang toolchain running anyways. Perhaps will then try this out a bit.

[hlopko]

Exactly :) I hope I'll see you next week on the crosstool breakout session during bazel conference? :) Btw, just wrote this yesterday, I really hope I'm not one day late: https://github.com/bazelbuild/bazel/wiki/Yet-Another-CROSSTOOL-Writing-Tutorial

Anyway, don't hesitate to ask if you get stuck on the way. I'm also grateful for any input/comments/suggestions.

[abergmeier]

Since I do have problems with the include paths for libc (https://stackoverflow.com/questions/47078520/clang-toolchain-fails-for-system-libc-files), how does libc_top (inputsForLibc) work?

Especially since I cannot seem to be able to glob system files.

[ArekSredzki]

Now that C++20 Modules are nearing standardization, could we get some updates on the status of properly supporting them within Bazel?

I was fortunate to attend CppCon this year and was excited to see that many build systems were making progress on supporting modules. Unfortunately, myself and other Bazel users that I spoke to were concerned that there was no Bazel-specific information on the topic.

[hlopko]

We don't plan to work on C++ modules in 2019. We are discussing working on them in 2020 mostly in the context of objc and swift rules. We'll update this issue once we have more precise plans.

[ArekSredzki]

Thanks @hlopko, I'm sure that much of the C++ Bazel community will very much value support for them. Looking forward to hearing more.

[ctrysbita]

Any update here? :D

[ArekSredzki]

@hlopko Is there anyone that we could ping regarding the timeline for supporting C++20 modules?
There appears to be significant community interest in this feature, it'd be good to set expectations :)

[hlopko]

@oquenchil and @c-mita are authorities there.

[ArekSredzki]

@oquenchil @c-mita Bump on the above? :)

[oquenchil]

It might be that this gathers steam at the beginning of next year but I can't promise anything. In any case, it would be clang modules, not C++20 modules.

Whoever is interested in clang modules, definitely thumbs up the original issue above and that might help with prioritization.

[technoir42]

it would be clang modules, not C++20 modules.

Should we create a separate issue for C++20 modules then? I'm only interested in those as I'm using MSVC on Windows.

[oquenchil]

Yes, you can create a separate issue for tracking C++20 module support. Be aware though that as it looks right now it will be a long time before we work on it, probably not next year.

[thii]

Can you rename this issue to "Support Clang modules"?

[rnburn]

In any case, it would be clang modules, not C++20 modules.

@oquenchil - By "Clang modules", do you mean Clang's non-standard module system that pre-dates C++20 modules?

Why target that instead of C++20 modules? Clang supports a lot of the standard module spec already (blog post) -- I expect enough that people could start using it.

[oquenchil]

A lot of the implementation for Clang modules already exists for the Google internal version of Bazel, the effort required to implement Clang modules in Bazel would be less than for C++20 modules. We are not discarding ever implementing the latter though, just that it would come at a later time.

[rnburn]

But why hijack this issue into a clang module feature request? The original poster clearly worded it for C++20 modules and I expect that's what most people following it want.

If someone's looking for clang's homegrown module system, can we let them submit a different issue for it? Is it even broadly used outside of Google?

C++20 module support may not be on Google's short-term roadmap, but there are a lot of project that would benefit from being able to use standard modules, so perhaps someone else might sponsor the work.

[ArekSredzki]

I second that C++20 module support would be far more beneficial to the community at this point.
Many organizations use multiple toolchains, in which case, supporting Clang's module implementation brings little value.

I suspect that most people following this ticket are interested in C++20 modules since that's what it was created for.
Could a separate ticket be created for Clang modules instead of renaming this one as that's a separate request with a niche application? 🙏

[ArekSredzki]

Thanks @oquenchil !

[det]

Hello! I took a shot at implementing Clang Modules in a local bazel repo as this ticket said that everything for modules is already open-sourced. I believe I have implemented most of the crosstool features nearly correct (but maybe not?) and I am running into a couple issues:
1) I'm not sure how module maps for the C++ stdlib and sysroot (libc) should be handled:

• You must provide a crosstool.cppmap as the module_map for your cc_toolchain. It's not clear if this should include maps for both your c++ stdlib and sysroot.
• Bazel doesn't ever seem to compile this module map into a pcm file. How are we supposed to acheive this?
  
  
  
  • Are you supposed to precompile a .pcm file and include it in your compiler files and add -fmodule-file=path/to/crosstool.pcm to the use_header_modules feature?
    
    
    
    
    
    • Can this even work since pcm files are sensitive to compile flags which can be changed on the command line?
      
      
      2) Bazel seems to support 2 modes of modules support, with and without modules codegen. When using --features=header_module_codegen it will correctly try to compile the .pcm file to a .o file but it is not making available all the .pcm dependencies of the .pcm file that is being compiled. For example, if foo.pcm imports bar.pcm but when foo.pcm is being compiled to foo.pcm.o it will try to open bar.pcm which is not available in the sandbox and so will error. I also verified with aquery that the dependent pcm files are not provided as inputs to that compile action.
    
    
    
  
  

Except for the layering check, there is no documentation or examples to be found anywhere on the internet so I would appreciate any pointers as well as clarification of whether clang modules should currently be usable in the open source bazel. @hlopko @oquenchil

I am using bazel 3.7.1 and Clang 11.0.0 in my toolchain if that matters.

Thanks!

[EvanHonnold]

@hlopko @oquenchil Might you have a chance to take a look at @det 's comment above? I am running into many of the same issues and would be curious to know if there are any workarounds.

[hlopko]

I haven't yet, @c-mita, maybe you have worked with Bazel modules recently?