Bazel: java rules should be able to depend on Skylark rules

[johnynek]

See:

https://github.com/bazelbuild/rules_scala/blob/master/test/BUILD#L15

The java libraries should probe to see if there are jars, transitive_runtime deps, etc...

[philwo]

Assigning to @laurentlb to get an idea how feasible this is.

[johnynek]

I would say there could be some contract that the java rule expects, and if skylark rules for scala, clojure, etc... follow that, the java rule would accept them.

This kind of jvm interop is pretty important for these other jvm languages.

[johnynek]

here is a possible work around approach:

https://github.com/bazelbuild/rules_scala/issues/18

[davidzchen]

@johnynek - Just out of curiosity, why do the deps labels for some of the scala targets not have the leading : in https://github.com/bazelbuild/rules_scala/blob/master/test/BUILD#L15?

scala_library(
    name = "HelloLib",
    srcs = ["HelloLib.scala"],
    deps = [
        "OtherJavaLib",
        "OtherLib",
        "MacroTest"
    ],
)

[johnynek]

@davidzchen it worked and I started removing them (fewer characters the better). Should targets always have : in the same package? Why does this work if so?

[ulfjack]

We've generally been using ":" to indicate rules, and "" to
indicate files.

Bazel handles both kinds of references as label references relative to the
current package. It performs lookup by first checking for a rule or an
output file, and if neither exists, it assumes that you're referencing a
source file. At the point in time where that happens, it does not check if
the source file exists.

On Thu, Mar 3, 2016 at 10:29 AM, P. Oscar Boykin [email protected]
wrote:

@davidzchen https://github.com/davidzchen it worked and I started
removing them (fewer characters the better). Should targets always have :
in the same package? Why does this work if so?

—
Reply to this email directly or view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-191676935.

[davidzchen]

Thanks for the explanations!

I think using : to distinguish between rules and files makes it more immediately clear whether a label referenced is a rule or file and would save time when reading BUILD files and understanding the dependencies between targets. There are cases where you could have input files that have no file extensions, in which case it would look like a rule unless you use the leading :. It would be better to continue using the same convention of using : to distinguish between rules and files for consistency.

@ulfjack - Is there a particular reason why Bazel allows referencing rules without the leading :? Would it be a good idea for us to enforce this convention?

[johnynek]

+1 to stronger checks. Would be a nice bazelrc option to require that.

[ulfjack]

Yes, I think it would be possible to have a stricter check.

On Fri, Mar 4, 2016 at 12:21 AM, P. Oscar Boykin [email protected]
wrote:

+1 to stronger checks. Would be a nice bazelrc option to require that.

—
Reply to this email directly or view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-192016779.

[jcoveney]

This conversation got derailed a bit around the :label, but I am curious how we want to deal with interop between JVM languages

I agree with @johnynek that the best way would be to create a contract that jar producing things can follow, and then some tools to pick them up and build a classpath

[ulfjack]

Carmi started to work on a Skylark API for Java such that custom rules can interop with the built-in ones and that we can eventually rewrite the built-in rules in Skylark. It's not there yet, but it's coming.

[ulfjack]

Maybe we can publish the current proposal and have the discussion publically?

[johnynek]

That sounds great.

On Tuesday, May 24, 2016, Ulf Adams [email protected] wrote:

Maybe we can publish the current proposal and have the discussion
publically?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-221301976

P. Oscar Boykin, Ph.D. | http://twitter.com/posco | http://pobox.com/~boykin

[ittaiz]

any news about this? I just stumbled on another use-case- I want to use the java_test rule from scala since my testing library (specs2) integrates with JUnit and that's how we work with it today.
If we had this then it's simply a matter of combining the two

[cgrushko]

I'm currently refactoring bazel/src/main/java/com/google/devtools/build/lib/rules/java/JavaLibraryHelper.java to make it easier to expose.
@ulfjack, IIUC, actually allowing java_library --> Skylark java to interop is slightly tangent to this?

[ulfjack]

@dslomov is working on a more complete proposal. Dmitry, would you mind giving more details on that?

[ittaiz]

I know the importance of this is understood but just to give a small reminder- I want to add junit support for scala.
Naturally I tried to use composition and use scala_library and java_test until I remembered this issue...
So instead of something along the lines of:

def scala_junit_test(name, srcs, visibility = None, **kwargs):
    scala_library(
        name = name + "_junit_lib",
        srcs = srcs,
        visibility = ['//visibility:private'],
        **kwargs
    )
    native.java_test(
        name = name,
        srcs = java_srcs,
        deps = [":" + name + "_junit_lib"] + kwargs.get("deps",[]),
        visibility = visibility,
        **kwargs
    )

I'll have to go and reimplement junit running much like java and groovy rules already do

[dslomov]

Work is underway to fix this, along the lines framed by this design document: https://bazel.build/designs/2016/08/04/extensibility-for-native-rules.html

[ittaiz]

Thanks! Any rough estimate on when we can expect java concretely? From the doc and the declared providers doc it sounds far away.

[dslomov]

Declared providers are in already (with caveats). Software engineers' estimates for the work not done are known to be horribly wrong, so I am not venturing anything here beyond "work is underway".

[ittaiz]

😁 Thanks!

If you can update the status of "Declared Providers" in the design doc that
would be great.

And again, thanks for your hard work!

On Sun, Nov 6, 2016 at 11:56 AM Dmitry Lomov [email protected]
wrote:

Declared providers are in already (with caveats). Software engineers'
estimates for the work not done are known to be horribly wrong, so I am not
venturing anything here beyond "work is underway".

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-258670672,
or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABUIFzR6R9FPuwU4aa8A2cfQgYkY2E3jks5q7aQxgaJpZM4HhVSL
.

[ittaiz]

@dslomov Does the 0.5 milestone label mean we can expect this to land this month? fingers_crossed :)

[dslomov]

@iirina wdyt? Is 0.5 realistic?

[iirina]

Yup, it should be done by then.

[ittaiz]

Wow, awesome!
When this will be in will I need to do anything else to have a java_test for example depend on a scala_library?

[iirina]

Hi,

A basic Java sandwich for java_library just landed on Github. You can find a working example here.

For now it's only possible for java_library to depend on a custom Java rule in Skylark, but if you have immediate need for another native Java rule (like java_test) to depend on Skylark rules, I can prioritize implementing that accordingly.

Please let me know if you have any questions or need something more than what is already implemented, I will gladly help.

Cheers,
Irina

[johnynek]

I would say java_test is next most interesting. Binary can be circumvented
pretty easily by adding a java_library in between.
On Fri, Jan 20, 2017 at 03:10 Irina Iancu [e-mail] wrote:

Hi,

A basic Java sandwich for java_library just landed on Github. You can find
a working example here
https://github.com/bazelbuild/bazel/blob/master/src/test/shell/bazel/bazel_java_test.sh#L161
.

For now it's only possible for java_library to depend on a custom Java
rule in Skylark, but if you have immediate need for another native Java
rule (like java_test) to depend on Skylark rules, I can prioritize
implementing that accordingly.

Please let me know if you have any questions or need something more than
what is already implemented, I will gladly help.

Cheers,
Irina

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-274068464,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEJdmclOYtZhZvnUAfjlf_Uc4H3fSQYks5rULIugaJpZM4HhVSL
.

[ittaiz]

I agree with @johnynek. Mainly since we have a workaround for java_library which is to use scala_library for scala, scala+java AND java only targets. We'd probably like to switch to java_library but it's not blocking us. Having java_test would allows us to run our tests right now which we currently can't (without re-implementing java_test for scala).
@johnynek given java_test supports this would you say junit scala support should be along the lines of a macro creating a scala_library and a java_test and linking those together?

[johnynek]

Yeah, as much as possible, let's reuse built in rules.

I haven't looked yet: but can we call java_library from another skylark
rule? When mixing scala and java targets we actually need to first call
scalac then javac then scalac again.
On Fri, Jan 20, 2017 at 11:19 Ittai Zeidman [e-mail]
wrote:

I agree with @johnynek https://github.com/johnynek. Mainly since we
have a workaround for java_library which is to use scala_library for scala,
scala+java AND java only targets. We'd probably like to switch to
java_library but it's not blocking us. Having java_test would allows us to
run our tests right now which we currently can't (without re-implementing
java_test for scala).
@johnynek https://github.com/johnynek given java_test supports this
would you say junit scala support should be along the lines of a macro
creating a scala_library and a java_test and linking those together?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-274182556,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEJdmjabd4ElWXD-0ktVefH8FUVFl9gks5rUSTQgaJpZM4HhVSL
.

[ittaiz]

You mean in order to remove the hack of calling javac manually in
scala.bzl? Good question. I already forgot this was a hack.
On Fri, 20 Jan 2017 at 23:26 P. Oscar Boykin [e-mail]
wrote:

Yeah, as much as possible, let's reuse built in rules.

I haven't looked yet: but can we call java_library from another skylark
rule? When mixing scala and java targets we actually need to first call
scalac then javac then scalac again.
On Fri, Jan 20, 2017 at 11:19 Ittai Zeidman [e-mail]
wrote:

I agree with @johnynek https://github.com/johnynek. Mainly since we
have a workaround for java_library which is to use scala_library for
scala,
scala+java AND java only targets. We'd probably like to switch to
java_library but it's not blocking us. Having java_test would allows us
to
run our tests right now which we currently can't (without re-implementing
java_test for scala).
@johnynek https://github.com/johnynek given java_test supports this
would you say junit scala support should be along the lines of a macro
creating a scala_library and a java_test and linking those together?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-274182556,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/AAEJdmjabd4ElWXD-0ktVefH8FUVFl9gks5rUSTQgaJpZM4HhVSL

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-274184031,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF1GmGXeFLu8iqiDn4FLPfLElrmFtks5rUSZ1gaJpZM4HhVSL
.

[johnynek]

Yeah. I would consider that a hack and more over, it somehow is not
reproducible. Compiling the same thing twice in that path can change the
bytes (have not tracked that down yet).
On Fri, Jan 20, 2017 at 11:30 Ittai Zeidman [e-mail]
wrote:

You mean in order to remove the hack of calling javac manually in
scala.bzl? Good question. I already forgot this was a hack.
On Fri, 20 Jan 2017 at 23:26 P. Oscar Boykin [e-mail]
wrote:

Yeah, as much as possible, let's reuse built in rules.

I haven't looked yet: but can we call java_library from another skylark
rule? When mixing scala and java targets we actually need to first call
scalac then javac then scalac again.
On Fri, Jan 20, 2017 at 11:19 Ittai Zeidman [e-mail]
wrote:

I agree with @johnynek https://github.com/johnynek. Mainly since we
have a workaround for java_library which is to use scala_library for
scala,
scala+java AND java only targets. We'd probably like to switch to
java_library but it's not blocking us. Having java_test would allows us
to
run our tests right now which we currently can't (without
re-implementing
java_test for scala).
@johnynek https://github.com/johnynek given java_test supports this
would you say junit scala support should be along the lines of a macro
creating a scala_library and a java_test and linking those together?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<https://github.com/bazelbuild/bazel/issues/970#issuecomment-274182556
,
or mute the thread
<

https://github.com/notifications/unsubscribe-auth/AAEJdmjabd4ElWXD-0ktVefH8FUVFl9gks5rUSTQgaJpZM4HhVSL
>

.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-274184031,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/ABUIF1GmGXeFLu8iqiDn4FLPfLElrmFtks5rUSZ1gaJpZM4HhVSL

.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-274184817,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEJdgoKrbCTHACZTRPej7j2Qxbba6ttks5rUSdagaJpZM4HhVSL
.

[ittaiz]

@johnynek after thinking about it some more scalac->javac->scalac sounds like a mistake.
If you give the first scalac all sources (java and scala) then you only need scalac->javac. If you want to give the first scalac only the scala sources that can fail if there is cyclic dependency.
Can/Should we call the java_library from ScalaCInvoker? Keeping this discussion here since it's relevant to the requirements of this solution

[iirina]

I've added support for java_test. Here is a test so you can see a working example.
Basically there is a java_test that has a dependency on a Skylark library. The Skylark library compiles some Java source files which results in a provider that must be returned so that the native rules would pick it up.

Would this work for you? Could you briefly describe your use case? Could you maybe point me to a small example where I could try it out? Thanks!

[johnynek]

@iirina this looks very helpful.

The main use cases I have are here: https://github.com/bazelbuild/rules_scala/blob/master/scala/scala.bzl

We want to be able to have java rules depend on scala output. And here:
https://github.com/bazelbuild/rules_scala/blob/master/src/java/io/bazel/rulesscala/scalac/ScalaCInvoker.java#L275

where we have to compile java code for mixed scala + java targets, we are invoking javac by shelling out. It looks like you have java_common.compile method we can call now from skylark.

@ittaiz can comment on his interest in the java_test. I think he wants to test scala code using java_test. I'm not sure if that can work with the current setup, since the test code still needs to be java, but maybe that is okay with him. He might want something like java_common.test which he can call from a skylark rule which would accept a jar of a test target, and a list of dependencies. Would that be workable?

[ittaiz]

@iirina IIUC I can have the a custom bzl generate a jar and return a provider and have the java_test accept that as a runtime dependency, no? From my understanding of the docs the java_test doesn't have to compile my test code. I just want the runner. Am I missing anything?

[johnynek]

@ittaiz I think the problem is test discovery. The discovery looks for
classes that are in the sources I think. What would you put in the srcs for
your java_test? They have to be java no?
On Thu, Jan 26, 2017 at 08:53 Ittai Zeidman [e-mail]
wrote:

@iirina https://github.com/iirina IIUC I can have the a custom bzl
generate a jar and return a provider and have the java_test
https://bazel.build/versions/master/docs/be/java.html#java_test accept
that as a runtime dependency, no? From my understanding of the docs the
java_test doesn't have to compile my test code. I just want the runner.
Am I missing anything?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-275476811,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEJdnq4qXhpXYZy6H3QoLdUH8JjzPmVks5rWOuvgaJpZM4HhVSL
.

[ittaiz]

I think the content of the srcs has to be java but I don't intend to put anything there.
I was hoping to get a general nod from @iirina and then maybe pull in @ulfjack, the wizard of java test discovery, to get his opinion about it.
I have a vague recollection about a java annotation processor being used but I don't remember the details.
At least my use-case is Scala JUnit and Specs2 tests where both of them have an @RunWith annotation on them.

[ulfjack]

java_test can be fully configured in whatever way you want, although I'd like it to be more useful with the default setting. It does not require having any sources on the java_test rule itself. (This should be better documented.)

One way is to customize it is to override test_class (test_class="my.test.Class"), which tells the test runner to use that as the main test class. It can be a Junit4 suite class, which allows you to do test discovery however you want. We ship a class with Bazel that looks at the classpath, so feel free to copy (license annotation applies, blah blah blah).

Alternatively, you can also override main_class and specify a custom test runner. Bazel forwards the test_class attribute to the test runner by means of an env setting, so a custom test runner can do anything that our default test runner can do. However, you may have to re-implement a variety of test protocols to better integrate with Bazel (I'm not sure they're all fully documented).

I have a prototype on which I haven't made much progress since Ittai visited which uses a simple text file in the jar to find the classes, and uses a runtime annotation processor on java_library to generate the text file (but you can also generate such a text file yourself in whatever way you want). However, that'll only help you if we enable that by default in the Bazel test runner.

[ittaiz]

@ulfjack Thanks for the no-sources confirmation. Currently the default java_test behavior is to run all classes which have an @RunWith annotation on them, right?
If that is correct what do I need to do, if any, to get that functionality working provided my sources are scala?
All of my scala tests have the @RunWith annotation on them.

[ulfjack]

java_test does roughly this:
java -Dbazel.test.class= -cp

[ulfjack]

I don't think that we're looking for @RunWith by default right now. You can manually enable that by setting test_class="com.google.devtools.build.lib.AllTests", but you'd have to have that class as well as ClasspathSuite on your classpath. That's what we're currently doing.

See here for an example:
https://github.com/bazelbuild/bazel/blob/master/src/test/java/com/google/devtools/build/lib/BUILD#L178

ClasspathSuite is here:
https://github.com/bazelbuild/bazel/blob/master/src/test/java/com/google/devtools/build/lib/testutil/ClasspathSuite.java

[ittaiz]

Ok- I'll give it a try and verify.
Can you please remind me how I can depend on "//src/test/java/com/google/devtools/build/lib:test_runner" in an "outside" repo? Is there something similar to "@bazel_tools//tools/jdk:jar"?

[ulfjack]

You can't do that directly. I think the easiest way may be to copy out the
two or three classes that you need.

On Fri, Jan 27, 2017 at 2:33 PM, Ittai Zeidman [e-mail]
wrote:

Ok- I'll give it a try and verify.
Can you please remind me how I can depend on "//src/test/java/com/google/
devtools/build/lib:test_runner" in an "outside" repo? Is there something
similar to "@bazel_tools//tools/jdk:jar"?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-275665977,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHA9YS0bz0NBK3Tljs4fm2GZ5XP4Shrxks5rWfJAgaJpZM4HhVSL
.

[ittaiz]

@iirina
I'm trying to understand how to create a provider in the Scala rules and I'm uncertain on how to create a provider.
The [providers] (https://bazel.build/designs/2016/08/04/extensibility-for-native-rules.html) design docs hints to:

 java_p = lang.java.provider(
                transitive_p,
                jar = jar_output,
                # update transitive information that we care about
                transitive_jars =
                    transitive_p.transitive_jars | set(jar_output),
                    ... whatever other information is needed ...)

But when I do something like it I get

java_common.provider(**kwargs) does not accept positional arguments, but got 1.

Additionally I saw in JavaSkylarkApiTest that cannotConstructJavaProvider so I'm not sure how to attack this.
The tests you link to assume the "custom library" uses the common compilation which is not my use-case.
Would really appreciate some pointers since I'm stuck

[iirina]

First, sorry for the outdated documentation on our site. What you linked is a proposal document (also marked as not implemented) and is not what the API actually looks like. I will make time to update this accordingly.

Now. Yes, this is intended behavior. From Skylark you can only create a provider from a compilation action. Alternatively, you can retrieve a provider from native java rules, merge multiple providers into one, and pass it/them on from Skylark.

What is your use case? Why are you trying to create a provider? Do you want to add more information after the compilation?

[ittaiz]

I'm trying to get a java_test target to depend on a scala_library target.
scala_library target needs to somehow expose a provider BUT it can't use
the compilation action since it uses its own compiler (and with persistent
workers but I don't know if that's related).
Any more info needed?
On Wed, 22 Feb 2017 at 12:18 Irina Iancu [e-mail] wrote:

First, sorry for the outdated documentation on our site. What you linked
is a proposal document (also marked as not implemented) and is not what the
API actually looks like. I will make time to update this accordingly.

Now. Yes, this is intended behavior. From Skylark you can only create a
provider from a compilation action. Alternatively, you can retrieve a
provider from native java rules, merge multiple providers into one, and
pass it/them on from Skylark.

What is your use case? Why are you trying to create a provider? Do you
want to add more information after the compilation?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-281627539,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF1wh2auqOuyU_NrBKYKmS12Lxv4Aks5rfAtpgaJpZM4HhVSL
.

[iirina]

Aaah I see. Unfortunately this is not yet available. You can only use the provider in the situations I described. Opening up the provider would be tricky, as it would require compilation information (runtime jars, compile time jars, and maybe instrumentation metadata). Would it be easy for you to provide these?

[ittaiz]

Runtime jar and compile time jars should be easy.
What do you mean by instrumentation metadata?
On Wed, 22 Feb 2017 at 12:47 Irina Iancu [e-mail] wrote:

Aaah I see. Unfortunately this is not yet available. You can only use the
provider in the situations I described. Opening up the provider would be
tricky, as it would require compilation information (runtime jars, compile
time jars, and maybe instrumentation metadata). Would it be easy for you to
provide these?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-281634191,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF0RpmtNkwmKrdsy55wjeyRrfgWfPks5rfBIlgaJpZM4HhVSL
.

[ittaiz]

@iirina,
Any estimations on complexity and time on your side?
@johnynek,
Wdyt? Should be pretty easy to supply runtime jars and compile time jars right?

[iirina]

Let me talk it over with my team and I will get back to you tomorrow with an answer (regarding both instrumentation and ETA)

[ittaiz]

👍🏽 Thanks!
On Wed, 22 Feb 2017 at 19:47 Irina Iancu [e-mail] wrote:

Let me talk it over with my team and I will get back to you tomorrow with
an answer (regarding both instrumentation and ETA)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-281745866,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIFyAvhAVPPWy0jkHSn7_xarQ5LMViks5rfHSYgaJpZM4HhVSL
.

[googlestanke]

@steren as FYI

[ittaiz]

Just to make sure we're aligned these are our use cases:

1. Scala JUnit rule- rule/macro which calls scala_library to compile the code of the tests with scalac and then calls java_test to run the tests themselves.
2. Scala Java cyclic dependency- currently solved in the scala rules by calling scalac with all files and then manually calling javac. We'd like to replace it with a call from skylark to java_library or to java_common.compile. Both require to be able to depend on the outputs of scalac.
3. "Regular" java_library depending on scala_library- don't think there's info to add here but obviously this also requires scala_library to output a provider.

[johnynek]

I'm disappointed 3 is not really possible with the current design. I'm
actually confused as to what the use cases are that would currently be
supported. Is there a practical example in mind for what has currently been
implemented or is it purely an intermediate state on the way?
On Wed, Feb 22, 2017 at 20:09 Ittai Zeidman [e-mail]
wrote:

Just to make sure we're aligned these are our use cases:

1. Scala JUnit rule- rule/macro which calls scala_library to compile
   the code of the tests with scalac and then calls java_test to run the tests
   themselves.
2. Scala Java cyclic dependency- currently solved in the scala rules
   by calling scalac with all files and then manually calling javac. We'd like
   to replace it with a call from skylark to java_library or to
   java_common.compile. Both require to be able to depend on the outputs of
   scalac.
3. "Regular" java_library depending on scala_library- don't think
   there's info to add here but obviously this also requires scala_library to
   output a provider.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-281903952,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEJdmQn6CYqShs7-7scyR7yI2uprNMTks5rfSKNgaJpZM4HhVSL
.

[ittaiz]

Any news?

[iirina]

@ittaiz So this should be done before the end of this quarter (end of march), probably earlier though. I think 2 could be done with the current implementation, calling java_common.compile. What are the outputs of scalac?

@johnynek Actually yes, we started migrating our native java rules to Skylark rules. Right now we are almost done implementing java_lite_proto_library in Skylark. This is the reason why we didn't open up the provider until now, because we created it from Java compilation.

[ittaiz]

The outputs of scalac are .class files

1 is what I must have. I'll see if I can tackle other things in the mean

time. If not I'll write my own JUnit test rule in skylark. Thanks!
On Thu, 23 Feb 2017 at 16:04 Irina Iancu [e-mail] wrote:

@ittaiz https://github.com/ittaiz So this should be done before the end
of this quarter (end of march), probably earlier though. I think 2 could be
done with the current implementation, calling java_common.compile. What
are the outputs of scalac?

@johnynek https://github.com/johnynek Actually yes, we started
migrating our native java rules to Skylark rules. Right now we are almost
done implementing java_lite_proto_library in Skylark. This is the reason
why we didn't open up the provider until now, because we created it from
Java compilation.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-281999391,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF1i3NuzsOH_RSaipgyNJzbEZ1nyAks5rfZHvgaJpZM4HhVSL
.

[ittaiz]

Any news about the instrumentation metadata?
On Thu, 23 Feb 2017 at 18:17 ittai zeidman [e-mail] wrote:

The outputs of scalac are .class files

1 is what I must have. I'll see if I can tackle other things in the mean

time. If not I'll write my own JUnit test rule in skylark. Thanks!
On Thu, 23 Feb 2017 at 16:04 Irina Iancu [e-mail] wrote:

@ittaiz https://github.com/ittaiz So this should be done before the end
of this quarter (end of march), probably earlier though. I think 2 could be
done with the current implementation, calling java_common.compile. What
are the outputs of scalac?

@johnynek https://github.com/johnynek Actually yes, we started
migrating our native java rules to Skylark rules. Right now we are almost
done implementing java_lite_proto_library in Skylark. This is the reason
why we didn't open up the provider until now, because we created it from
Java compilation.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-281999391,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF1i3NuzsOH_RSaipgyNJzbEZ1nyAks5rfZHvgaJpZM4HhVSL
.

[ulfjack]

The instrumentation metadata is for coverage only. The specific data needed there depends on the coverage implementation, though we currently only have support for Jacoco. Not every coverage implementation may require instrumentation metadata; we do something for Jacoco, but I can't remember the exact details. You can ignore this if you don't support coverage for Scala.

[ittaiz]

@ulfjack I tried seeing what files I needed to copy to have the AllTests behavior and I'm pretty sure I need 10 files which seems like a lot of code duplication...
I think I can trim it down if I copy and change the implementation to be a lot less generic and powerful but I don't know if that is ok (both legal and etiquette).
I'd really wish there was an easier way to res-use code from within Bazel.
Needed files:
AllTests
BazelTestSuiteBuilder
Classpath
ClasspathSuite
OS
Preconditions
Suite
TestSpec
TestSuiteBuilder
Seeing as I need also the TestRunner in the meanwhile this means copying ~ 200 files...

[steren]

P1 to reflect the high customer priority of this issue.

[dslomov]

P1 and 0.5 release means this is close to release blocker for 0.5. Is that the accurate picture?

[dslomov]

I also suggest to split out individual issues: provider construction, instrumentation metadata etc.
This feature request corresponds to phase 1 of https://bazel.build/designs/2016/08/04/extensibility-for-native-rules.html, whereas those extra requests are in phase 2)

[ittaiz]

I agree we might be to split this issue to ensure focus but I'd appreciate having the issue focus on the problem being solved (scala_junit as an example) and not on the solution (provider construction).
I iterated in this comment what are the use-cases that I need to solve:
https://github.com/bazelbuild/bazel/issues/970#issuecomment-281903952

[dslomov]

Well phase 2 of https://bazel.build/designs/2016/08/04/extensibility-for-native-rules.html directly talks about the Scala use case :)

[ittaiz]

Yes but the first comment of this issue talks about scala. This was the motivation of this issue. The fact that the design tried to first solve proto-lite was a big surprise to @johnynek and me but that's ok. I think this ticket is about those use-cases.

[dslomov]

Ok, fine. Splitting this into multiple issues according to https://bazel.build/designs/2016/08/04/extensibility-for-native-rules.html would be more accurate and would make everyone more informed about what gets shipped when and why.

[ittaiz]

Sure thing. I'm not opposed.
My main objection was to the sentiment I, maybe mistakenly, heard in your
comment about the issue actually being solved and that the rest are
advanced features which will happen at some future time.
On Tue, 28 Feb 2017 at 18:15 Dmitry Lomov [e-mail] wrote:

Ok, fine. Splitting this into multiple issues according to
https://bazel.build/designs/2016/08/04/extensibility-for-native-rules.html
would be more accurate and would make everyone more informed about what
gets shipped when and why.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-283085445,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIFwHtUHUjDft9lA2ChTN5CvJiLg2Xks5rhEgsgaJpZM4HhVSL
.

[johnynek]

As far as phase 1 goes, maybe some homework we can do is this: for mixed java and scala targets, we actually wind up calling javac ourselves, which is not great. It would be cool if phase 1 at least allowed us to use the bazel standard javac invocation (and all options that would go along with that).

For scala and java, the trick is that you first run the scala compiler including the .java files. It parses them to get the types, but does not compile them, then after scalac is invoked and the output of it is available, javac is invoked on just the java code and you include on the classpath all the compiled scala.

See here:
https://github.com/bazelbuild/rules_scala/blob/master/src/java/io/bazel/rulesscala/scalac/ScalacProcessor.java#L97

It would be great if we could at least in phase 1 handle this.

[iirina]

@johnynek I think if you wrap the compiled scala in a jar and pass it as a dependency tojava_common.compile, it should work.

[dslomov]

@ittaiz right, the goal with Bazel sandwich (which is our affectionate name for https://bazel.build/designs/2016/08/04/extensibility-for-native-rules.html) is to enable scale use cases, and it will happen, but it is a lot of engineering, so we have to do it in stages. I apologize for any miscommunication.

[lberki]

@ittaiz this thread is long and after re-reading it multiple times, I still haven't been able to figure out what changes are needed to Bazel :( Could you share a little "Hello World" repository with us that demonstrates your desired end state and what it is that you guys can't do at the moment? It would go a long way towards making the situation clear.

My understanding is that it's these things:

1. Have a scala_test rule (or macro) that works like this:

scala_test(name="t", srcs=["Test1.java", "Test2.scala"], deps=[":javarule", ":scalarule"])

1. Have Scala rules depend on Java rules and vice versa:

scala_binary(name="a", deps=[":b"])
java_library(name="b", deps=[":c"])
scala_library(name="c", deps=[":d"])
java_library(name="d")

1. Have Java and Scala source files in the same rule:

scala_library(name="lib", srcs=["Java.java", "Scala.scala"])

Is this correct?

[ittaiz]

Exactly correct.
2 clarifications:

1. scala_test is actually scala_junit_test since scala_test already exists
   and uses a different scala testing library
2. Use case #3 exists but with a hack where we call javac ourselves.
   On Thu, 2 Mar 2017 at 11:54 lberki [e-mail] wrote:

@ittaiz https://github.com/ittaiz this thread is long and after
re-reading it multiple times, I still haven't been able to figure out what
changes are needed to Bazel :( Could you share a little "Hello World"
repository with us that demonstrates your desired end state and what it is
that you guys can't do at the moment? It would go a long way towards making
the situation clear.

My understanding is that it's these things:

1. Have a scala_test rule (or macro) that works like this:

scala_test(name="t", srcs=["Test1.java", "Test2.scala"],
deps=[":javarule", ":scalarule"])

1. Have Scala rules depend on Java rules and vice versa:

scala_binary(name="a", deps=[":b"])
java_library(name="b", deps=[":c"])
scala_library(name="c", deps=[":d"])
java_library(name="d")

1. Have Java and Scala source files in the same rule:

scala_library(name="lib", srcs=["Java.java", "Scala.scala"])

Is this correct?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-283608142,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIFyl_LW5RtPUHT4Qfddgutj1H42lBks5rhpHCgaJpZM4HhVSL
.

[iirina]

@ittaiz What is the difference between scala_test and scala_junit_test? Also, wouldn't it work to use a java_import rule to import the scala jar and have scala_junit_test depend on that?

[ittaiz]

@iirina it's a bit of an unfortunate naming issue. Like java_test actually means java_junit_test and it doesn't support TestNG then the current scala_test rule supports one scala testing library and not junit. I just wanted to be clear that the new rule will probably be called scala_junit_rule.
WRT java_import then I think the problem there is that we lose all of the knoweldge we have about the scala target (like runtime deps etc). The current workaround indeed uses a deploy_jar and a java_import but then we have class-path pollution and other nasty issues.

[lberki]

We also need Groovy support and the requirements seems to be very similar (Groovy is a bit more flexible because Java stubs can be generated from Groovy but not from scala IIUC) so it would be nice if we provided tools that supported both. How about this plan for mixed rules:

1. We use scalac -sourcepath to compile Scala
2. We then use our Java compiler with the classpath set to the output of the Scala compilation to build Java (maybe with ijar in between for better incrementality)

If I understand correctly Scala -> Java -> Scala dependencies should work if java_common.compile is used.

For tests, at least in the short term, we could use a macro to replace scala_junit_test(name="joe", srcs=["Joe.java"], deps=[":john"]) with:

scala_library(name="joe_lib", srcs=["Joe.java"], deps=[":john"])
java_test(name="joe", deps=[":joe_lib"])

WDYT?

[ittaiz]

Regarding JUnit scala test that is definitely my intention.
I didn't understand the first part.
1) scalac knows to create java stubs so what we currently do is call scalac
with all sources and then call javac with the java sources and the output
of scalac.
I think that like noted here it's possible that the mixed target is already
supported with jaca_common.compile but that needs verification. The other
two use cases require java_library, java_test and java_library to accept
providers (I think that's in already) and for scala and groovy rules to
build providers (I think that's the missing piece)
On Fri, 3 Mar 2017 at 11:23 lberki [e-mail] wrote:

We also need Groovy support and the requirements seems to be very similar
(Groovy is a bit more flexible because Java stubs can be generated from
Groovy but not from scala IIUC) so it would be nice if we provided tools
that supported both. How about this plan for mixed rules:

1. We use scalac -sourcepath to compile Scala
2. We then use our Java compiler with the classpath set to the output
   of the Scala compilation to build Java (maybe with ijar in between for
   better incrementality)

If I understand correctly Scala -> Java -> Scala dependencies should work
if java_common.compile is used.

For tests, at least in the short term, we could use a macro to replace scala_junit_test(name="joe",
srcs=["Joe.java"], deps=[":john"]) with:

scala_library(name="joe_lib", srcs=["Joe.java"], deps=[":john"])
java_test(name="joe", deps=[":joe_lib"])

WDYT?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-283906063,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIFySBhdihOv0bO0sPDKlCnJGR-8plks5rh9v8gaJpZM4HhVSL
.

[lberki]

Oh, I see. I didn't realize that scalac can also create Java stubs (I don't really like using scalac in mixed mode because then you end up compiling some Java sources with the Java compiler embedded in Bazel and some others, the one that scalac has).

java_common.compile() has a deps argument and its return value can be passed tojava_common.merge(). Is this enough?

[ittaiz]

IINM Scala rules uses the compiler embedded in bazel
("@bazel_tools//tools/jdk:javac").
I think it's enough but we need to verify it in the code.
I'm busy re-implement JUnit support in scalac until this is resolved so I
can't verify it quickly.
On Fri, 3 Mar 2017 at 12:08 lberki [e-mail] wrote:

Oh, I see. I didn't realize that scalac can also create Java stubs (I
don't really like using scalac in mixed mode because then you end up
compiling some Java sources with the Java compiler embedded in Bazel and
some others, the one that scalac has).

java_common.compile() has a deps attribute and its return value can be
passed tojava_common.merge(). Is this enough?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-283915640,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF5sKmnK9NDJM5Q1lgxzgPMfhxEaXks5rh-aMgaJpZM4HhVSL
.

[lberki]

@bazel_tools//tool/jdk:javac is not the javac embedded in Bazel, it's the Java compiler that comes with whatever JDK Bazel is using (confusing, I know). How much of the JDK does scalac need and how does it interface with it? I was under the impression that it is a binary that reads Scala source and emits Java byte code without much involvement from the Java compiler itself.

[johnynek]

The scala compiler (scalac) is written in scala and thus is just like any
scala or java code: usually packaged as a jar and run with the java virtual
machine.

It does not require the java compiler at all, since as you say, it compiles
scala code into .class files for the java virtual machine.

In order to have mixed mode scala and java projects with circular
dependencies between the java and scala, the scala compiler can parse java
to understand the types in the code, but not compile the java to class
files. That compilation step must be done with javac.

Thus, the bazel scala rules, for targets which have both scala and java,
need to invoke the java compiler, for which we use the bazel_tools one
above.
On Sun, Mar 5, 2017 at 21:38 lberki [e-mail] wrote:

@bazel_tools//tool/jdk:javac is not the javac embedded in Bazel, it's the
Java compiler that comes with whatever JDK Bazel is using (confusing, I
know). How much of the JDK does scalac need and how does it interface
with it? I was under the impression that it is a binary that reads Scala
source and emits Java byte code without much involvement from the Java
compiler itself.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-284323111,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEJdv9jYU_1Nupu6yclL9TC44yu9dAfks5ri7f9gaJpZM4HhVSL
.

[lberki]

I see. Then, to be consistent, how about using java_common.compile() for compiling the Java code? Then you'd be automatically consistent with what Bazel otherwise does. Confusingly, it does not use @bazel_tools//tools/jdk:javac by default, but a javac bundled in it (called JavaBuilder)

[iirina]

@ittaiz Could you provide a little (non-working of course) demo for the scala_junit_test? Just a few Java and scala files and a BUILD fille to reflect how you would like the API to look like? It would make me more confident that my solution would work for your case and I would realize faster what needs to be done.

[ittaiz]

@iirina I actually have something working already in my fork
It's very basic but it shows the gist. I still need to evolve it to support class patterns.
It also uses JUnitCore and not the bazel runner which is not as feature rich.
I think that in essence this could be minimized to just building the scala library and generating the list of classes to run. I'd like to turn it into a macro but like I commented in #2539 it's unclear to me how to pass some of the info when in macro land.

[ittaiz]

@iirina please see my pull request to see how I worked around it. IIUC the main changes I'll need to make will be that in _scala_junit_test_impl I will replace the creation of the launcher and the scala binary with calling native.java_test and passing to it the generated test suite name and a jvm arg which is needed. Any news btw?

[iirina]

(sorry for the late reply!)

I'd like to turn it into a macro

I'm not sure what you would like to turn into a macro? scala_junit_test? I don't think that's feasible, as you'll have to make use of the java_common module and return providers.

I will replace the creation of the launcher and the scala binary with calling native.java_test and passing to it the generated test suite name and a jvm arg which is needed.

Are you asking about details on your current implementation of scala_junit_test or about when you would use the java_common module? Also, what jvm arg is needed?

Any news byw?

I added the possibility of creating a provider from compile and runtime jars (36958806f2cd38dc51e64cd7bcc557bd143bbdb6) but I don't think it's going to be in the next release (scheduled this week). You could still try it out by building bazel at head.

I am trying now to play around with your implementation of scala_junit_test and try to incorporate java_common. Does _compile(ctx, _jars, dep_srcjars, buildijar) compile both Java and scala sources? How does it work?

[ittaiz]

Re macro- In a nutshell I mainly need to change scala_library to return the providers but then I'll have other issues. To keep this issue sane I'll skip the rest of the details but if this of interest to you I'd really appreciate continuing this on #2539

About my question about the launcher creation never mind- It relates to my implementation of scala_junit_test and it might be unclear. Again for sanity of this issue I think we can move on.

About provider creation that's great! I'm actually usually on HEAD so that shouldn't be a problem.

About _compile- yes it does compile both Java and Scala sources. Currently this work is done in the worker. You can see there compileScalaSources and compileJavaSources. This outputs one jar file which contains the java and scala classes. I think this should make the introduction of the java_common provider fairly easy no? Sounds to me like it's collecting the providers and adding the current output. The thing I'm not sure about (and which might complicate things) is the relation between the _collect_jars method and the providers. Maybe if the scala rules start returning a java_common provider with compile-time/runtime jars then _collect_jars becomes obsolete.

[iirina]

To keep this issue sane I'll skip the rest of the details but if this of interest to you I'd really appreciate continuing this on #2539

I would actually like to have #2539 for tracking the progress on fixing that issue from the native java_test. If you would like to discuss this particular issue with scala_library the best way would be to send an email to [email protected] and I'll reply there. The same goes for other issues that you are trying to solve.

I think this should make the introduction of the java_common provider fairly easy no?

Yes, you should be able to pass the output jar to java_common.create_provider.

The thing I'm not sure about (and which might complicate things) is the relation between the _collect_jars method and the providers. Maybe if the scala rules start returning a java_common provider with compile-time/runtime jars then _collect_jars becomes obsolete.

So you should be able to collect the providers from both scala and java deps (meaning scala_library must also return java_common.provider), collect the one that would be returned by _scala_binary_common, merge all of them and return the result. And then you could have a java_test depending on your rule. In theory it should work. _collect_jars could just be replaced by _collect_providersI guess.

[sdtwigg]

I have been working with this experimentally and have had a few rough parts:
Original stream-of-conscious discussion here https://github.com/bazelbuild/rules_scala/issues/164#issuecomment-290275055 but will try to distill into better action items. The key items are in bold, my analysis will follow after each item. I apologize in advance that they aren't necessarily 'ordered' well....

• 'java_skylark_library' that provides JavaProvider can be in a deps or runtime_deps for java_library but not exports.
  The attempt is blocked by the attribute restrictions in https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/bazel/rules/java/ . Whereas deps and runtime_deps let you bypass the rule type whitelist by providing a JavaProvider, exports simply looks at the rule type whitelist. From my analysis of how exports actually 'flows' is that exports only really needs a JavaProvider as well and so exports should also check for a JavaProvider before erroring due to bad rule type.
  In JavaLibrary, compile jars are collected simply from the JavaCompilationArgsProvider inside each JavaProvider (via JavaLibraryHelper).
  As a separate mechanism, after actions have been generated and are now just setting up its providers for future rules, JavaLibrary (via JavaCommon) creates its JavaCompilationArgsProvider in its JavaProvider from all the jars it created plus all jars in the merger of JavaProvider.JavaCompilationArgsProvider from all rules in exports, runtime_deps, and deps. Note: JavaCompilationArgsProvider itself contains a recursive and non-recursive JavaCompilationArgs, which each contains compile and runtime jars. The non-recursive JavaCompilationArgs usually only adds jars from exports, not deps or runtime_deps. Also, the JavaCompilationArtifacts for the current target are usually split into ijar->compile jars and real jar -> runtime jars. Finally, runtime_deps only add to runtime jars and neverlink stops runtime jars from being added (except runtime_deps will always add jars).

• Cannot get compile_jars from JavaProvider
  In setting up a compilation, it would be highly convenient to just be able to merge all the JavaProviders from dependencies and then call compile_jars on the resulting provider to get all the jars needed to compile (which, from the prior analysis, looks to be the ijar from each dep and then all the compile_jars in your exports). This would then very closely mimic how JavaLibraryHelper.addLibrariesToAttributesInternal works.
  This seems like an easy add: The constructor for JavaProvider already exposes "transitive_runtime_jars" to skylark as getRecursiveJavaCompilationArgs().getRuntimeJars(). Just need another entry in JavaProvider of "compile_jars" as getJavaCompilationArgs().getCompileTimeJars().

• minor issue: create_provider produces a JavaCompilationArgsProvider with recursive==non-recursive
  Since most of the time compile jars cares about the non-recursive one and runtime jars cares about the recursive one, you can work around this by making compile_time_jars = non-recursive set of your ijar + all export compile_jars and runtime_jars = your jar + runtime jars of deps, exports, and runtime_deps.
  However, if you had the ability to rebuild JavaProvider where only the non-recursive set in JavaCompilationArgsProvider was blank and the recursive set stayed the same, would be closer to fine by doing the following set of operations:
  merge all deps JavaProvider together -> merge in all runtime_deps JavaProvider -> clear non-recursive set -> merge in all exports -> merge in create_provider with just local artifacts

• minor issue: Cannot provide a JavaSkylarkApiProvider
  This might not actually be a problem as more fields of JavaProvider become readable by Skylark rules. Also, technically, since JavaSkylarkApiProvider is only used by OTHER Skylark rules, it is possible to completely fake one by using nested structs.
  That said, it seems like one could envision a world (especially with compile_jars above) where either JavaProvider totally subsumes JavaSkylarkApiProvider or any rule with a JavaProvider automatically uses said provider to construct a JavaSkylarkApiProvider.The latter is tempting because JavaSkylarkApiProvider is actually documented so other rules clients might be more likely to attempt dep.java before dep[java_common.provider]. Perhaps ultimately, dep.java just becomes an alias to dep[java_common.provider].

• minor issue: JavaSkylarkApiProvider.transitive_exports just a list of labels
  Again, this may not be a problem if compile_jars in JavaProvider becomes available since that is likely a sufficient replacement for . Also, this can potentially be worked around via [dep for dep in target.java.transitive_deps if dep.owner in target.java.transitive_exports] although that seems inefficient. Note, the docs say dep.owner can be empty but my intuition is that this is only true when said item is a raw file and, fortunately as discovered in first item, all exports must be a rule so all files in deps from exports should have an owner.

I left out addressing/mimicking a bunch of other features in the native java rules (neverlink, non_strict, whatever DependencyArtifacts in JavaCompilationArgs add, etc.) but I think this would go a long way to allowing users to closely mimic what is the 'common path' of a java_library rule. It would actually be an interesting exercise (perhaps for testing purposes) to see in how many places then java_library in https://github.com/bazelbuild/bazel/blob/master/tools/build_rules/java_rules_skylark.bzl could just be used instead of native.java_library without the rest of the build graph noticing at all.

[iirina]

Thanks for all your feedback, Stephen! To address your items:

'java_skylark_library' that provides JavaProvider can be in a deps or runtime_deps for java_library but not exports.
Yes, you are right. exports should also allow rules that return JavaProvider. This can be easily fixed.

Cannot get compile_jars from JavaProvider

JavaProvider was first designed as a black-box until java_common had a pretty stable API. The plan was to open it up steadily, as soon as there was a need for something. Returning a compile_jars is a very reasonable and easy to implement request, so it should be added ASAP.

minor issue: create_provider produces a JavaCompilationArgsProvider with recursive==non-recursive

Yup, this API needs some more work, that's why it's undocumented. Your idea sounds good, I will look into it.

minor issue: Cannot provide a JavaSkylarkApiProvider

JavaSkylarkApiProvider should go away eventually. It was added to facilitate retrieving information from java rules, but it wasn't adequate to be used for the sandwich implementation. That's when JavaProvider came along. It will wrap more native providers than it does now, probably all of those in JavaSkylarkApiProvider, and its API will be more open and will include most of the fields exposed via JavaSkylarkApiProvider.

minor issue: JavaSkylarkApiProvider.transitive_exports just a list of labels

This should be solved by previous comments.

[ittaiz]

thanks @sdtwigg for leading this integration in the rules_scala area! I'm sure we're all waiting eagerly for progress there

[hlopko]

Hello Irina, is this still expected to be fixed in 0.5? Is this a release blocker? We're still at least 2 weeks away from the release but we should be getting ready.

[iirina]

Yup, this is expected to be in the next release. All important changes have been submitted and a few small patches will follow soon.

[hlopko]

After offline discussion not treating as a release blocker. Although it seems this issue will be closed by then anyway :)

[iirina]

@ittaiz Is there anything else you would need for implementing the scala rules using java_common? The issues raised by @sdtwigg are WIP.

[ittaiz]

@sdtwigg took the lead there so if he thinks he has all the tools he needs
then we're good.
@sdtwigg,
Any loose ends towards our ability to use java_common.compile, depend on
scala from java and remove the scala provider?
On Mon, 10 Apr 2017 at 14:43 Irina Iancu [e-mail] wrote:

@ittaiz https://github.com/ittaiz Is there anything else you would need
for implementing the scala rules using java_common? The issues raised by
@sdtwigg https://github.com/sdtwigg are WIP.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-292925731,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIFzWweNkWemiK-n6gyb0WcdvEFMFEks5ruhX1gaJpZM4HhVSL
.

[sdtwigg]

I am reasonably confident everything is in order with those proposals.

Using
https://github.com/sdtwigg/rules_scala/commit/9c8f7201c1539fda0bc432264f01cde1a2f40adc
as a base, it looks like java_common should be a drop in. I haven't
experimented with the java_common.compile stuff yet but it seems reasonable
to add as a follow-on (modulo a few questions wrt java_toolchain
interacting with other toolchain proposals and host_javabase).

Once I get convergence with Irina on the proposals (primarily on the
Skylark API although Friday PR discussion resolved almost all of this):
will make a bazel fork with the changes integrated. This can then be used
to test an experimental follow-on rules_scala commit that drops using the
scala provider in favor of JavaProvider. If that works then would be enough
to say the sandwich is fulfilled. This can also be done in parallel to the
various pending bazel PRs (since the bazel fork would just assume they are
all cherry-picked in).

There are a handful of potential follow-on concerns regarding the sandwich
and inter-operating with other features in java rules but I don't think is
worth primary consideration. Just something to keep in mind for eventual
consideration.

On Mon, Apr 10, 2017 at 2:54 PM, Ittai Zeidman [e-mail]
wrote:

@sdtwigg took the lead there so if he thinks he has all the tools he needs
then we're good.
@sdtwigg,
Any loose ends towards our ability to use java_common.compile, depend on
scala from java and remove the scala provider?
On Mon, 10 Apr 2017 at 14:43 Irina Iancu [e-mail] wrote:

@ittaiz https://github.com/ittaiz Is there anything else you would
need
for implementing the scala rules using java_common? The issues raised by
@sdtwigg https://github.com/sdtwigg are WIP.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-292925731,
or mute the thread
n6gyb0WcdvEFMFEks5ruhX1gaJpZM4HhVSL>

.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-293090413,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA9_-DLduFieSVYhFaMJp76aZuEUqpUoks5ruqUfgaJpZM4HhVSL
.

[damienmg]

Moving to 0.6 for the remaining work to do, the basics for rules_scala should be already there.

[hlopko]

Ok, thanks everybody!

[ittaiz]

Maybe the right thing to do is to open several smaller issues for the
remaining work? Dmitry already commented the thread might be too long here
On Tue, 11 Apr 2017 at 11:30 Marcel Hlopko [e-mail] wrote:

Ok, thanks everybody!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/970#issuecomment-293188362,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF73ThQpdseao394ikRinENrc3J47ks5ruzpBgaJpZM4HhVSL
.

[steren]

+1 to this suggestion, it will be easier to track and to understand what's remaining.

[lberki]

In that case, let me close this bug and continue the discussion on per-issue ones.

[ittaiz]

@lbrelki do we have additional issues for these? I don't think so

[iirina]

@ittaiz There is one that I know of and that is creating a provider with empty non-recursive compilation jars. I'm already working on that and it should be submitted next week.
@sdtwigg Is there anything else?

[ittaiz]

@iirina do you mean 9671: Loosen java_library.exports and java_import.*?
IINM there is no github issue for that but if indeed that's the only issue left then maybe it's ok. Maybe @sdtwigg can add more.