Wire: Document the proper way to use protoPath for protos defined in a module dependency

[jdkoren]

I have a project with two modules, app and lib. Both modules use Wire and define protos; moreover, app depends on lib and the proto in app wants to use definitions from a proto in lib.

Looking at the documentation, protoPath does not appear to support something like project(':lib'), as this is neither "a local directory, a local .jar file, [n]or an external artifact specified with Maven coordinates."

Currently I work around this by exploiting the fact that I know the directory structure of the project to located the other module's protos: app/build.gradle.

What is the proper way to use protoPath to identify protos in another module of the same project?

[oldergod]

This should work though.

dependencies {
  protoPath project(":lib")
}

Have you tried?

[jdkoren]

@oldergod When I do that and try to rebuild, it gives the following error:

Execution failed for task ':app:generateProtos'.
> Could not resolve all files for configuration ':app:protoPath'.
   > Could not resolve project :lib.
     Required by:
         project :app
      > Cannot choose between the following variants of project :lib:
          - debugRuntimeElements
          - releaseRuntimeElements
        All of them match the consumer attributes:
          - Variant 'debugRuntimeElements' capability Proto Dep:lib:unspecified:
              - Unmatched attributes:
                  - Found com.android.build.api.attributes.BuildTypeAttr 'debug' but wasn't required.
                  - Found com.android.build.api.attributes.VariantAttr 'debug' but wasn't required.
                  - Found org.gradle.usage 'java-runtime' but wasn't required.
                  - Found org.jetbrains.kotlin.platform.type 'androidJvm' but wasn't required.
          - Variant 'releaseRuntimeElements' capability Proto Dep:lib:unspecified:
              - Unmatched attributes:
                  - Found com.android.build.api.attributes.BuildTypeAttr 'release' but wasn't required.
                  - Found com.android.build.api.attributes.VariantAttr 'release' but wasn't required.
                  - Found org.gradle.usage 'java-runtime' but wasn't required.
                  - Found org.jetbrains.kotlin.platform.type 'androidJvm' but wasn't required.

Edit: Some more info.

Android Studio 4.0 Canary 9
Build #AI-193.5233.102.40.6137316, built on January 15, 2020
Runtime version: 1.8.0_212-release-1586-b4-5784211 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Linux 5.7.17-1rodete3-amd64
GC: ParNew, ConcurrentMarkSweep
Memory: 1931M
Cores: 8
Registry: ide.new.welcome.screen.force=true
Non-Bundled Plugins: Dart, org.jetbrains.kotlin, com.dubreuia, com.google.services.firebase, idea.plugin.protoeditor, io.flutter, org.intellij.plugins.markdown


$ ./gradlew --version

------------------------------------------------------------
Gradle 6.1.1
------------------------------------------------------------

Build time:   2020-01-24 22:30:24 UTC
Revision:     a8c3750babb99d1894378073499d6716a1a1fa5d

Kotlin:       1.3.61
Groovy:       2.5.8
Ant:          Apache Ant(TM) version 1.10.7 compiled on September 1 2019
JVM:          11.0.8 (Debian 11.0.8+10-post-Debian-1deb10u1)
OS:           Linux 5.7.17-1rodete3-amd64 amd64

[oldergod]

I am not sure this is a Wire related error. You might have a more helpful message upgrading Gradle since it looks they've been slowly iterating on this kind of error. https://github.com/gradle/gradle/issues/12126

[martinbonnin]

Follow up from https://github.com/gradle/gradle/issues/12126#issuecomment-755237088, I think for

dependencies {
  protoPath project(":lib")
}

to work, the Gradle plugin should create a consumable configuration in the lib project as explained in https://docs.gradle.org/current/userguide/cross_project_publications.html

That doesn't seem to be the case at the moment so protoPath will resolve any existing flatDir or jar file but has no way to know how to generate one from the lib project

[jdkoren]

As an experiment, I tried changing my sample's :lib module to a java library module on a separate branch. Now instead of the above issue, there seems to be a problem importing the ice_cream.proto in user_prefs.proto, and I have not found an import statement that resolves it. Here's the error when building:

Cause: unable to find com/example/lib/ice_cream.proto
  searching 8 proto paths:
    ${HOME}/projects/testing/ProtoDep/lib/build/libs/lib.jar
    ${HOME}/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.4.10/ea29e063d2bbe695be13e9d044dcfb0c7add398e/kotlin-stdlib-1.4.10.jar
    ${HOME}/.gradle/caches/modules-2/files-2.1/org.jetbrains/annotations/13.0/919f0dfe192fb4e063e7dacadee7f8bb9a2672a9/annotations-13.0.jar
    ${HOME}/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-common/1.4.10/6229be3465805c99db1142ad75e6c6ddeac0b04c/kotlin-stdlib-common-1.4.10.jar
    ${HOME}/.gradle/caches/modules-2/files-2.1/com.squareup.wire/wire-runtime/3.4.0/4b04136c8543a37c5c9e52db9c50dfb03e578e39/wire-runtime-jvm-3.4.0.jar
    ${HOME}/.gradle/caches/modules-2/files-2.1/com.squareup.okio/okio/2.8.0/49b64e09d81c0cc84b267edd0c2fd7df5a64c78c/okio-jvm-2.8.0.jar
    ${HOME}/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.4.10/998caa30623f73223194a8b657abd2baec4880ea/kotlin-stdlib-jdk8-1.4.10.jar
    ${HOME}/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk7/1.4.10/30e46450b0bb3dbf43898d2f461be4a942784780/kotlin-stdlib-jdk7-1.4.10.jar
  for file ${HOME}/projects/testing/ProtoDep/app/src/main/proto/com/example/protodep/user_prefs.proto
unable to resolve IceCream
  for field ice_cream (${HOME}/projects/testing/ProtoDep/app/src/main/proto/com/example/protodep/user_prefs.proto:8:3)
  in message UserPrefs (${HOME}/projects/testing/ProtoDep/app/src/main/proto/com/example/protodep/user_prefs.proto:7:1)

Of those jars, only the first one seems like it should contain ice_cream.proto inside, but I can confirm that it currently does not contain that file:

~/projects/testing/ProtoDep $ jar tvf lib/build/libs/lib.jar
     0 Thu Jan 07 14:42:58 PST 2021 META-INF/
    25 Thu Jan 07 14:42:58 PST 2021 META-INF/MANIFEST.MF
     0 Thu Jan 07 14:42:58 PST 2021 com/
     0 Thu Jan 07 14:42:58 PST 2021 com/example/
     0 Thu Jan 07 14:42:58 PST 2021 com/example/lib/
  1523 Thu Jan 07 14:42:58 PST 2021 com/example/lib/IceCream$Companion$ADAPTER$1.class
   922 Thu Jan 07 14:42:58 PST 2021 com/example/lib/IceCreamUtilKt.class
  2654 Thu Jan 07 14:42:58 PST 2021 com/example/lib/IceCream.class
  1354 Thu Jan 07 14:42:58 PST 2021 com/example/lib/IceCream$Companion.class
    59 Thu Jan 07 14:42:58 PST 2021 META-INF/lib.kotlin_module

Am I doing something wrong?

[oldergod]

Don't you just need to add it into the jar then?

[jdkoren]

Don't you just need to add it into the jar then?

@oldergod I don't know how to do this in Gradle, so far my attempts to do so have not been successful. (Any guidance here would be greatly appreciated).

Regardless, I think this illustrates the point that the documentation for Wire does not adequately address using dependencies { protoPath(':lib') }. In the first place, I don't even see that in the documentation. Beyond that, if something more is needed to build properly in this situation, then that should also be part of the documentation (or the wire plugin should take care of that under the hood).

[trevjonez]

I am hitting similar issues.
you can add the protos to the jar by adding some config to the jar task like this:

in my proto only module im using the java lib plugin just for the easy jar setup and later maven config:

plugins {
  `java-library`
}

tasks.named<Jar>("jar") {
  from("src/main/proto") {
    into("proto")
  }
}

which produces this:

then in my client module i do:

plugins {
  kotlin("jvm")
  id("com.squareup.wire")
}

wire {
  kotlin {
  }
}

dependencies {
  protoSource(project(":protos"))
}

which produces this when you try to generate.

Execution failed for task ':cci-client:generateMainProtos'.
> unable to find com.aetna.avdf.messages.Start
    searching 0 proto paths:

    for file /HyperX16/Source/android-virtual-device-farm/runners/protos/build/libs/protos-0.1.0.jar/proto/com/aetna/avdf/services/JobRunner.proto

so then I think why don't I add it to the path as well so it can find it:

protoPath(project(":protos"))

which then produces:

Cause: unable to find com.aetna.avdf.messages.Start
  searching 1 proto paths:
    /HyperX16/Source/android-virtual-device-farm/runners/protos/build/libs/protos-0.1.0.jar
  for file /HyperX16/Source/android-virtual-device-farm/runners/protos/build/libs/protos-0.1.0.jar/proto/com/aetna/avdf/services/JobRunner.proto

which isn't any better so I am thinking for generating against previously packed proto files it should probably only be on the protoSource config?

so it seems more like a schema loader issue to me at this point once you get the gradle files sorted to package and distribute the jar

[oldergod]

We're working on a comfortable solution right now.

[trevjonez]

just found my issue as well. I had some package names typo'd and I needed to remove the into("proto") on the jar task config.

as suspected protoSource was the only one required for my usecase.

[oldergod]

https://square.github.io/wire/wire_compiler/#dependencies-between-gradle-modules

[jdkoren]

The current solution is working when the library project is a java-library (yay!), but there is still an issue if the library project is an android-library. This can be worked around, but it would be ideal if it could work in either scenario. Here's the error I see when building:

A problem occurred configuring project ':app'.
> Task with name 'compileJava' not found in project ':app'.

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

I've updated the master branch of my sample with these changes, please feel free to check it out and run ./gradlew app:assembleDebug to reproduce.

[oldergod]

That should be fixed on Wire 4.x alpha stuff