Quarkus: gRPC extension

[hpehl]

Provide an extension to deploy gRPC services in Quarkus. An initial PoC is available here:

• https://github.com/hpehl/quarkus-grpc-extension: Extension which starts a gRPC server and registers annotated gRPC services and interceptors.
• https://github.com/hpehl/quarkus-grpc-quickstart: Demo how to use gRPC with Quarkus.

The extension works in Java mode, but fails in native mode. Which might be related to #764.

[hpehl]

Status Update

The extension works now in both JVM and native mode using a simple gRPC service.

However, there are still some issues / open points:

• [x] Test more & complex gRPC services (route guide example)
• [x] Streaming support
• [x] Enable DI for services and interceptors
• [ ] Devmode support
• [ ] SSL support
• [x] Interceptors
• [x] Client extension (wip)
• [x] Generated Java code depends on reflection (see https://developers.google.com/protocol-buffers/docs/reference/java-generated#message option optimize_for = CODE_SIZE)
• [ ] In the long term get rid of the protobuf-maven-plugin and make the stub / message generation part of the deployment phase / dev mode. The proto definitions would then end up somewhere under the META-INF folder.

[rsvoboda]

Hi Harald, what are your plans with Quarkus and gRPC ?
I noticed some activity on your hpehl/quarkus-* repositories.

CCing @emmanuelbernard and @patriot1burke as gRPC was mentioned last week

[hpehl]

I'd like to finish the missing steps. I still have trouble getting native mode working for the grpc-client extension. and I need to test SSL support. Once these steps are done, the extensions could become part of Quarkus or I can make them available as externally.

Unfortunately I didn't have much time lately to push this further. I'm updating the versions numbers from time to time.

[rverma-nikiai]

Can we use vertx grpc https://vertx.io/docs/vertx-grpc/java/ with quarkus?

[sean-scott-lr]

Hey Guys,

I was wondering where the support for grpc falls as plans for official support. We are doing a PoC on Quarkus to determine its feasibility and grpc is a requirement ( in native mode ).

[rsvoboda]

@emmanuelbernard @cescoffier what's the plan for grpc ?

[cescoffier]

We need to stabilize Http/2 first, then gRPC should be easy.

[hpehl]

@cescoffier Thanks for thew update. Let me know when you're done.

[sean-scott-lr]

I have noticed that 1.0-RC1 has been released. However, it appears that there is still no support for GRPC. Does this mean that HTTP/2 has also not been stabilized?

When is support for these features planned on the roadmap? ( Note, I know this is an open source project... and thank you for your efforts! But, we are wrapping up technology selection phase, and GRPC support is a requirement for us. )

thanks!

[hpehl]

@sean-scott-lr FYI: There's a discussion on the forum [1] how to bring gRPC support to Quarkus.

[1] https://groups.google.com/d/msg/quarkus-dev/FpPN5Bei6-0/2YtEXSxoDgAJ

[sean-scott-lr]

Http/2 first,

@cescoffier is the Http/2 stabilization being tracked on a particular issue that I can follow?

thanks!

[sean-scott-lr]

@hpehl Hi Harald.

I am trying to get the grpc-client extension to compile in native mode. Can you point me to some of the issues you have had and overcome and which remain? Currently, my native image compilation is failing with this error:

[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] /Library/Java/JavaVirtualMachines/graalvm-ce-19.2.1/Contents/Home/bin/native-image -J-DCoordinatorEnvironmentBean.transactionStatusManagerEnable=false -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 --initialize-at-run-time=io.vertx.core.net.impl.transport.EpollTransport,io.netty.channel.epoll.Epoll,io.netty.channel.epoll.EpollEventLoop,io.netty.channel.epoll.EpollEventLoopGroup --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -jar _artifactName_-0.0.0-SNAPSHOT-runner.jar -J-Djava.util.concurrent.ForkJoinPool.common.parallelism=1 -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:+AddAllCharsets -H:EnableURLProtocols=http,https --enable-all-security-services -H:+JNI --no-server -H:-UseServiceLoaderFeature -H:+StackTrace _artifactName_-0.0.0-SNAPSHOT-runner
[_artifactName_-0.0.0-SNAPSHOT-runner:54860]    classlist:  17,530.64 ms
[_artifactName_-0.0.0-SNAPSHOT-runner:54860]        (cap):   2,102.24 ms
[_artifactName_-0.0.0-SNAPSHOT-runner:54860]        setup:   3,722.79 ms
08:49:02,254 INFO  [org.hib.Version] HHH000412: Hibernate Core {5.4.10.Final}
08:49:02,281 INFO  [org.hib.ann.com.Version] HCANN000001: Hibernate Commons Annotations {5.1.0.Final}
08:49:02,325 INFO  [org.hib.dia.Dialect] HHH000400: Using dialect: org.hibernate.dialect.PostgreSQL95Dialect
08:49:02,377 INFO  [org.hib.val.int.uti.Version] HV000001: Hibernate Validator 6.1.0.Final
08:49:03,462 INFO  [org.jbo.threads] JBoss Threads version 3.0.0.Final
Warning: RecomputeFieldValue.FieldOffset automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.objectFieldOffset(Field) was detected in the static initializer of protostream.com.google.protobuf.UnsafeUtil. Detailed failure reason(s): The argument of Unsafe.objectFieldOffset(Field) is not a constant field., Could not determine the field where the value produced by the call to sun.misc.Unsafe.objectFieldOffset(Field) for the field offset computation is stored. The call is not directly followed by a field store or by a sign extend node followed directly by a field store.
Warning: RecomputeFieldValue.FieldOffset automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.objectFieldOffset(Field) was detected in the static initializer of protostream.com.google.protobuf.UnsafeUtil. Detailed failure reason(s): The argument of Unsafe.objectFieldOffset(Field) is not a constant field., Could not determine the field where the value produced by the call to sun.misc.Unsafe.objectFieldOffset(Field) for the field offset computation is stored. The call is not directly followed by a field store or by a sign extend node followed directly by a field store.
Warning: RecomputeFieldValue.FieldOffset automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.objectFieldOffset(Field) was detected in the static initializer of com.google.protobuf.UnsafeUtil. Detailed failure reason(s): The argument of Unsafe.objectFieldOffset(Field) is not a constant field., Could not determine the field where the value produced by the call to sun.misc.Unsafe.objectFieldOffset(Field) for the field offset computation is stored. The call is not directly followed by a field store or by a sign extend node followed directly by a field store.
[_artifactName_-0.0.0-SNAPSHOT-runner:54860]     analysis:  24,630.89 ms
Error: Classes that should be initialized at run time got initialized during image building:
 io.netty.channel.epoll.Epoll the class was requested to be initialized at build time (from the command line and from feature io.quarkus.runner.AutoFeature.beforeAnalysis).  To see why io.netty.channel.epoll.Epoll got initialized use -H:+TraceClassInitialization
com.oracle.svm.core.util.UserError$UserException: Classes that should be initialized at run time got initialized during image building:
 io.netty.channel.epoll.Epoll the class was requested to be initialized at build time (from the command line and from feature io.quarkus.runner.AutoFeature.beforeAnalysis).  To see why io.netty.channel.epoll.Epoll got initialized use -H:+TraceClassInitialization
    at com.oracle.svm.core.util.UserError.abort(UserError.java:65)
    at com.oracle.svm.hosted.classinitialization.ConfigurableClassInitialization.checkDelayedInitialization(ConfigurableClassInitialization.java:494)
    at com.oracle.svm.hosted.classinitialization.ClassInitializationFeature.duringAnalysis(ClassInitializationFeature.java:188)
    at com.oracle.svm.hosted.NativeImageGenerator.lambda$runPointsToAnalysis$8(NativeImageGenerator.java:711)
    at com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:63)
    at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:711)
    at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:526)
    at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:444)
    at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Error: Image build request failed with exit status 1

[hpehl]

Native compilation for the quarkus-grpc-client extension does not work atm. I haven't looked at this in much detail yet. As soon I have time, I'll try to provide a fix though.

[sean-scott-lr]

Native compilation for the quarkus-grpc-client extension does not work atm. I haven't looked at this in much detail yet. As soon I have time, I'll try to provide a fix though.

@hpehl Thanks. I was trying to get it working myself but not succeeding. As you can see above I moved a bunch of classes around EPoll to Runtime init... but I have yet to succeed.

[gustf]

Hi there, any info if and when this will be prioritized/implemented?

[emmanuelbernard]

Paolo and Clément have been working on it in the past few weeks. If you
want preview it, it should not be loo long

On Fri 10 Apr 2020 at 16:49, Gustaf Andersson [e-mail]
wrote:

Hi there, any info if and when this will be prioritized/implemented?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/quarkusio/quarkus/issues/820#issuecomment-612061982,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACJNWGNWSWIH2WYV2Y45ZLRL4WYNANCNFSM4GWCN6VQ
.

[gustf]

Sounds awesome! Thanks for info!

[cescoffier]

@gustf The work is done in https://github.com/quarkusio/quarkus-grpc.
It supports native, TLS, mutual auth, interceptors (even if I'm still wondering what would make the most sense for these), all types of interactions and is integrated with Mutiny.

[gustf]

@cescoffier Cool!

[boosey]

@cescoffier I've cloned the repo and run mvn clean install, but there is a build error:

[ERROR] Some problems were encountered while processing the POMs:

[ERROR] Non-resolvable import POM: Could not find artifact io.quarkus:quarkus-bom:pom:999-SNAPSHOT @ line 38, column 25

[ERROR] Non-resolvable import POM: Could not find artifact io.quarkus:quarkus-bom-deployment:pom:999-SNAPSHOT @ line 45, column 25`

[cescoffier]

At the moment, the master branch depends on the master branch of Quarkus. So you will need to build Quarkus first.

To get it running with Quarkus 1.3.2.Final, run:

# Checkout and build Quarkus
git clone https://github.com/quarkusio/quarkus.git 
cd quarkus
mvn -B clean install -DskipTests -DskipITs -DskipDocs

# Update Quarkus version to release
cd ..
mvn versions:set-property -Dproperty="quarkus.version" -DnewVersion="1.3.2.Final" -DgenerateBackupPoms=false

# Build Quarkus gRPC
mvn -B clean install

[gaetancollaud]

@cescoffier

The work is done in https://github.com/quarkusio/quarkus-grpc.

Did you move the repository ? I cannot find any repo having "grpc" as title in the quarkusio project. :frowning_face:

Edit: Found it here: https://github.com/quarkusio/quarkus/tree/master/extensions/grpc Sorry for the stupid question.

[emmanuelbernard]

Should we close that issue @michalszynkiewicz ?

[michalszynkiewicz]

I think so @emmanuelbernard

[michalszynkiewicz]

We have a gRPC extension already merged, most of the listed items are already resolved, the ones that are left are tracked in different issues. Closing