Quarkus: MongoClientBuildItem is never produced

[jamesnetherton]

Describe the bug
MongoClientProcessor has a build method which is supposed to produce MongoClientBuildItem(s) when the relevant Quarkus Mongo configuration is present. I never see the fields in MongodbConfig be anything other than null, hence the build items never get created.

Having MongoClientBuildItem available, is useful for other extensions like the Camel Mongo extension, where it can fetch the client and wire it into the relevant Camel component.

Expected behavior
If any of the Mongo configuration options are set, E.g quarkus.mongodb.hosts or quarkus.mongodb.connection-string then a MongoClientBuildItem should be produced.

Actual behavior
No MongoClientBuildItem ever seems to be produced, regardless of the Mongo configuration used.

To Reproduce
Steps to reproduce the behavior:

1. In your IDE, set a breakpoint in MongoClientProcessor at the start of the build method
2. Launch test BookResourceTest in debug mode
3. Observe conditions for creating MongoClientBuildItem are never satisfied

[cescoffier]

/cc @loicmathieu

[loicmathieu]

/cc @dufoli and @geoand I remember you discussed about it ... It may be a regression due to the inclusion of multi-tenant MongoDB ?

[geoand]

@lburgazzoli is this an issue you are seeing in Camel? I thought that https://github.com/quarkusio/quarkus/pull/7063 fixed it

[lburgazzoli]

https://github.com/quarkusio/quarkus/pull/7063 was more to have it semantically fixed, I didn't check if it was working on quarkus side to be honest

[geoand]

OK thanks, I'll look into it

[geoand]

This is a problem TBH... We can't know at build time how many MongoClientBuildItem there will be... How important is this build item for Camel?
I have never liked it personally and with this multi-mongo support is actually totally falls apart...

[lburgazzoli]

we do consume it at RUNTIME_INIT: https://github.com/apache/camel-quarkus/blob/master/extensions/mongodb/deployment/src/main/java/org/apache/camel/quarkus/component/mongodb/deployment/MongoDbProcessor.java#L40-L59

it is not uber important but we do try to reuse whatever quarkus produces so we don't have to have yet another way to configure the same "concept"

[geoand]

The problem is that now we can't really support it... The only reason something similar works in Agroal is that in that case there is also an accompanying build time configuration that can be used to determine how many (and which) datasources are needed...

In the mongo case all the configuration is runtime so it's not possible to even know how many clients there will be...

[geoand]

@gsmet FYI

[geoand]

We can do the following however:

We can read the configuration at build time to pick up the connection names and then at runtime make sure that the same connection named were used.
That way we would be able to produce the necessary build items and they would correspond to the the connections that will be used at runtime.

How does that sound?

[loicmathieu]

@geoand in fact I'm facing the same issue when I try to integrate multi-tenant MongoDB inside Panache.

The current implementation looks for the named MongoDB at build time via the usage of @MongoClientName and for Panache, we should not mandate the usage of the annotation.
I try to update the current code to look at the different Panache client based on the configuration but I didn't success in it ...

I think we need the same aproach as Agroal : to have a build time config for the list of MongoDB client and a runtime configure for the configuration details of those clients.

[geoand]

I think we need the same aproach as Agroal : to have a build time config for the list of MongoDB client and a runtime configure for the configuration details of those clients.

We can't really do that in a natural way here, since there is nothing inherently build time about the mongo config - in Agroal's case the driver is build time only so it's a natural fit.

So what I propose is that we read the Mongo configuration at build, extract only the connection name and use that to build the MongoClientBuildItem and setup a runtime check to enforce that the connection names are the same as were used at build time.
Does that make sense?

If we agree on that, I can have it ready today

[loicmathieu]

@geoand it makes sense. I'm not sure it will fix my issue as I need a way to be able to build the different MongoDB client based on the configuration and not the annotation but it's a step in the right direction :)

[geoand]

I was thinking of generating a MongoConnectionNameBuildItem that will contain the connection name. Would that be enough for you?

We can't really know anything more about the configuration at build time

[loicmathieu]

It will not be enought for me as, if I didn't use any @MongoClientName annotation the MongoClient will not be built because of this: https://github.com/quarkusio/quarkus/blob/master/extensions/mongodb-client/deployment/src/main/java/io/quarkus/mongodb/deployment/MongoClientProcessor.java#L165

Maybe it's the same issue that Camel is facing and that cause the MongoClientBuildItem to be null ?

[geoand]

Yeah most likely. Let me create a draft PR soon with what I have in mind and then you and the Camel folks can try something concrete and see if it's fits your needs :)

[geoand]

It seems like this opens up a rabbit whole... I'll try to figure it out ASAP

[geoand]

@loicmathieu @jamesnetherton @lburgazzoli please try out #7385 and let me know. If it works for you I'd like to get it in as soon as possible

[jamesnetherton]

Just tested #7385 with camel-quarkus. It looks like it can work, I now see that MongoClientBuildItem is available :+1:

However, I can't fully prove its working because I get an NPE in MongoClientRecorder.getClient(). For some reason there are no MongoClientProducer instances in the bean container.

[geoand]

I am pretty sure that the only way that can happen is if the MongoClient bean is not being used anywhere. Would you mind sharing your test?

[dufoli]

hello, just see the ping. @geoand why based the generation on parsing of config planned for runtime during buildtime. it seems odd to me. Can we be based on annotation like createMongoClientProducerBean

[dufoli]

just see that loic have same remark ;-)

[geoand]

I'm thinking about it, so far I am leaning on: no I won't do it with the annotations :)

The reasoning in simple: It is almost certain that at build time you will have at least one config property for the each connection.
Furthermore, if we do it with the annotations I will also have to take into account meta-annotations, cases where @Inject isn't needed at all (see constructor injection for example) and who knows what else.
The config solution seems more intuitive to me, closer to the actual problem and more fail-proof. Of course all this remains to be seen in real world usage :)

[jamesnetherton]

Here's the test code:

https://github.com/jamesnetherton/camel-quarkus/tree/mongo-test/integration-tests/mongodb/src/test/java/org/apache/camel/quarkus/component/mongodb/it

And the extension:

https://github.com/jamesnetherton/camel-quarkus/tree/mongo-test/extensions/mongodb

You wont find any @Inject of the client in the test case. It's supposed to be auto discovered by Camel via the CamelRuntimeBeanBuildItem that's created in MongoDbProcessor.

This did work fine at one point. Maybe with some of the recent changes to the Quarkus Mongo extension, it is no longer possible.

I can try to look at working around the problem if there's no easy solution.

[dufoli]

interesting point. @geoand. ;-)
Not sure about the fail proof because if properties in config file is not setup during build time it will not worked either.

Anyway, annotation is used for generated the named Mongo client producer. So if you are based on config and it remain like that it is not consistent and you can have a config without producer and reverse.

So, do we have to switch the createMongoClientProducerBean to use config too ?

Maybe we have to introduce an intermediary build item with name of MongoClientBuildItem which will be generated from annotation (without client just name) and configuration.

[geoand]

I'll take another look tonight and tomorrow and let you know

[geoand]

OK, then it looks like we used to make the Mongo beans unremovable. I'll make that happen again although I don't the solution that much :)

[geoand]

@jamesnetherton I just pushed a new iteration of the PR.

Everything should now be back to normal from your POV. I would appreciate it if you gave a go

[geoand]

@loicmathieu, @dufoli your concern has been addressed as well.

With the current version of the PR we all get what we want:

• I get the fact that the beans are not un-removable by default. They only become un-removable when a build step actually consumes the MongoClientBuildItem (which doesn't happen by default - Camel is only consumer I know of).
• Camel get a working version of this build item
• Loic and Olivier get the implementation they wanted

So we should all be happy with this iteration :smile:

[dufoli]

@geoand great !

[jamesnetherton]

@geoand It works! Thanks :1st_place_medal:

[geoand]

Thanks for checking!

@gsmet can we merge the PR in order to unlock Camel?

[geoand]

s/unlock/unblock