vertx-web-openapi's RouterBuilder implementation doesn't find $refs external to the specification's directory

[lggmonclar]

Version

vertx-web-openapi:4.0.2

Context

Attempting to create a RouterBuilder using an open api spec that has $refs contained outside of the spec's directory always fails to load, even though the files are available in the classpath.
Example, given the following project structure:

project/
└── src/
    ├── main/
    └── resources/
        ├── openapi/
        │   ├── openapi-spec.yaml
        │   └── common/
        │       └── foo-ref.yaml // ok
        └── bar-ref.yaml // error   

openapi-spec.yaml:

openapi: 3.0.0
info:
  version: 1.0.0
  title: "My service"
servers:
  - url: /
paths:
  /foo:
    $ref: "common/foo.yaml" #would like to reference it using the classpath's location `openapi/common/foo.yaml`
  /bar:
    $ref: "../bar-ref.yaml" #same as above -- would like to use `bar-ref.yaml`

Attempt to load the spec:

RouterBuilder.create(vertx, "openapi/openapi-spec.yaml");

Looking at the code, this seems to be caused by this part of the library: https://github.com/vert-x3/vertx-web/blob/0000c488fae303943817934f09b1ffaf9ab05c0c/vertx-web-openapi/src/main/java/io/vertx/ext/web/openapi/impl/OpenAPIHolderImpl.java#L75-L77

Using the example above, the implementation is including everything under the openapi directory in the vertx temp file cache, and all subsequent reference lookups are using the cache as the base location to search for the resources, instead of searching for them in the classpath.

For our projects, we are getting references from different places in the classpath, since we are sharing references across many services that are imported from a single common module. This worked as expected when loading the specs using the swagger-parser dependency in vertx-web-api-contract.

Do you have a reproducer?

https://github.com/lggmonclar/openapi-refs-mwe

The micro-service module depends on common, which includes its resource files in the classpath.

micro-service starts the verticle and attempts to create the RouterBuilder with an openApi spec containing a $ref that can be found in common.

[MeYoGui]

Hello, same situation here and it's blocking us to move to the new module actually.
I've been trying to find workarounds but nothing yet.

[tsegismont]

@slinkydeveloper can you please take a look?

[slinkydeveloper]

Update: I've managed to reproduce the issue here https://github.com/vert-x3/vertx-web/pull/1884, now I'm trying to figure out how to fix it :smile:

[slinkydeveloper]

The PR https://github.com/vert-x3/vertx-web/pull/1884 should be ready and should fix the issue, @MeYoGui @lggmonclar can you check it out and, if you can, try the patch in your codebases? To try the patch: checkout my branch, run mvn install -DskipTests and then in your projects add the dep to 4.0.3-SNAPSHOT

[lggmonclar]

Hi @slinkydeveloper, I've tested a bit further and I see that with your changes, OpenAPIHolderImpl doesn't search for the reference only in the file cache anymore, which is good for the case where the reference is in the same module, but now the issue is that the file loader it is assuming that the file is in that module's resources at all, which isn't always the case.

My issue (and the one I've linked in the minimum working example), is that I have references that are included in the classpath by other modules -- In my example, the micro-service module is referencing an operation in the common module. That case is still not working.

Keep in mind that this was supported by swagger-parser because it doesn't use absolute paths for resolving files.

Were you able to run my MWE? Please let me know if there's anything else I can do to help you resolve the issue.

[slinkydeveloper]

but now the issue is that the file loader it is assuming that the file is in that module's resources at all, which isn't always the case.

I don't think so, because vert.x file loader is able to try searching first in the classpath, then in the local FS, here is a test: https://github.com/vert-x3/vertx-web/blob/master/vertx-web-openapi/src/test/java/io/vertx/ext/web/openapi/impl/OpenAPIHolderTest.java#L100

Were you able to run my MWE? Please let me know if there's anything else I can do to help you resolve the issue.

TBH I didn't had much time, I'm sorry, I'll try to do it soon. I've used your repro to extract a test case which exactly looks like yours: there are two jars, one has the spec, the other a referenced schema, and both are loaded in the test classpath:

• https://github.com/vert-x3/vertx-web/blob/master/vertx-web-openapi/src/test/java/io/vertx/ext/web/openapi/impl/OpenAPIHolderTest.java#L444
• https://github.com/vert-x3/vertx-web/tree/master/vertx-web-openapi/src/test/resources/jars
• https://github.com/vert-x3/vertx-web/blob/master/vertx-web-openapi/pom.xml#L149

[lggmonclar]

I've used your repro to extract a test case which exactly looks like yours: there are two jars, one has the spec, the other a referenced schema, and both are loaded in the test classpath:

I've gone through the new included tests for vertx-web and indeed it works in that case, but the test case is not exactly like our project. There's seems to be one key difference:

We aren't loading the initial spec from a .jar file, which means that the scheme of the scope for building the solvedURI is not a jar, but rather a file here: https://github.com/vert-x3/vertx-web/blob/master/vertx-web-openapi/src/main/java/io/vertx/ext/web/openapi/impl/OpenAPIHolderImpl.java#L190

Which ends up producing the wrong extracted path here: https://github.com/vert-x3/vertx-web/blob/master/vertx-web-openapi/src/main/java/io/vertx/ext/web/openapi/impl/OpenAPIHolderImpl.java#L320

Worth mentioning:

• At runtime we are using a fat jar (shadow jar). In which case everything does work because everything has the same scope.
• In the development environment however, we are using a Gradle "project" dependency. In this case it doesn't work

[slinkydeveloper]

We aren't loading the initial spec from a .jar file, which means that the scheme of the scope for building the solvedURI is not a jar, but rather a file here: https://github.com/vert-x3/vertx-web/blob/master/vertx-web-openapi/src/main/java/io/vertx/ext/web/openapi/impl/OpenAPIHolderImpl.java#L190

Ahhh I see, so in your case, the "root" openapi is in the classpath but not in a jar, while the child is in a jar. right? I'll try to develop another test to reproduce exactly this case :)