Schema-registry: http://packages.confluent.io/maven is not browsable which causes problems with proxying via a local nexus

[trentsummerfield]

I've had trouble at work the last few days gaining access to http://packages.confluent.io/maven via a local nexus repository. The problem is that http://packages.confluent.io/maven is not browsable/indexed. This means when you add it as a 3rd party repository to a local instance of nexus, none of the artifacts are discovered and cached locally. This will cause builds to fail as they are unable to resolve confluent dependencies.

You can work around this issue by manually adding the confluent repository to a globally whitelisted group. However our security team would rather we didn't have to whitelist a whole repository.

It seems that this has been an ongoing source of confusion for people based on my google searching (see https://groups.google.com/forum/#!topic/confluent-platform/efHQvLFKRKk).

This could be solved by making http://packages.confluent.io/maven browsable. Is there a reason that this hasn't been done?

[DennisD2]

11 thumbs up means a whole community requires a solution here. My company accesses all external repos via Nexus and Nexus cannot access a non-browsable maven repo.

In summary, the Confluent maven repo seems unusable for all people behind a Nexus proxying external repos, which is AFAIK a very common case.
So what is the problem in making the maven repo browsable?

[z4f1r0v]

Just to keep this alive. Is this prioritized?

[rumatoest]

You know that in some companies security assessment would not allow to use such repositories at all.
BTW Artifactory is also not able to proxy packages from packages.confluent.io/maven

Could you please fix it or just publish your artifacts to jcenter as any other developers do.

[rumatoest]

Any updates?

[z4f1r0v]

I don't think that the correct channel to communicate the importance of this is here. More than a year has passed since this thread was opened and nobody from confluent has responded. This is both sad and frustrating.

[rumatoest]

@alexanderzafirov Well you can try tweet to @gAmUssA this man at least will answer your. Or you can send message to my thread where I'm already trolling him about this issue https://twitter.com/rumatoest/status/1120269186209583104

[OneCricketeer]

I work in an environment where we were able to proxy it...

I don't have access to Nexus to tell you how, but I see all download requests definitely coming from it in my Maven output rather than pacakges.confluent.io

[z4f1r0v]

@cricket007 that looks interesting. Can you dig into it so that we can solve this for the community :)

[OneCricketeer]

Yeah, works fine at home

Started up https://github.com/sonatype/docker-nexus3

Added a proxy to https://packages.confluent.io/maven with no other special settings - https://help.sonatype.com/repomanager3/formats/maven-repositories#MavenRepositories-ProxyingMavenRepositories

Updated POM to point at it

    <repositories>
        <repository>
            <id>confluent-proxy</id>
            <url>http://pico2.lan:8081/repository/confluent/</url>
        </repository>
    </repositories>

Updated a version to pull something new that wouldn't be in my local cache

And voilà

[z4f1r0v]

@cricket007 I tried once again following your suggestion. Unfortunately I get nothing but a test.txt file.

[z4f1r0v]

[z4f1r0v]

[leachad]

@trentsummerfield did you ever get a response from confluent about this? I'm really interested in how/if this is being worked on

[trentsummerfield]

I never heard anything from confluent. I've thankfully moved on in my life and have a new job where I no longer have to deal with confluent products.

I'll get in touch with my old team and get someone to comment here if they ever made progress on this. I wouldn't hold my breath though.

[leachad]

I never heard anything from confluent. I've thankfully moved on in my life and have a new job where I no longer have to deal with confluent products.

I'll get in touch with my old team and get someone to comment here if they ever made progress on this. I wouldn't hold my breath though.

Sorry to hear you never found a solution. Here's to hoping I get farther down the rabbit hole than you did 🤞

[daniel-shuy]

Any updates? This is still a valid issue. Currently having to resort to uploading all artifacts to Artifactory manually ☹️.

Anyone knows if Confluent's license permits forking the repository and publishing all artifacts to Maven Central?

[daniel-shuy]

For anyone who needs to upload artifacts manually, you will need:

• assembly-plugin-boilerplate : pom
• build-tools : jar, pom
• common : pom
• common-config : jar, pom
• common-metrics : jar, pom
• common-parent : pom
• common-utils : jar, pom
• kafka-avro-serializer : jar, pom
• kafka-connect-avro-converter : jar, pom
• kafka-json-serializer : jar, pom
• kafka-schema-registry : jar, pom
• kafka-schema-registry-client : jar, pom
• kafka-schema-registry-parent : pom
• kafka-streams-avro-serde : jar, pom
• licenses : jar
• rest-utils : jar, pom
• rest-utils-parent : pom
• rest-utils-test : jar, pom

Note that when uploading the artifacts marked with jar, pom to Artifactory, make sure to upload the POM manually instead of using Artifactory's Generate Default POM/Deploy Jar's Internal POM, as it will generate POMs with the wrong parent.

[NathanNam]

Sorry for the delayed response. We're evaluating a plan to provide indexes for packages.confluent.io., and will get back to you in a few weeks.

[t321]

Any news about the indexes @NathanNam ? 🤞
Thanks.

[gAmUssA]

@t321 @rumatoest could you try if http://packages.confluent.io/maven/ works for you? Make sure you add trailing / at the end.

[cristianoperez]

It's working now 🎉