Pulsar: Cannot create Pulsar-IO connector with nar maven plugin

[itaross]

Describe the bug
Ni-fi maven plugin is not copying /resources/META-INF/services/pulsar-io.yaml
Pulsar cannot load the connector because a yaml file is not inside the .nar archive

To Reproduce
Steps to reproduce the behavior:

1. Clone an existing pulsar-io connector
2. mvn compile
3. Use nar plugin to generate .nar
4. Copy the nar under connectors folder
5. Type the command: pulsar-admin sink reload
6. Type the command: pulsar-admin sinks available-sinks

Expected behavior
Should see the plug-in available

Screenshots

Desktop (please complete the following information):

• macOS

Additional context
Add any other context about the problem here.

[itaross]

Ok sorry @sijie you can close this.
I have found the solution.
I needed to add this inside the build tag in pom.xml:

  <resources>
            <resource>
                <directory>src/main/resources/META-INF/services</directory>
                <targetPath>META-INF/services</targetPath>
            </resource>
        </resources>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-resources-plugin</artifactId>
                <version>3.1.0</version>
            </plugin>

[sijie]

Thank you @itaross

[itaross]

I do know if it is too specific, but I'd love too see it in the official doc here: https://pulsar.apache.org/docs/en/io-develop/

[sijie]

@itaross it seems like a bug. Do you want to contribute a fix?

[itaross]

@itaross it seems like a bug. Do you want to contribute a fix?

I am not quite sure it is a bug from Pulsar side.
The Pulsar API expects to find "pulsar-io.yaml" inside the .nar archive, but when building it is not copied under target/classes/META-INF/services.
From Ni-Fi plugin side I understand that they could not be aware that it is required to copy that pulsar-io.yaml. Yesterday I digged down a bit and I have found that on apache/nifi-maven there is this method:
https://github.com/apache/nifi-maven/blob/b81db233a200847998a57ac9b1f5b6218ab66a1b/src/main/java/org/apache/nifi/extension/definition/extraction/ExtensionDefinitionFactory.java#L104
I am not quite sure if it is referred to the target folder or resources under src.

Anyway, the use of that maven goal resources:resources I came up with overcomes the missing file in the target directory.
So a correct maven pipeline is:

mvn clean compile resources:resources nifi-nar:nar

I don't know if this is really a bug or not, if you can suggest one I can try to fix it.
For now I think the first action is to update the documentation with the use of this resources plugin for maven.

PS. I wonder how people wrote connectors until now. I discovered that one of my colleagues was copying the file manually inside the folder before using the nar plugin :sunglasses: cheeky move :sunglasses: