Logstash: [Plugin submission] Logstash-filter-wurfl plugin

[andreacastello]

Hello, I'm opening a plugin submission request, according to this page instructions: https://www.elastic.co/guide/en/logstash/current/submitting-plugin.html#_contributing_your_source_code_to_logstash_plugins_5

The plugin I wrote adds WURFL device data to a Logstash event data.

The plugin is written in Java, you can find the source code here.

[kares]

Thanks Andrea, the repository seems non existent.

[andreacastello]

Thanks Andrea, the repository seems non existent.

Hi Karol, we made the repo public. Can you try again? Thanks.

[kares]

Hello Andrea, appreciate your submission.
The docs isn't up-to-date, the team paused pushing all plugins users create under the plugins org.

There simply is only that much the LS owners can handle (e.g. we'd like to make reviews for new plugins as a minimum).

For discovery on GH you can tag your repo with logstash, logstash-plugin, wurfl or whatever it relates to.

UPDATE: The preferred way these days is that we list user plugins in our docs - linking to your repo directly instead of having to mantain a repo on the logstash-plugins repo.

Seems this is a manual process for now, would you be interested in that (your plugin listed in docs - similar to this one) ?

[andreacastello]

@kares
Hi Karol, thanks for the answer. The option to keep the plugin in our public repo and list it on your page sounds good to me. I guess the documentation should be provided in markdown format, am I right?

[andreacastello]

@kares

Hi, I've added to our repository the documentation for your listing page.
You can find it here: https://github.com/WURFL/logstash-filter-wurfl/blob/master/listing_page/listing_page_doc.asciidoc

I used asciidoc format. Please let me know if it's OK or you need some modification.

[andreacastello]

Hey, @kares Any news about the listing? In logstash plugin page?
Thanks.

[andreacastello]

Hello, @kares sorry for being insistent, but I'd need a feedback on the logstash plugin listing on Logstash plugin page. Thanks

[kares]

sorry I was out for several weeks and since the process is manual I will need some help from the docs folks.

[kares]

@karenzone would need some help looking at potential content for the 3rd plugins listing page like this one

note-to-self: someone might also need to review the (Java) plugin's repository for some base-line checks ...

[andreacastello]

Hi @kares no worries. Thanks for your help.

[andreacastello]

Hi @kares any news on this?

[kares]

Hey Andrea, please take a look at the docs as I do not see the plugin released this is needed for users to be able to bin/logstash-plugin install logstash-filter-wurfl.

[andreacastello]

Hi @kares , thanks for the heads up! I released it now. (version 0.3.2)

[kares]

Thanks, I brought this to the team's attention and we realized LS' external plugin listing process isn't formalized.
Thus more work to-do on our end to come up with something for anyone wanting to list a plugin (a kind of "quality-control").

[andreacastello]

Thank you for your answer, @kares. Can you please advise on the next steps?
What should we do to get our plugin listed?

[karenzone]

Hi @andreacastello We haven't forgotten your request! As Kares said, this is a new process for us, and we're working out the details to make sure that it's scalable. I hope to have an update for you soon.
cc:/ @roaksoax

[roaksoax]

Hi @andreacastello

First let me thank you for submitting your plugin for inclusion to our ecosystem. I also wanted to apologize for the delay, but as we were reviewing the submission process, we noticed some information was missing. That said, as we were discussing improvements to our documentation, we noticed a few things were missing in your codebase that need to be addressed before we can add your plugin. These are.

• [x] Add missing files (at least the 'logstash-filter-wurlf_device_detection.gemspec' file). I'd suggest you use our Plugin Generator to figure out the recommended organization of the codebase and missing files (if any apart from the gemspec), but at the very minimum, the gemspec should be available. An example can be found here

• [ ] Integrate with Logstash Travis CI. This is OPTIONAL step but HIGHLY recommended. This would give the community confidence that your plugin is being actively tested, as well as give us confidence to list your plugin in the official documentation. If you decide to add this, you'd need to:

  • Enable travis on your account.
  • Import our standard travis.yaml as shown in this example
  • Ensure your 'specs' tests are in the spec folder.

• [x] Match naming convention with source repository. This is an OPTIONAL step and a recommendation. While the repository is called 'logstash-filter-wurlf' the released gem is 'logstash-filter-wurlf_device_detection'. While there's no actual issue in doing so, I'd recommend consistency between the codebase and the released gem. Either way, the gemspec has to match the actual gem as per the first point.

Once all the tasks above are completed, @karenzone will make the necessary changes in our docs to include your plugin.

Thank you again, and please let me know if there's anything we can do to help.

/cc @karenzone @jsvd

[andreacastello]

Hi @roaksoax
Thanks for the detailed explaination. I'll try to make the plugin compliant with your requests ASAP. I'll update here, when it'll be ready or if I have any question about it.

[andreacastello]

@roaksoax I've added the gemspec file. I've also renamed the repo in order to match the gem file name.
I'm trying to add all other missing files following the structure of the plugin generator project but, please, also notice that I have created the plugin following this specific doc for Java logstash plugins: https://www.elastic.co/guide/en/logstash/current/java-filter-plugin.html

As for the CI, it is more difficult. The unit test are structured to use maven and comply with the standard "Java on Maven/Gradle" project structure, where tests are placed under /src/test/java folder. But, most of all, the test requires a running WURFL Microservice server which is not available to your CI. So, unless it's really mandatory I'd skip it.
I've added the link to the PR on our repo so that you can review the file

[andreacastello]

Here's a list of file generated by logstash plugin generator with an "OK" for each one present in the repo.
Some of the missing files seem not necessary for a Java plugin, but if you think some of them should be there, tell me.
Thanks

Gemfile OK
wurfl_test_spec.rb
spec_helper.rb
logstash-filter-wurfl_test.gemspec OK
docs/index.asciidoc OK
LICENSE OK
README.md OK
CONTRIBUTORS OK
Rakefile
filters/wurfl_test.rb -> no ruby test but java test with junit/testNG
DEVELOPER.md
CHANGELOG.md OK

[roaksoax]

Thank you for addressing all requests @andreacastello . @karenzone will take over and complete the process to list your plugin now that all my concerns have been resolved.

[karenzone]

@andreacastello Your logstash-filter-wurfl_device_detection plugin is now included in the list of filter plugins in the Logstash Reference, and it has its own landing page. Thanks again for submitting this plugin.