Telegraf: Feature Request: Webhooks plugin supporting events published by particle.io

[DazWilkin]

Feature Request

particle.io devices support the publication of device events (e.g. sensor output) through Particle Cloud. These may then trigger Webhooks against 3rd-party services.

I'm interested in IoT, Telegraf and InfluxDB and thought it would be educational to build a Webhooks sample for Telegraf to consume particle events.

Is there any interest (other than mine) in the result?

Here's particle.io dashboard showing events being published from a device and submitted to Webhooks:

Here's the result of the device's events being published by a Webhook to Telegraf onto InfluxDB and viewed with Grafana all running as containers on Google Cloud Platform

Accessible, of course, through InfluxDB:

Proposal:

If there's interest, I'll publish what I have to my GitHub and reference it. I'm interested in feedback on how it could be improved.

Current behavior:

It works for the very simple case of a single 'data' point being published by the particle device. Particle supports arbitrary complexity in the Webhook payload and it would be interesting to explore how this could be implemented.

Of course, as an IoT device, the other dimension of interest would be to have multiple|many particle devices submitting event data.

Desired behavior:

Undetermined.

Use case: [Why is this important (helps with prioritizing requests)]

It's unimportant.

It provides another plugin for Telegraf.

My intent was intellectual curiosity. If there's interest, I'll publish. If not, I've satisfied my own intent.

Full disclosure: Am a Googler working in Cloud but have pursued this primarily for intellectual curiosity.

[francois2metz]

Please publish ;) Also I would love to have some feedbacks on the webhooks plugin.

[DazWilkin]

Excellent. Thanks for the support!

My day job has intervened but I'll get this done before the end of the weekend.

Honestly, the WebHooks plugin is very straightforward. My challenges were:

1. Debugging and testing the structure of particle.io's incoming WebHooks requests. I used a combination of requestbin, putsreqs and my own JavaScript code (which took me on a misleading detour into CORS)
2. Debugging and testing my plugin's parser/responses. particle.io submits x-www-form-urlencoded (by default) rather than JSON and so I learned Gorilla Schema (==excellent).

To make life more enjoyable (!) I'm also deploying Telegraf, InfluxDB and Grafana in containers and have been building Telegraf in the container (don't ask ;-))

[francois2metz]

FYI, I had to parse _x-www-form-urlencoded_ too in the mandrill plugin (not yet merged) https://github.com/influxdata/telegraf/pull/1408/files#diff-f08c5c4d18cd63d40f23d72d821b837eR39

Look at the tests too.

After writing tests and code, I use ngrok to make sure that it's okay.

[DazWilkin]

OK

I created DazWilkin/telegraf-particle-webhook-plugin repo.

I'm working on a blog post that documents an end-to-end solution that incorporates this to show folks how to get their Photon events published to it, stored in InfluxDB and viewed through Grafana. When the post's ready, I'll link it here too.

Feedback always welcome!

[francois2metz]

It looks good. Do you plan to create the pull request?

[sparrc]

@DazWilkin Can you submit that as a Telegraf plugin? I would rather if you're blog post could advise simply downloading and installing telegraf rather than having to build from source :)

[danielnelson]

Support for particle webhooks was added in #3477

[towynlin]

Thanks from a Particle founder!
Particle ❤️ Telegraf

[davidgs]

I'm working on a doc for Particle that will show exactly how to use this new Plugin (since I wrote it :-) ) and should have that soon!