Openhab-addons: [netatmo] Support Presence Camera

[rklueber]

Expected Behavior

Being able to turn on notification about movements seen dependent on other OH2 variables.

Target is to activate/deactivate the messaging of persons from the camera dependent of another OH2 item. I have a presence variable if at least someone is at home. If this is the case I would like to turn on/off the alarming of persons in the garden via a OH2 rule.

Current Behavior

Netatmo Presence is yet not supported by the v2 of the binding.

Possible Solution

Use the provided API from netatmo to access and modify the properties

Context

Your Environment

OH2, Netatmo Presence camera

[mister78]

I am looking forward for support of the presence camera too.

[lolodomo]

I am currently testing the Welcome camera and I am trying to fix all the little problems I can identify.
Then I will try to add the support for the Welcome Presence. It should even be very easy to add as the API is almost the same between Welcome and Presence.

@clinique: my feeling is that we have mainly to add the specific scopes for Presence camera and think about the few differences between the Welcome and the Presence camera. For example, an event for a Presence camera can include a list of sub-events.

[jannegpriv]

I've made some local tests implementing support for my Presence based on the latest Netatmo 2.x binding and as @lolodomo mentions the Swagger API needs to be updated with support for the sub-event list array that is present for each event.

My local implementation uses c&p from the Welcome implementation adding a NAPresenceHomeHandler and a NAPresenceCameraHandler. A better solution might be to have a more generic CameraHome/Handler that supports both Welcome and Presence.

I'm able to beta-test your Presence implementation since I have a Presence camera (and soon a Welcome camera too).

Regarding the web hook support, is the only way of using the web hook to have my Openhab on a public IP or are there other ways of configuring it? The web hook is the only way as I understand to get immediate openhab events based on the Netatmo events/sub-events.

[lolodomo]

Yes for the webhook, unfortunately.

Is there any difference in your presence class compared to the welcome class ? I initially thought there would be no need of new classes. Or at least an abstract class that will contain everything in common, that is probably 95%.

[clinique]

Yes, for sake of clarity, I think we should have common abstract class, and specifics held by distinct classes.

Yes also regarding webhooks, it need a plublic IP and the tricky thing is that past very few unresponsive responses, you get blacklisted by Netatmo API for 24h. Currently, just to try it, I use a simple script hosted on a public cloud server that redirects to my home server.

[jannegpriv]

Regarding the need to update the Swagger API, is that done by you guys or is it done by Netatmo? It looks like the 'Try It' functions on dev.netatmo.com also uses the Swagger API.

Regrading webhooks, is it possible to somehow integrate the webhook towards myopenhab.org?

[kaikreuzer]

is it possible to somehow integrate the webhook towards myopenhab.org?

That would be a question that @marziman or @digitaldan could answer best.

[clinique]

@jannegpriv : swagger api evolution can be submitted on Github to cbornet repository: cbornet/netatmo-swagger-decl

Integrating webhooks to myopenhab.org could be a good approach, but it not bulletproof neither. This can also be down :(
My feeling is that the current approach of Netatmo is really not appropriate for a reliable notification system outside their own apps.

[jannegpriv]

This can also be down :(

I've not had so much issues with it being down, isn't the new Google Home Action API also integrated towards myopenhab.org? Seems to be a natural way of integrating towards it since it is already there :-) Having the possibility to have almost immediate events in OpenHAB from Netatmo Events would be so useful and cool to have :-)

[lolodomo]

Integrating in myopenhab is not possible as Netatmo has not imagined that an authentication could be important for webhook !
I agree that this webhook architecture is unusable in practice due to security considerations.

[digitaldan]

If my openhab can possibly help, ping me with some of the webhool details

[lolodomo]

@digitaldan : pushing data to myopenhab without any authentication is forbidden, I imagine ?

[digitaldan]

We can support oauth tokens or basic http auth, I don’t know the specifics of how you configure the webhook so it’s hard to know what the options are

[jannegpriv]

The documentation is not so extensive, can you make something out of this API description of adding a webhook?

[digitaldan]

Without knowing more, if they respect embeding the http auth parameters in the URL like "https://username:[email protected]/someRestEndpoint" then that might work. I understand this is passing your credentials around, so not ideal. Also note that myopenhab does support multiple "users" per account, so you could create a user in myopenhab with a complex password and use that (maybe a little better security)

[digitaldan]

also note that you may need to esacpe the @ symbol with %40 when encoding :-)

[lolodomo]

This will at least provide our myopenhab credentials to netatmo !!
For better security, I think we should stop logging this URL. And maybe consider this setting as a password to display stars in Paper UI.

[digitaldan]

nginx would probably log this, I would need to check for a way for it to not.

[nbartels]

Just a short hint. The netatmo webhook sends an additional http header field called X-Netatmo-secret. This contains a hash sum. And it is calculated by the json content that netatmo is sending with the request and the client secret of the netatmo app. Here is an (official) example in PHP and I think this can be ported to Java: https://github.com/Netatmo/Netatmo-API-PHP/blob/master/Examples/Webhook_Server_Example.php

I use something similar using the Facebook Webhooks and because the secret is only known by the user and Netatmo and is not transferred it should be possible to identify the sender. Only Netatmo should be in the position to know this secret and the hash ensures that the content is not modified during the transfer. So we have a authorisation and a content check at once. If the calculated check sum and the one transferred do not match drop the request.

[lolodomo]

@jannegpriv : I would like to add support for Presence camera but I don't understand how I can distinguish Welcome camera from Presence camera in the API.
result.getCameras() returns a type List<NAWelcomeCamera>. There is no abstract class for camera in the API ? No class NAPresenceCamera ?
camera.getType() returns NACamera for a Welcome camera. What does it return for a Presence camera ?

@clinique : I don't understand on what library we rely on for the netatmo API ? I see a file netatmo-java-retrofit-1.1.1.jar in libs. Where does this file come from ? Where is the corresponding source code ?

[lolodomo]

Is the source code coming from https://github.com/cbornet/netatmo-swagger-api/tree/master/samples/netatmo/client/java-retrofit ?

[lolodomo]

Where is the source code of the class NAWelcomeCamera for example ?

[lolodomo]

Or maybe https://github.com/cbornet/netatmo-swagger-api/tree/master/generated/netatmo/client/java-retrofit ? But still no class NAWelcomeCamera.

[nbartels]

@lolodomo I analysed this, too. And it seems @cbornet wrote a swagger json to describe the Netatmo API and uses some tools to autogenerate a java lib from that. So someone has to extend the json file for the presence camera and then a new auto generated lib can be created and afterwards it is possible to improve this openhab2 addon. It's a bit bad we rely on a seldom updated 3rd party library. @lolodomo do you like to open an issue in @cbornet repo?

[lolodomo]

But the swagger json has maybe already all what we need (Presence support) ?
What we need is only a new corresponding jar file ?

[nbartels]

I don't think so, please check here: https://github.com/cbornet/netatmo-swagger-decl/blob/master/spec/swagger.yaml

There is no reference to presence and the "new" presence permissions are missing, too.

[clinique]

@lolodomo : the spec of the Netatmo API is in netatmo-swagger-decl, netatmo-swagger-api is used to generate the according retrofit client. Generated client that is present in the swagger-api repo has not been updated since a while.

[cbornet]

The netatmo-swagger-decl has the definitions for the Welcome camera endpoints which are the same as for Presence. So it should just be about adding the oauth scopes in the security definitions.

[clinique]

@lolodomo : don't know if I skipped it in previous reading but I discovered this in webhooks documentation today : "One valid request resets the failing counter of your application."
This is maybe the solution for a reliable usage of webhooks in events now !

[bruestel]

@clinique I saw that you contributed to https://github.com/cbornet/netatmo-swagger-decl . How did you build the netatmo-java-retrofit-1.1.3.jar? Which arguments did you used with ´swagger-codegen`?

Thanks for you help.

[clinique]

@bruestel : in the generated retrofit netatmo client I use to mvn package to generate the jar file.

[bruestel]

@clinique what command parameters did you used for the generator? Per default it uses okhttp instead of retrofit.

[clinique]

@bruestel : you could use all-netatmo.sh to generate all clients

[bruestel]

@clinique Sorry for asking, but where can I find all-netatmo.sh

[clinique]

In the bin directory of the swagger-api project

[bruestel]

@clinique In the project https://github.com/cbornet/netatmo-swagger-decl should be a all-netatmo.sh?

What I tried so far was to use the swagger.yaml from https://github.com/cbornet/netatmo-swagger-decl to build the jar with https://github.com/swagger-api/swagger-codegen.

At https://github.com/swagger-api/swagger-codegen is no all-netatmo.sh script. I don't see any reason why there should be one - it has nothing to do with netatmo etc.

I'm kinda lost here.

[clinique]

@bruestel : take a look at the sister project netatmo-swagger-api, it relies on netatmo-swagger-decl to generate all clients.

[bruestel]

Thanks :-) Had to modify the netatmo-swagger-api but it worked.

java -jar bin/swagger-codegen-cli.jar generate -i https://raw.githubusercontent.com/cbornet/netatmo-swagger-decl/master/spec/swagger.yaml -l java -o generated/netatmo/client/java-retrofit --library retrofit --artifact-id=netatmo-java-retrofit

[bruestel]

@clinique do you remember which swagger-codgen-cli.jar version you've used? For me it only works with 2.3.0 and 2.3.1. Any version below gives me errors. I also had to change the url paths of swagger_access_token.json and swagger.yaml in the all-netatmo.sh script to get it running.

So far I've managed to build a client. The strange thing is, that some generated methods differ from the latest /org.openhab.binding.netatmo/lib/netatmo-java-retrofit-1.1.3.jar. For example:
Latest: module.getIsLocal() My version: module.isIsLocal();

My current goal is to generate a compatible client myself. This should be my starting point to contribute to this openhab module.

Thanks for your help.

[cbornet]

Yes there's been a change on how the Boolean getters are generated (wrong in my opinion since the Java beans contracts allows isXXX only for the primitive type, not the wrapper one). So you have 2 solutions : generate with the same version that was used previously or use openapi-generator for which the last version should have an option to choose between is or get getter.

[rklueber]

Hello,

I wanted t give it a 2nd try. I was able to get HomeID and Mac of camera from he netatmo Dev Website

I added the thing

Thing NACamera  garten   [ id="70:...", parentId="5a..." ]
Thing NACamera  einfahrt [ id="70:...", parentId="5a..." ]

Both things are being created, but offline

Status: OFFLINE Pending parent object initialization

Any hints?

UPDATE

The Home Thing was missing. I added it. But the cameras are not working.

Is this the expected behavior?

[Bagunda]

I don't understand. Binding for the camera there?

[giejay]

Any updates?

[Stefan-83]

any updates?

[Novanic]

I will give it a try again. I'm a Java developer/architect with 15 years of experience, maybe I have some luck. ;-) And I'm planning to order also the Netatmo Doorbell which I would also like to see fast integrated in OpenHab... ;-) Is the source code of your attemps somewhere available?

[nbartels]

You should have a look at https://github.com/cbornet/netatmo-swagger-api and build the retrofit jar (https://github.com/cbornet/netatmo-swagger-api/blob/master/samples/netatmo/client/java-retrofit/pom.xml)

[Novanic]

Yeah, I got it running. ;-) The Presence is now found and I can successfully create the image item. But I didn't tried to receive events yet, maybe there are still problems. It's just a first version where the device can get found.

You can find the changes here: https://github.com/Novanic/openhab-addons/tree/3059-Support-Presence-Camera

The swagger api wasn't the problem, it seems to be still valid and compatible with the welcome camera structure.

[pgruetter]

Great, I'm glad that someone is taking the time to work on this again. Would love to see the Presence properly integrated. Will check out your branch and see if I can help testing it. Would particularly be interested in getting events in Openhab2 too (like when person is seen I can react to that).

[Novanic]

It seems that the events are also available without configuring a webhook, but with a few minutes delay. This works because there is already a job which permanently refreshes the home data and this home data endpoint returns also the last 30 events. The additional presence event data is also available (human / animal / vehicle), but it is required to update and extend the swagger spec inclusing retrofit mapping, therefore there are dependent projects which require an update which makes it more complicated.

I suggest the following:
1.) I will create a pull request to support the Presence camera. This is just a composition of small changes without depending projects and all owners of a Presence camera will notice that OpenHab will discover the Presence camera and it can already get integrated and used (accessing the live picture, etc.).
2.) We create a new issue for the event support for the Presence camera. This can get fixed separately and will take a few more time and coordination.

Here is the pull-request: https://github.com/openhab/openhab-addons/pull/7660

[Owlbertz]

Hi @Novanic, great to see that there is finally something happening with this.
Any idea when you can look into the event support? Is there any way I can help you, e.g. do some testing?

[Novanic]

Hi @Novanic, great to see that there is finally something happening with this.
Any idea when you can look into the event support? Is there any way I can help you, e.g. do some testing?

I'm also already working on it, but it will take a few time and I have no much time. ;-) I hope that I can send pull requests next weekend (for 3 projects ...). I'm also interested in getting events, because that allows more intelligent notifications (to only get notifications when it is really important/interesting).

[nokyyz]

Hi @Novanic , it is a great news !
Does your pull request also to stop the camera monitoring and set the floodlight ?

[Novanic]

Hi @Novanic , it is a great news !
Does your pull request also to stop the camera monitoring and set the floodlight ?

No, unfortunately the official Netatmo API doesn't seem to provide a solution to switch the floodlight. :-( We could try to ask Netatmo to extend the API.

[nokyyz]

No, unfortunately the official Netatmo API doesn't seem to provide a solution to switch the floodlight. :-( We could try to ask Netatmo to extend the API.

There is a solution to stop/resume camera monitoring, to set the floodlight mode and to set the floodlight intensity !

To stop camera monitoring, run the Http request:
http://<netatmo presence camera ip address>/<vpn hash code>/command/changestatus?status=off

To set the floodlight mode in "auto" mode, run the Http request:
http://<netatmo presence camera ip address>/<vpn hash code>/command/floodlight_set_config?config={%22mode%22:%20%22auto%22}

To set the floodlight intensity to 50%, run the Http request:
http://<netatmo presence camera ip address>/<vpn hash code>/command/floodlight_set_config?config={%22intensity%22:%20%2250%22}

To get the <_vpn hash code_>:
1- Enter this URL into your web browser: https://dev.netatmo.com/apidocumentation/security#gethomedata
2- Click "Try it out" button
3- In Server Response, search the "vpn_url" field of your NOC camera
E.g:

...
"cameras": [
{
"id": "xx:xx:xx:xx:xx:xx",
"type": "NOC",
"status": "on",
"vpn_url": "https://prodvpn-eu-2.netatmo.net/restricted/255.255.255.255/a1f239843993d2a1k2201d336687d9a9/MTU5MDAxOTIwMDrR56gz25_ZK5vE3RLBH6s633rXxw,,",   

4- Identify/extract the from (here: a1f239843993d2a1k2201d336687d9a9)

@Novanic, according to you, is it possible to integrate this into Netatmo binding ? :-)

[Novanic]

@nokyyz Thank you for the information. :-) Technically it should be no problem and I would love to add that. But Netatmo hasn't documented these endpoints, so it seems to be inofficial and I don't know how Netatmo feels when we provide a public binding which accesses inofficial functions... When someone of Netatmo could confirm that we can use/call that, I would start to realize that.

[Novanic]

The first pull-request was now applied for OpenHAB 2.5.6. :-) Now the Presence camera is discovered (for example the image of the camera can get integrated with an image item).

Events: The events are still missing. I have now extended/updated the Netatmo API within the netatmo-swagger-decl and cbornet applied the pull-request. The next step is to update the netatmo-swagger-api project. I'm able to build and test successfully with it (local). But I don't know how we can build a new release for the artefact com.github.cbornet.netatmo-swagger-api:netatmo-java-retrofit. @cbornet should I create a pull-request and you release it? Which version number will get it (1.1.4 or 1.2.0)? Thank you in advance and best regards

[cbornet]

I don't really update or publish artifacts for netatmo-swagger-api. I think OpenHAB usually builds its own jar from the spec. @clinique , I think you did it initially ?

[clinique]

I don't really update or publish artifacts for netatmo-swagger-api. I think OpenHAB usually builds its own jar from the spec. @clinique , I think you did it initially ?

Yes, IIRW using netatmo-swagger-api, I built locally the target jar and uploaded it to the openhab lib folder.

[Novanic]

I don't really update or publish artifacts for netatmo-swagger-api. I think OpenHAB usually builds its own jar from the spec. @clinique , I think you did it initially ?

Yes, IIRW using netatmo-swagger-api, I built locally the target jar and uploaded it to the openhab lib folder.

Ok, thank you. That sounds simple, but which lib folder do you mean? I don't see a "lib" folder within the OpenHAB GitHub projects.

[clinique]

You're right, now the lib folder has been suppressed, it's a dependency in the pom.xml :

    <dependency>
      <groupId>com.github.cbornet.netatmo-swagger-api</groupId>
      <artifactId>netatmo-java-retrofit</artifactId>
      <version>1.1.3</version>
      <scope>compile</scope>
    </dependency>

[Novanic]

You're right, now the lib folder has been suppressed, it's a dependency in the pom.xml :

    <dependency>
      <groupId>com.github.cbornet.netatmo-swagger-api</groupId>
      <artifactId>netatmo-java-retrofit</artifactId>
      <version>1.1.3</version>
      <scope>compile</scope>
    </dependency>

Yes, I saw that but it seams to be difficult to get a new version of it uploaded to that repository (and I don't know which repository it is and who could do the release, it is not maven central).

Currently I try to integrate it within the maven build of the binding. So the maven build would download the swagger file (or uses a persisted copy of it) and generates the code with the swagger-codegen-maven-plugin. The advantage would be that it could get updated and built/"released" with just executing a maven phase and without involving 3 projects and various people. Is that a valid alternative solution?

For the events I would realize an event channel like "welcomeHomeEvent". But additionally to it I would realize 3 channels for boolean/switch items: humanDetectedChannel, animalDetectedChannel and vehicleDetectedChannel. That makes it easy to realize simple rules which are triggered by an item state change and to display the information within HabPanel for example. Would that be useful?

Thank you in advance and best regards

[clinique]

Yes, I think this is a very valid alternative solution, having these three projects has always been a pain preventing me to update more frequently this binding. Happy that you jumped in !

[Novanic]

I have created a new pull-request which ...

• uses the updated Netatmo API (to support the sub-event structure which is used by the Presence events)
• delivers all information of the Presence events to the home thing (the message and snapshot is now available/set)
• adds new channels to the home thing to report if a human, animal or vehicle was detected
• simplifies the process of updating the Netatmo API (the retrofit code generation is now embedded within the Maven build)

The webhook is still untested and therefore still not yet activated for the Presence camera. I will check and support that later.

And it seems that I have a problem with updating my fork/branch (there are too many changes and required reviewers within the pull-request, I will try to solve it within the next days and create a new pull-request).

[Novanic]

The pull-request is now available for review, see https://github.com/openhab/openhab-addons/pull/7807

[Novanic]

No, unfortunately the official Netatmo API doesn't seem to provide a solution to switch the floodlight. :-( We could try to ask Netatmo to extend the API.

There is a solution to stop/resume camera monitoring, to set the floodlight mode and to set the floodlight intensity !

To stop camera monitoring, run the Http request:
http://<netatmo presence camera ip address>/<vpn hash code>/command/changestatus?status=off

To set the floodlight mode in "auto" mode, run the Http request:
http://<netatmo presence camera ip address>/<vpn hash code>/command/floodlight_set_config?config={%22mode%22:%20%22auto%22}

To set the floodlight intensity to 50%, run the Http request:
http://<netatmo presence camera ip address>/<vpn hash code>/command/floodlight_set_config?config={%22intensity%22:%20%2250%22}

To get the <_vpn hash code_>:
1- Enter this URL into your web browser: https://dev.netatmo.com/apidocumentation/security#gethomedata
2- Click "Try it out" button
3- In Server Response, search the "vpn_url" field of your NOC camera
E.g:

...
"cameras": [
{
"id": "xx:xx:xx:xx:xx:xx",
"type": "NOC",
"status": "on",
"vpn_url": "https://prodvpn-eu-2.netatmo.net/restricted/255.255.255.255/a1f239843993d2a1k2201d336687d9a9/MTU5MDAxOTIwMDrR56gz25_ZK5vE3RLBH6s633rXxw,,",   

4- Identify/extract the from (here: a1f239843993d2a1k2201d336687d9a9)

@Novanic, according to you, is it possible to integrate this into Netatmo binding ? :-)

Hi, good news, I got the floodlight running. ;-) I have created a feature request for it, see #7912 . There is also the information where to find the current state if you would like to test it.

Do you or someone other know if the start / stop video recording feature is also available for the Welcome camera? Is it available within the App for Welcome cameras? Or is it Presence exclusive?

[lolodomo]

In the netatmo app, with the Welcome Camera, you can start and stop the live video. Is it the answer to your question ?

[Novanic]

In the netatmo app, with the Welcome Camera, you can start and stop the live video. Is it the answer to your question ?

I see an option called "Überwachung" (german) at the Presence outdoor camera settings, see screenshot. That should be called "Monitoring", "Observation" or "Surveillance" in english. Is that the same what you see at the Welcome camera.

[lolodomo]

Ah ok, yes of course, there is this option called "surveillance" in French to enable or disable the surveillance.
If the binding could control this option, that would be just wonderful. IMHO, this is the most important control I need in openHAB with the camera. I would like to enable the camera when leaving the house and stop it when I return, automatically by openHAB.

[Novanic]

Ah ok, yes of course, there is this option called "surveillance" in French to enable or disable the surveillance.
If the binding could control this option, that would be just wonderful. IMHO, this is the most important control I need in openHAB with the camera. I would like to enable the camera when leaving the house and stop it when I return, automatically by openHAB.

Ah, ok. Then it should get realized for all/both camera models. Thank you. :-) The floodlight support #7912 is a good preparation for it, I will try to realize it after that.

[Novanic]

Ah ok, yes of course, there is this option called "surveillance" in French to enable or disable the surveillance.
If the binding could control this option, that would be just wonderful. IMHO, this is the most important control I need in openHAB with the camera. I would like to enable the camera when leaving the house and stop it when I return, automatically by openHAB.

I have now opened issue #7938 for that.