Deconz-rest-plugin: description.xml

[posixx]

Hello,

I build V2.04.60 from dev and build rest api from latest source today, my description.xml contains 127.0.0.1 as ipaddress instead of the raspbee ip:

<URLBase>http://127.0.0.1:80/</URLBase>

Is this related to this new build or do i have another problem?

Regards,

Frans

[ebaauw]

Same here, with published binary for v2.04.60.

Worse, the UPnP anouncements also contain 127.0.0.1 and the gateway doesn't seem to respond to UPnP search requests. I'm away from home behind a double NAT so I cannot check the appspot. config.ipaddress in the REST API is also 127.0.0.1.

[ebaauw]

@posixx what address range are you using for your network? 10.x.x.x?

[posixx]

@ebaauw: yes 10.10.11.x

[ebaauw]

For some reason unknown to me, the REST API rejects this address range, https://github.com/dresden-elektronik/deconz-rest-plugin/blob/9e24dfb2441c8d94b516810967715acd7b79e7c2/rest_configuration.cpp#L316. @manup, what's your take on this? I don't think this code has changed recently so it's not really related to v2.04.60.

I changed the local address range from 10.0.0.x to 192.168.x.x and now I see the correct local address.

Still I see no response from the gateway to UPnP search requests, this is probably an unrelated problem, requiring further investigation.

[manup]

Yes there were changes to UPNP and IP address filtering for specific private IP ranges, not long ago. I'll review the IP code and check why your subnets are not included later today.

The UPNP code was changed due problems with Amazon Echo discovery which is working now again.
UPNP filtering needs to pass the following in order to receive a M-SEARCH response:

        if (!line.startsWith(QLatin1String("ST:")))
        {
            continue;
        }

        if (line.contains(QLatin1String("ssdp:all")) ||
            line.contains(QLatin1String("device:basic")) ||
            line.contains(QLatin1String("unpn:rootdevice")))
        {
            searchTarget = line;
            break;
        }

Can you please provide you UPNP M-SEARCH request payload?

[ebaauw]

M-SEARCH * HTTP/1.1
HOST: 239.255.255.250:1900
MAN: "ssdp:discover"
MX: 5
ST: upnp:rootdevice

There's a typo for the rootdevice: unpn instead of upnp.

[ebaauw]

Maybe related: does deCONZ open 0.0.0.0:1900 in shared mode? I get an EADDRINUSE error when running my upnp logger on the Pi running deCONZ.

[manup]

Ah I see, guess that's the problem. Fixed here: https://github.com/dresden-elektronik/deconz-rest-plugin/commit/918542a31e2da82155ed4d537b115372b438589c

Maybe related: does deCONZ open 0.0.0.0:1900 in shared mode? I get an EADDRINUSE error when running my upnp logger on the Pi running deCONZ.

Shared mode should be ok since otherwise only one application on the RPi could act as UPNP server.

[ebaauw]

Shared mode should be ok since otherwise only one application on the RPi could act as UPNP server.

That's what I figured from the code:
https://github.com/dresden-elektronik/deconz-rest-plugin/blob/master/upnp.cpp#L28.

Still, when I stop deCONZ, my logger works, and when deCONZ is running, it receives the EADDRINUSE error. When starting deCONZ while my logger is running, I see: QNativeSocketEngine::joinMulticastGroup() was not called in QAbstractSocket::BoundState. I can run multiple instances of my logger without any problem.

[posixx]

upnp response works again, only the ipaddress still 127.0.0.1...

[posixx]

@manup just to be sure it's not subnet related: my subnet = 10.10.10.0/23 so 512 addresses

[manup]

Fix will be online shortly, figuring out right now :)

[manup]

Commit is online: https://github.com/dresden-elektronik/deconz-rest-plugin/commit/5f928d4e2f4df1d8626fcb5548aea09e0340d40c this should correctly detect private network IPs.

[ebaauw]

I still don't understand the code. What are you trying to do - only allow RFC 1918 addresses?

[manup]

It's a filter for internal addresses (LAN accessible), some machines have public addresses which should not be included. Sometimes Qt lists some special addresses which aren't useful either.

[posixx]

@ebaauw: can you confirm upnp is working? Mine is not found yet.. Could be my switching hardware though. IP is OK in description,xml

[posixx]

i can confirm UPNP is working and also the ipaddress is OK. Thanks manup!!

[ebaauw]

Yes, @posixx, with v2.04.61 I get a response to the search as well as the regular alive announcements.

@manup, could you please have a look at the shared mode?

[manup]

@manup, could you please have a look at the shared mode?

Yes it's on the list, from the error message it looks like the joinMulticastGroup()call is missing.

[manup]

There's a new commit to address the shared address mode problem: https://github.com/dresden-elektronik/deconz-rest-plugin/commit/046379898c50a375c5b89ab94c44b99fa7df5a1e

The Qt documentation is a bit unclear on this matter.

[ebaauw]

There's a new commit to address the shared address mode problem: 0463798

After compiling and installing this commit:

• When starting my logger first, and then starting deCONZ, I no longer see the joinMulticastGroup message. However, deCONZ doesn't respond to M-SEARCH requests and doesn't send any alive announcements.
• When starting deCONZ first, it makes an alive announcement and responds to M-SEARCH requests. However, when starting my logger afterwards, it still reports EADDRINUSE on 0.0.0.0:1900.

Reverting to v2.04.61:

• When starting my logger first, and then starting deCONZ, I see the message:
  QNativeSocketEngine::joinMulticastGroup() was not called in QAbstractSocket::BoundState. deCONZ seems to respond to a M-SEARCH request with an alive announcement instead of a response. It also issues periodic announcements.
• When starting deCONZ first, it makes alive announcements and responds to M-SEARCH. However, when starting my logger afterwards, it reports EADDRINUSE on 0.0.0.0:1900.

[manup]

Hmm pretty strange. Which logger are you using?
I'd like to recreate the case here, it might be needed to set some low level flags to the socket, in Qt there are no more further options available (might be an issue in Qt, albeit that's unlikely).

[ebaauw]

Just a small javascript programme I wrote and run on node v6.11.1.

upnp.js.zip

[manup]

On Arch Linux with Qt 5.9 and Node 8.2.1 UPNP runs without problems regardless which one is started first.

On Raspbian Jessi with Node 0.10.29 the upnp.js doesn't start and throws following error:

    dgram.js:87
        throw new Error('Bad socket type specified. Valid types are: udp4, udp6');
                 ^
    Error: Bad socket type specified. Valid types are: udp4, udp6
        at newHandle (dgram.js:87:9)

    at new Socket (dgram.js:112:16)
    at Object.exports.createSocket (dgram.js:129:10)
    at new Search (/home/pi/upnp.js:41:19)
    at Object.<anonymous> (/home/pi/upnp.js:79:14)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)

Which node version to you use?

[ebaauw]

node v6.11.2 - I just got an update. I installed it following the instructions from https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions

curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
sudo apt-get install -y nodejs

[ebaauw]

I see the same behaviour with deCONZ v2.04.62 on Ubuntu and nodejs v6.11.2 and v8.2.1 as on the Raspberry Pi. Not sure about the Qt version, I just did sudo apt-get install qt5-default libqt5sql5 libqt5websockets5-dev libqt5serialport5-dev to install.

[manup]

The Raspbian Jessi Qt version is 5.3.2, which is quite old. Anyway it has to work with that version, I'll add some low level checks to see if the socket is configured properly.

[ebaauw]

I wiped and re-installed my test raspberry, starting by downloading the latest NOOBS distribution. It now runs Raspbian stretch instead of jessie. uname -a yields:

Linux pi 4.9.41-v7+ #1023 SMP Tue Aug 8 16:00:15 BST 2017 armv7l GNU/Linux

deCONZ v2.04.78 seems happy enough, but I haven't paired any devices yet. The GUI looks a lot better, due to the new fonts and more pronounced UI elements. I will try and upgrade my production Raspberry next.

deCONZ still reports Qt 5.3.2, but I no longer get the EADDRINUSE error from my upnp.js logger when deCONZ is running. Also, when starting the logger first, deCONZ responds to M-SEARCH requests. Not sure if you changed anything, @manup, or if this is due to the new Raspbian version and/or new Qt run-time libraries.

Incidentally, it looks like there might be a newer Qt version for the Raspberry. sqlitebrowser now reports Version 3.9.1, SQLite Version 3.15.2 and Qt Version 5.7.1.

[manup]

Yes I think this is due the new Qt version in stretch, there were no other changes on the topic.
Be aware that GCFFlasher doesn't work on stretch yet, so there are firmware update issues, which I hope to fix during the next days.

[manup]

Be aware that GCFFlasher doesn't work on stretch yet, so there are firmware update issues, which I hope to fix during the next days.

Need to correct myself, no problems with GCFFlasher_internal and recent beta versions in Stretch.
The update loops in stretch are related to standalone GCFFlasher and stable deCONZ version 2.04.35.

[ebaauw]

I think we can close this issue.

[davidgraeff]

This is the response I get within my local network with the PI being on address 192.168.1.8:

HTTP/1.1 200 OK
CACHE-CONTROL: max-age=100
EXT:
LOCATION: http://127.0.0.1:80/description.xml
SERVER: FreeRTOS/7.4.2, UPnP/1.0, IpBridge/1.8.0
ST: ssdp:all
USN: uuid:d14660d5-d84b-4a91-8f8e-56cb4a3xxxxx::upnp:rootdevice
GWID.phoscon.de: 00212EFFxxxxxx51
hue-bridgeid: 00212EFFxxxxxx51

ip address returns:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether b8:27:eb:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.8/24 brd 192.168.1.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::9d09:f770:69f2:e2e4/64 scope link 
       valid_lft forever preferred_lft forever
3: wlan0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether b8:27:eb:xx:xx:xx brd ff:ff:ff:ff:ff:ff

[manup]

Interesting does the REST-API /config also return the 127.0.0.1 ip address? I thought there was a filter to prefer the non local ip address.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.