Deconz-rest-plugin: Announcement: ZSHARK - Wireshark Sniffer for ConBee (Beta)

[ebaauw]

macOS? Please?

[snozzlebert]

I think @manup means that you can use Wireshark on macOS while the ConBee with ZShark is running on another machine with Raspbian/Ubuntu/Windows.

[manup]

I think @manup means that you can use Wireshark on macOS while the ConBee with ZShark is running on another machine with Raspbian/Ubuntu/Windows.

Also possible ZShark is running in a Ubuntu VM on a Mac and forwards data to Wireshark which runs natively on the same Mac.

Not the perfect solution, a native macOS version is challenging due the firmware flashing part which needs super user rights. I can't provide a ETA but we may provide a ZShark version for macOS there firmware must be installed separately via GCFFlasher in a terminal (same goes for deCONZ).

[ebaauw]

I want to ditch my Ubuntu VM, if possible. I only use it for BitCatcher. Happy to flash the ConBee manually, if that what it takes.

I don't suppose I can use a single Raspberry with a RaspBee _and_ a ConBee installed, running deCONZ on the RaspBee and ZShark on the ConBee in parallel.

[manup]

I don't suppose I can use a single Raspberry with a RaspBee and a ConBee installed, running deCONZ on the RaspBee and ZShark on the ConBee in parallel.

Just tried that, yes works too :)

[ebaauw]

Indeed, that works!

I get more junk (ICMP messages and ACKs) then ZigBee messages when filtering on port 17754. If I apply a display filter for zbee_nwk or zbee_aps, WireShark only shows the (ethernet frames with encapsulated) ZigBee frames.

I do seem to miss quite a few packages when sniffing the local deCONZ network.

The Network Settings dialog in the deCONZ GUI shows zeroes for all fields. Not sure if this is related to ZShark, or RaspBee firmware 0x261D0500 (I use a self compiled version of the REST API plugin, so it updated to this version). deCONZ seems to work fine nevertheless, even restarting does work. ZShark was happy to update the ConBee firmware (this is different from the BitCatcher firmware?) while deCONZ was running.
EDIT manually updated the RaspBee to 0x261E0500 and the network settings are shown again.

[ebaauw]

I do seem to miss quite a few packages when sniffing the local deCONZ network.

I suspect it's due to radio interference between the RaspBee and the ConBee (both connected to the same Raspberry). I dug up an old USB Female-A to A 5m extension cable (from a previous life), and connected the ConBee through that. It works (even through 5m is pushing from a USB standard perspective). I can now place the ConBee halfway in between the RaspBerry and the device being sniffed, and it looks like I now capture the full traffic.

Can I only enter an IP address in the Remote Capture field? I tried a hostname instead, but that doesn't seem to work?

[manup]

It works (even through 5m is pushing from a USB standard perspective). I can now place the ConBee halfway in between the RaspBerry and the device being sniffed, and it looks like I now capture the full traffic.

Good catch I'll try this too, radio interference can be a beast :)

We also are investigating in the missed packages in some scenarios like OTA traffic, for one the RX circular buffer was quite small (8) we raised that to 32. Also sniffer is running at 38400 baud which we can raise since ConBee has a FTDI in a future firmware.

Meanwhile here is the version with larger circular buffer, you may give it a try, it might help but this isn't verified yet:

sniffer-1.00-32packets.zip

Flashed as usual:

$ sudo GCFFlasher_internal -d 0 -f sniffer-1.00-32packets.bin

Can I only enter an IP address in the Remote Capture field? I tried a hostname instead, but that doesn't seem to work?

I'll forward this, the sniffer is developed by a colleague, should be easy to fix.

[Framspott]

Hi,

ist there a possibility to sniff the communication between a Zigbee device and a Raspbee?
I want to know the communication over the serial port /dev/ttyS0 between my Hue bulb and my Raspberry.

All the best

Framspott

[manup]

To see whole ZigBee packets you need a ConBee which acts as sniffer device to monitor the ZigBee traffic. The RaspBee can't act as sniffer and coordinator at the same time.

[manup]

ZSHARK is now officially released.

If you are running an older ZSHARK firmware it's strongly advised to update the firmware contained in the package, we fixed some nasty bugs which should improve sniffer sensitivity a lot.

https://www.dresden-elektronik.de/funktechnik/products/software/zshark/?L=1

[ebaauw]

Dare I ask about macOS support?

[manup]

We currently experimenting with the Qt Installer Framework (ZSHARK Windows) and automated builds (deCONZ Raspbian/Ubuntu). As soon as it's stable I hope to also derive automated macOS builds for ZSHARK and deCONZ.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ebaauw]

We might want to pin this topic.

[manup]

Yes maybe also add a reference in the Wiki, otherwise the Website always has the latest release.

https://www.dresden-elektronik.de/funktechnik/products/software/zshark/?L=1

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ebaauw]

Pin

[manup]

Cool didn't know there is a pin option :)

[timbru31]

I assume the CC2531 USB Stick is not supported? Have a spare one lying around. (so far I've used https://github.com/andrewdodd/ccsniffpiper)

---

A quick question, too: @manup - It seems that a NWK key with 32bit MIC (Message Integrity Code) is used (security level 5) - why no 128bit key for the MIC (security level 7)? Any reason?

[lynaghk]

Does ZShark support the ConBee II?

On Windows 10 deCONZ lists the ConBee II as a device option, but ZShark's device list is empty (zshark version 1.02):

I've also tried Ubuntu 18.04 LTS in a VirtualBox VM, but ZShark doesn't pick up the device there either (though lsusb shows that it's visible within the VM: Bus 002 Device 002: ID 1cf1:0030 Dresden Elektronik)

I was under the impression that it should be compatible, based on this PDF:

[manup]

Sorry this is an older PDF, ZSHARK will support ConBee II but it isn't ported yet. There is no ETA, but I hope it will be done within the next two months.

[lynaghk]

Alright, I'll find another solution. Thanks for the quick reply!

[MarcoBodega]

Sorry this is an older PDF, ZSHARK will support ConBee II but it isn't ported yet. There is no ETA, but I hope it will be done within the next two months.

are there any advancement on ZSHARK for ConBee II ?

[mascygp]

Sorry this is an older PDF, ZSHARK will support ConBee II but it isn't ported yet. There is no ETA, but I hope it will be done within the next two months.

there are some news about ZSHARK for ConBee II ?

[Otnow]

Sorry this is an older PDF, ZSHARK will support ConBee II but it isn't ported yet. There is no ETA, but I hope it will be done within the next two months.

there are some news about ZSHARK for ConBee II ?

+1

[manup]

Sorry no news yet. The work on it will continue as soon as the new bootloader and deCONZ firmware is finished.

[mfilippo]

Hi,
where can we get ZSHARK?

[tannyl]

The link to the Zshark software (https://www.dresden-elektronik.de/funktechnik/products/software/zshark/?L=1) does not work anymore, nor can I find it on the phoscon.de as otherwise state on dresden-elektronik.de? (there seems to be a lot of 404 going on your site?)

[FlyingPersian]

Am I right to assume that Zshark cannot be installed newly atm cuz there isn'r any download available?

[manup]

Sorry for the delay, turns out the relaunch of dresden-elektronik.de broke many links. We are working to fix these asap (auto redirect to new location).

For ZSHARK downloads please refer to:

https://phoscon.de/downloads/zshark/

[FlyingPersian]

Can I use this tool with a CC2531 USB Dongle on Ubuntu or a RPI? I don't have a CC Debugger, but the tutorials I find describe using a CC Debugger. Is it needed though?

Edit: Following this tutorial, do I need the CC2531 Downloader Cable? I connected the RPI GPIOs to the GPIOs of the CC2531 with dumont cables and crocodile clips, but when I run ./cc_chipid it returns 0000 or ffff. The cabling is as described on the site: https://www.zigbee2mqtt.io/information/alternative_flashing_methods.html

[kentwinther]

I just got my ConBee II, and that's great, but I am not able to select it in the Zshark windows app?
Is ConBee II still not supported by Zshark?

[kentwinther]

Hi again...
Is ConBee II supported by Zshark or not?
I bought the ConBee II in order to do ZigBig sniffing!

[FlyingPersian]

According to here yes: https://www.dresden-elektronik.de/funk/software/zshark.html

[kentwinther]

@FlyingPersian Yes, that's right. ConBee II should be supported by Zshark according to the website. But I´m not able to select my ConBee II in the Zshark app. It does not showup in the dropdown list...
Do you have a ConBee II and does it work in Zshark?

[manup]

Unfortunately the ConBee II is not yet supported in ZSHARK, the support for it is scheduled for 2020.

[kentwinther]

@manup That's too bad, as I bought this for sniffing...
Do you know when this will be supported in 2020 (month)?

[dandrzejewski]

After installing this firmware and playing with it, then going back to the normal firmware, none of my zigbee devices can connect and are not responsive. They're all EcoSmart bulbs from Home Depot.

[dandrzejewski]

After installing this firmware and playing with it, then going back to the normal firmware, none of my zigbee devices can connect and are not responsive. They're all EcoSmart bulbs from Home Depot.

Interestingly, I power-cycled ONE of the bulbs and the entire mesh came back online.

I'm not 100% sure of this, but it looks like the _direct_ links between the raspbee and the other devices have not come back, but it all seems to be functioning using the bulb I power-cycled as a repeater.

Edit: And after a reboot, it's all disconnected again.

[ebaauw]

After installing this firmware and playing with it, then going back to the normal firmware, none of my zigbee devices can connect and are not responsive.

Reflashing the RaspBee or ConBee with the ZShark firmware and then back with the deCONZ firmware will likely lose the data in the device’s non-volatile memory. You might want to double-check that the network parameters (PAN ID, network key, channel) are still valid.

Interestingly, I power-cycled ONE of the bulbs and the entire mesh came back online.

That’s not how ZigBee works. Your bulbs have been “online” all the time; they don’t need the coordinator to communicate with each other. It’s just the RaspBee or ConBee not having discovered the network.

The graph in the GUI is just a graphical representation of the info deCONZ has received from your devices. It does _not_ show any “active” connections, just the info from the routing tables from the time the device was last queried. When you power-cycle a device, it sends a _Device Announcement_ message, which deCONZ picks up to query the device with prejudice.

[manup]

Reflashing the RaspBee or ConBee with the ZShark firmware and then back with the deCONZ firmware will likely lose the data in the device’s non-volatile memory. You might want to double-check that the network parameters (PAN ID, network key, channel) are still valid.

The values should be stable and preserved since ZSHARK firmware doesn't touch the NVRAM of ConBee I and RaspBee I, which is stored in the EEPROM.

[dandrzejewski]

After installing this firmware and playing with it, then going back to the normal firmware, none of my zigbee devices can connect and are not responsive.

Reflashing the RaspBee or ConBee with the ZShark firmware and then back with the deCONZ firmware will likely lose the data in the device’s non-volatile memory. You might want to double-check that the network parameters (PAN ID, network key, channel) are still valid.

The OP in this issue states that this should not happen. But it did end up changing the PAN ID, which I changed back.

Interestingly, I power-cycled ONE of the bulbs and the entire mesh came back online.

That’s not how ZigBee works. Your bulbs have been “online” all the time; they don’t need the coordinator to communicate with each other. It’s just the RaspBee or ConBee not having discovered the network.

Got it. The UI doesn't make that clear (to me, at least).

The graph in the GUI is just a graphical representation of the info deCONZ has received from your devices. It does _not_ show any “active” connections, just the info from the routing tables from the time the device was last queried. When you power-cycle a device, it sends a _Device Announcement_ message, which deCONZ picks up to query the device with prejudice.

Right, well my point is that I want to rely on this device to be an interface between HomeSeer and ZigBee. After going through this, the fact that it lost track of all the devices when the original post said it should not is concerning. And the fact that things didn't automatically come back up is, too.

This same issue even happens if I simply power cycle the pi (shutting it down cleanly of course) and then bring it back up. Even after several hours the device is unable to talk to any other devices. I would expect things should be able to heal within minutes at worst.

Nevertheless, power cycling one bulb seems to get it "un-stuck," probably for the reason you stated - it sends a device announcement.

On closer inspection, it appears that the raspbee is only able to contact the other devices "through" the bulb I power cycled (that bulb is a repeater). That's according to the diagram in deCONZ, and evidenced by the fact that everything else stops working again if I power off that bulb.

Is there some parameter I'm missing, or is this a bug, or is there something I just don't understand about how ZigBee is supposed to work?

[ebaauw]

On closer inspection, it appears that the raspbee is only able to contact the other devices "through" the bulb I power cycled (that bulb is a repeater). That's according to the diagram in deCONZ, and evidenced by the fact that everything else stops working again if I power off that bulb.

In theory, that could be valid, if this one bulb is the only device in direct range of the RaspBee or ConBee. What happens after you power-cycle another bulb?

[Cyrille63]

Hi
ConBee II is not supported by Zshark, have you got a support date?
Thk

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tkl-git]

Unfortunately the ConBee II is not yet supported in ZSHARK, the support for it is scheduled for 2020.

Hi, do you have updated information? I think Zshark support for ConBee 2 would be a very useful feature and would really help people to debug Zigbee network problems.

[abmantis]

Any way to run zshark on a headless machine (without VNC nor X-Forwarding)?