Pwnagotchi: nexmon blindness bug (brcmf_cfg80211_nexmon_set_channel)

Created on 12 Oct 2019  路  85Comments  路  Source: evilsocket/pwnagotchi

every once in a while, nexmon dies with:

[ 4341.527847] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110
[ 4344.327806] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[ 4347.127853] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[ 4349.927917] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 4352.728074] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[ 4355.527970] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110
[ 4358.328022] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 4361.208095] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 4364.008157] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[ 4366.808218] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 4369.608431] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[ 4372.408345] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[ 4375.288408] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 4378.088474] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[ 4380.891399] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110

And only a reboot can fix the wifi, this is why the mon_max_blind_epochs parameter exists, to reboot the board when this happens.

Ideally we should document this known issue, the configuration and some day maybe fix it.

bug hacktoberfest hardware help wanted
Source
picture evilsocket

Most helpful comment

here's my nexmon blindness error watchdog script I whipped together that seems to work pretty well until the bug is fixed. needs to be run as root. seems to do its job from the testing I've done:

#!/usr/bin/env bash
set -euo pipefail
red=$(tput setaf 1)
green=$(tput setaf 2)
yellow=$(tput setaf 3)
reset=$(tput sgr0)

if ! [ $(id -u) = 0 ]; then
   echo ${red}"need to run this as root!${reset}"
   exit 1
fi

echo "${yellow}patiently waiting for nexmon blindness error (https://github.com/evilsocket/pwnagotchi/issues/267)..${reset}"
tail -n 0 -f /var/log/kern.log | while read -r kernel_log_line; do
    if [[ $kernel_log_line == *"brcmf_cfg80211_nexmon_set_channel: Set Channel failed"* || $kernel_log_line == *"brcmf_cfg80211_get_tx_power: error"* || $kernel_log_line == *"brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed"* ]]; then
        echo "${red}caught blindness error, reloading nexmon driver..${reset}"
        set -x
        echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/unbind
        sleep 1
        echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/unbind
        sleep 3
        echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/bind
        sleep 3
        echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/bind
        sleep 3
        modprobe brcmfmac #this modprobe may not be needed, but it seems to help sometimes.
        sleep 3
        #running a couple extra binds here in case the modprobe made a difference. it can't hurt!
        echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/bind || true
        sleep 3
        echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/bind || true
        sleep 3
        iw dev wlan0 interface add mon0 type monitor
        sleep 3
        ifconfig mon0 up
        set +x
        date
        echo "${green}nexmon driver reloaded and mon0 brought back up! looping until next error..${reset}"
    fi
done
picture soaringswine on 6 Jan 2020
👍3

All 85 comments

cc @hexwaxwing

evilsocket picture evilsocket on 12 Oct 2019

docs added https://pwnagotchi.ai/usage/#known-issues

evilsocket picture evilsocket on 12 Oct 2019

You could try reloading the driver instead of rebooting the pi, using :

modprobe -r brcmfmac
modprobe brcmfmac

If this still fails, then a reboot will fix this.
There is a great discussion about that problem here : https://www.bountysource.com/issues/56252669-wlan-freezes-in-raspberry-pi-3b

Leajian picture Leajian on 13 Oct 2019
😄1

tried that way, it doesn't always work, the only reliable way is rebooting

evilsocket picture evilsocket on 13 Oct 2019

So, The Nexmon firmware is a little picky on how its interfaced, How are your bringing up the mon interface in linux before bettercap gets to it?

JRWR picture JRWR on 14 Oct 2019

iw phy phy0 interface add mon0 type monitor && ifconfig mon0 up

evilsocket picture evilsocket on 14 Oct 2019

from https://github.com/evilsocket/pwnagotchi/blob/master/builder/pwnagotchi.yml#L333

evilsocket picture evilsocket on 14 Oct 2019

(from re4son monstart script)

evilsocket picture evilsocket on 14 Oct 2019

I just noticed on the Nexmon Repo that the bcm43455c0 does not support wifi frame injection. That might be the reason for the drivers crashing as they are not handling the requests to do frame injection correctly from bettercap.

JRWR picture JRWR on 15 Oct 2019

Ok, So I looked, We are using the older version of the firmware provided by Nexmon, I think Re4son kernel does it as its the default for Nexmon to use that. Patches and Commits from the Nexmon Project show 7.45.189 as the latest version you can use (The base firmware comes from the OEM, not the RPI foundation) as of RC4, we are using version 7.45.154 of the bcm43455c0 firmware This issue should only affect 3B+ and 4s, the 3B and the 0W use the same wifi chip and are listed as supporting injection.

JRWR picture JRWR on 15 Oct 2019

More Details, I've looked into how the Re4son kernel builder pulls down its firmware, its even /worse/ Looks like it pulls from https://github.com/Re4son/re4son-nexmon as its source of nexmon patches... its years out of date. I'm doing some prototyping to update this now.

JRWR picture JRWR on 15 Oct 2019

Good News, The Nexmon Patches with the stock kernel works well, Here is my DMesg output after running for 10 minutes, I will be running it for the next 24 hours to see if its stable, 

[    5.128852] brcmfmac: loading out-of-tree module taints kernel.
[    5.128864] brcmfmac: loading out-of-tree module taints kernel.
[    5.181306] brcmfmac: F1 signature read @0x18000000=0x15264345
[    5.190412] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    5.190907] usbcore: registered new interface driver brcmfmac
[    5.566003] brcmfmac: brcmf_sdio_bus_preinit: before brcmf_sdio_debugfs_create
[    5.569776] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    5.597271] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Oct 15 2019 20:30:25 version 7.45.189 (nexmon.org: -4) FWID 01-e1db26e2
[    5.675296] brcmfmac: brcmf_bus_started: before brcmf_debugfs_add_entry
[    8.495009] brcmfmac: brcmf_vif_add_validate: Attempt to add a MONITOR interface...
[    8.495024] brcmfmac: brcmf_mon_add_vif: brcmf_mon_add_vif called
[    8.495028] brcmfmac: brcmf_mon_add_vif: Adding vif "mon0"

Here is my uname

Linux pwnagotchi 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux

JRWR picture JRWR on 15 Oct 2019

And I failed, You get a little more debug output this time overall..

[ 5387.511962] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[ 5390.551934] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[ 5393.601927] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5396.631949] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110
[ 5399.671984] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 5402.711975] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5405.752020] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[ 5408.791991] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5411.831985] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4106, -110
[ 5414.872013] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53284, -110
[ 5417.911966] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53288, -110
[ 5420.951985] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53296, -110
[ 5423.511967] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5426.312006] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[ 5429.111938] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5431.911970] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5434.711974] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53308, -110
[ 5437.511963] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53348, -110
[ 5440.311927] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53356, -110
[ 5443.111922] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53360, -110
[ 5445.911981] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53368, -110
[ 5448.711973] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53372, -110
[ 5451.511925] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53380, -110
[ 5454.311928] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 5457.111929] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[ 5459.911983] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5462.711974] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5465.511924] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53308, -110
[ 5468.311926] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53348, -110
[ 5471.111925] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53356, -110
[ 5473.912031] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53360, -110
[ 5476.471932] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 5479.271932] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 5482.081987] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[ 5484.871994] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[ 5487.671927] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53292, -110
[ 5490.471930] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53300, -110
[ 5493.271977] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53304, -110
[ 5496.071923] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53348, -110
[ 5498.872004] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53364, -110
[ 5501.671976] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53372, -110
[ 5504.471928] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53376, -110
[ 5507.031989] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[ 5507.032002] brcmfmac: brcmf_cfg80211_get_channel: chanspec failed (-110)
[ 5509.591930] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[ 5509.591939] brcmfmac: brcmf_cfg80211_get_tx_power: error (-110)
[ 5512.151989] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 5514.952027] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110

Edit:
Added brcmfmac.debug=30 to /boot/cmdline.txt to try and get a better look at what the driver is doing, It enables full trace mode for the drivers, I have attacked a full debug log up to where it strops responding.
syslog.gz

JRWR picture JRWR on 16 Oct 2019

yep when that happens even trying to change channel manually doesn't work, i think it's the heat

evilsocket picture evilsocket on 16 Oct 2019

Based off this photo, There is no TIM under the wifi can at all... Since its used for shielding RF, this is of no surprise. Someone should get a IR Camera on the board and take a look.

image

Also You notice that White IC in the top left, Thats the same chip they used to show people the SuperMicro Implant... lulz

JRWR picture JRWR on 17 Oct 2019
🚀2

So, I'm looking at the datasheet, for the CYW43455 It states that the Max temp for operation is 120C while under normal loads on a 4 layer board Of course Max oper temp is +85C The thing only puts out 1.2W but I did notice that on the older RPI0W Chips it had a self limiter for overheat, This one is not stated in the data sheet as having one

JRWR picture JRWR on 17 Oct 2019

so you're saying that it's the chinese sabotaging our wifi pwning, right?

evilsocket picture evilsocket on 17 Oct 2019
😄2

Maybe.... Until I can get a proper Temp readout of the die itself while under the heavy load, We wont know.

JRWR picture JRWR on 17 Oct 2019

_suspence music_

evilsocket picture evilsocket on 17 Oct 2019

Hi,
is the same issue afflicting both 43455c0 and 43430a1?
Have you got a reliable way to reproduce it?

DrSchottky picture DrSchottky on 20 Oct 2019

@evilsocket
hey, I am investigating "brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4101, -110" aka Nexmon blindness bug issue and have noticed something that can potentially lead to resolution of the problem. My understanding was that patched Nexmon failed to set a channel, so instead of hopping on all channels I specified the channels manually to use for sending deauth frames in config.yml. In my case most AP are sitting on 1,4,6 and 11. Gotchi has been running for 15 minutes so far without any crashes. Can somebody confirm if it can be a valid solution ?

hackabean picture hackabean on 9 Nov 2019

@hackabean well not really, you just worked around the issue, but when that happens the whole wifi chip stops being very responsive

evilsocket picture evilsocket on 9 Nov 2019

if you specify the channels manually it will still hop ... i wonder if that's an issue related to hopping too quickly ...

evilsocket picture evilsocket on 9 Nov 2019

maybe increasing the wifi.hop.period

evilsocket picture evilsocket on 9 Nov 2019

Took the gotchi for a walk in the busy neighborhood, have set the channels beforehand and the issue still remains. Got excited prematurely, this does not work either, sorry :(

hackabean picture hackabean on 9 Nov 2019

@hackabean -110 means firmware hanged.
When working on nexmon I experienced blindness followed by cmds timeout when I was leaking memory in rx queue. Timeout/traps are fw bugs.

DrSchottky picture DrSchottky on 9 Nov 2019

@DrSchottky thanks for stepping in, seems like you know more about this issue. I just wished I could do something to make this less problematic so I was just poking around hoping to find a workaround. I am crossing my fingers that somebody will take a look at it.

hackabean picture hackabean on 9 Nov 2019

@hackabean to work on the problem I need to know as much as possible about the setup (board and kernel/driver/fw versions), the environment (does it crash under particular conditions? How often?) and I need to have a reliable way to reproduce the bug on my testbed.
AFAIK monitor mode should be stable (never used injection in deep so far)

DrSchottky picture DrSchottky on 10 Nov 2019
❤1

@DrSchottky I can reproduce it pretty reliably on my Pi4 setup. I may need a little help gathering the info you need, but I'm pretty confident it will go blind within an hour every time I power it up. What can I do to help?

andrewbeard picture andrewbeard on 11 Nov 2019

@andrewbeard I need to know:

  • Kernel: uname -a
  • bcm fw: dmesg | grep 7.45
DrSchottky picture DrSchottky on 11 Nov 2019
❤1
  • Kernel: uname -a
    Linux pwnpi4 4.19.66-Re4son-v7l+ #1 SMP Mon Aug 19 08:12:03 AEST 2019 armv7l GNU/Linux
  • bcm fw: dmesg | grep 7.45
    [ 2.749080] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Oct 8 2018 09:18:41 version 7.45.154 (nexmon.org: 2.2.2-195-g01e0-1) FWID 01-4fbe0b04
andrewbeard picture andrewbeard on 11 Nov 2019

System went blind within 3 minutes of powering up:

[ 2.739241] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[ 2.749080] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Oct 8 2018 09:18:41 version 7.45.154 (nexmon.org: 2.2.2-195-g01e0-1) FWID 01-4fbe0b04
[ 3.185453] bcmgenet: Skipping UMAC reset
[ 3.301924] bcmgenet fd580000.genet: configuring instance for external RGMII (no delay)
[ 3.302331] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 3.692992] uart-pl011 fe201000.serial: no DMA platform data
[ 3.826165] fuse init (API version 7.27)
[ 3.892931] brcmfmac: brcmf_vif_add_validate: Attempt to add a MONITOR interface...
[ 3.892940] brcmfmac: brcmf_mon_add_vif: brcmf_mon_add_vif called
[ 3.892944] brcmfmac: brcmf_mon_add_vif: Adding vif "mon0"
[ 4.222668] Adding 102396k swap on /var/swap. Priority:-2 extents:2 across:106496k SSFS
[ 4.311539] bcmgenet fd580000.genet eth0: Link is Down
[ 8.471570] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 8.471593] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 10.595212] Bluetooth: Core ver 2.22
[ 10.595265] NET: Registered protocol family 31
[ 10.595270] Bluetooth: HCI device and connection manager initialized
[ 10.596016] Bluetooth: HCI socket layer initialized
[ 10.596026] Bluetooth: L2CAP socket layer initialized
[ 10.596069] Bluetooth: SCO socket layer initialized
[ 10.604001] Bluetooth: HCI UART driver ver 2.3
[ 10.604008] Bluetooth: HCI UART protocol H4 registered
[ 10.604064] Bluetooth: HCI UART protocol Three-wire (H5) registered
[ 10.604164] Bluetooth: HCI UART protocol Broadcom registered
[ 10.763404] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[ 10.763411] Bluetooth: BNEP filters: protocol multicast
[ 10.763423] Bluetooth: BNEP socket layer initialized
[ 156.774707] brcmfmac: brcmf_sdio_hostmail: mailbox indicates firmware halted
[ 160.556985] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 163.116341] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4107, -110
[ 165.996384] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[ 168.796416] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[ 171.596449] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4109, -110
[ 174.396468] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53284, -110
[ 177.196499] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53300, -110
[ 179.996566] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53304, -110
[ 182.636544] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[ 209.916754] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53284, -110
[ 247.996974] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[ 255.597015] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4104, -110
[ 258.397045] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4107, -110
[ 261.197034] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4108, -110

... and that continues ad nauseum

andrewbeard picture andrewbeard on 11 Nov 2019

7.45.154 is an outdated base fw (and if it cames form re4son-nexmon it also has outdated patches. Avoid it)
Broadcom's built in monitor mode for bcm43455c0(RPi 3B+/4) is broken, but we fixed it (at least the bug we found) from nexmon in 7.45.189 patchset. You should get the headers for your kernel and build fw + driver with the latest patches from nexmon official repo.

I'm not telling you that it'll solve your problem, but before we go any further I need you to have the latest nexmon running on your system.

DrSchottky picture DrSchottky on 11 Nov 2019

@DrSchottky I am totally willing to do that, but I may need a bit of hand holding as I don't have a dev environment for this system yet. So far I've just been writing images to SD cards and changing config files. Any chance you (or anyone else) can point me at some info for building and replacing the kernel driver on a pi?

andrewbeard picture andrewbeard on 11 Nov 2019

As far as I read the released pwnagotchi img includes the header package, so ssh into your device and start following the README (but do not install the raspberrypi-kernel-headers package!)

DrSchottky picture DrSchottky on 11 Nov 2019

I'm trying to run through that now, but I've run into a couple snags. Most of them I'm getting through (aclocal version mismatches, etc) but my end result didn't go all that well:

COLLECTING STATISTICS read /usr/src/nexmon/STATISTICS.md for more information
COMPILING src/version.c => obj/version.o (details: log/compiler.log)
GENERATING LINKER FILE gen/nexmon.pre => gen/nexmon.ld
GENERATING LINKER FILE gen/nexmon.pre => gen/flashpatches.ld
LINKING OBJECTS => gen/patch.elf (details: log/linker.log, log/linker.err)
GENERATING MAKE FILE gen/nexmon.pre => gen/nexmon.mk
GENERATING MAKE FILE gen/nexmon.pre => gen/flashpatches.mk
APPLYING FLASHPATCHES gen/flashpatches.mk => brcmfmac43455-sdio.bin (details: log/flashpatches.log)
APPLYING PATCHES gen/nexmon.mk => brcmfmac43455-sdio.bin (details: log/patches.log)
BUILDING DRIVER for kernel 4.19 brcmfmac_4.19.y-nexmon/brcmfmac.ko (details: log/driver.log)
COPYING brcmfmac43455-sdio.bin => /lib/firmware/brcm/brcmfmac43455-sdio.bin
UNLOADING brcmfmac
RELOADING brcmfmac
insmod: ERROR: could not insert module brcmfmac_4.19.y-nexmon/brcmfmac.ko: Invalid module format
make: * [Makefile:192: install-firmware] Error 1

I _think_ this might be related to a mismatch in the kernel header versions, but I'm really not familiar with ARM ABI conventions.

root@pwnpi4:/usr/src# ls -al
total 16
drwxr-xr-x 4 root root 4096 Nov 11 15:04 .
drwxr-xr-x 10 root root 4096 Jul 9 20:03 ..
drwxr-xr-x 24 root root 4096 Nov 11 15:03 linux-headers-4.19.66-Re4son-v7+
drwxr-xr-x 10 root root 4096 Nov 11 15:32 nexmon
root@pwnpi4:/usr/src# uname -a
Linux pwnpi4 4.19.66-Re4son-v7l+ #1 SMP Mon Aug 19 08:12:03 AEST 2019 armv7l GNU/Linux

The first build complained I didn't have a symlink in /lib/modules/4.19.66-Re4son-v7l+/build so I added one to /usr/src/linux-headers-4.19.66-Re4son-v7+. I'm thinking that l is more important than I hoped, though...

andrewbeard picture andrewbeard on 11 Nov 2019

The module you're compiling is not targeting your running kernel.
Post the result of modinfo brcmfmac_4.19.y-nexmon/brcmfmac.ko

DrSchottky picture DrSchottky on 11 Nov 2019

root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# modinfo brcmfmac_4.19.y-nexmon/brcmfmac.ko
filename: /usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon/brcmfmac_4.19.y-nexmon/brcmfmac.ko
license: Dual BSD/GPL
description: Broadcom 802.11 wireless LAN fullmac driver.
author: Broadcom Corporation
firmware: brcm/brcmfmac4373-sdio.bin
firmware: brcm/brcmfmac4356-sdio.bin
firmware: brcm/brcmfmac4354-sdio.bin
firmware: brcm/brcmfmac43455-sdio.bin
firmware: brcm/brcmfmac43430-sdio.bin
firmware: brcm/brcmfmac43430a0-sdio.bin
firmware: brcm/brcmfmac4339-sdio.bin
firmware: brcm/brcmfmac43362-sdio.bin
firmware: brcm/brcmfmac4335-sdio.bin
firmware: brcm/brcmfmac43340-sdio.bin
firmware: brcm/brcmfmac4334-sdio.bin
firmware: brcm/brcmfmac4330-sdio.bin
firmware: brcm/brcmfmac4329-sdio.bin
firmware: brcm/brcmfmac43241b5-sdio.bin
firmware: brcm/brcmfmac43241b4-sdio.bin
firmware: brcm/brcmfmac43241b0-sdio.bin
firmware: brcm/brcmfmac43143-sdio.bin
firmware: brcm/brcmfmac4373.bin
firmware: brcm/brcmfmac43569.bin
firmware: brcm/brcmfmac43242a.bin
firmware: brcm/brcmfmac43236b.bin
firmware: brcm/brcmfmac43143.bin
srcversion: A208FF46FF4FDC4A48B10E7
alias: usb:v04B4p0BDCddcdscdpiciscipin
alias: usb:v0A5Cp0BDCddcdscdpiciscipin
alias: usb:v043Ep3101ddcdscdpiciscipin
alias: usb:v04B4pBD29ddcdscdpiciscipin
alias: usb:v13B1p0039ddcdscdpiciscipin
alias: usb:v0A5CpBD27ddcdscdpiciscipin
alias: usb:v0A5CpBD1Fddcdscdpiciscipin
alias: usb:v0A5CpBD17ddcdscdpiciscipin
alias: usb:v0A5CpBD1Eddcdscdpiciscipin
alias: sdio:cv02D0d4373
alias: sdio:cv02D0d4356
alias: sdio:cv02D0d4354
alias: sdio:cv02D0dA9BF
alias: sdio:cv02D0d4345
alias: sdio:cv02D0dA9A6
alias: sdio:cv02D0d4339
alias: sdio:cv02D0d4335
alias: sdio:cv02D0dA9A4
alias: sdio:cv02D0dA962
alias: sdio:cv02D0dA94D
alias: sdio:cv02D0dA94C
alias: sdio:cv02D0d4334
alias: sdio:cv02D0d4330
alias: sdio:cv02D0d4329
alias: sdio:cv02D0d4324
alias: sdio:cv02D0dA887
depends: brcmutil,cfg80211
name: brcmfmac
vermagic: 4.19.66-Re4son-v7+ SMP mod_unload modversions ARMv7 p2v8
parm: txglomsz:Maximum tx packet chain size [SDIO] (int)
parm: debug:Level of debug output (int)
parm: p2pon:Enable legacy p2p management functionality (int)
parm: feature_disable:Disable features (int)
parm: alternative_fw_path:Alternative firmware path (string)
parm: fcmode:Mode of firmware signalled flow control (int)
parm: roamoff:Do not use internal roaming engine (int)
parm: iapp:Enable partial support for the obsoleted Inter-Access Point Protocol (int)
parm: ignore_probe_fail:always succeed probe for debugging (int)

It seems like the mismatch may be that I'm building for 4.19.66-Re4son-v7+ instead of 4.19.66-Re4son-v7l+ but I don't know how to force it.

andrewbeard picture andrewbeard on 11 Nov 2019

Yes, you have to use 4.19.66-Re4son-v7l+ headers. Don't you have that folder under /usr/src/?

DrSchottky picture DrSchottky on 11 Nov 2019

Nope

root@pwnpi4:/usr/src# ls -al /usr/src
total 16
drwxr-xr-x 4 root root 4096 Nov 11 15:04 .
drwxr-xr-x 10 root root 4096 Jul 9 20:03 ..
drwxr-xr-x 24 root root 4096 Nov 11 16:00 linux-headers-4.19.66-Re4son-v7+
drwxr-xr-x 10 root root 4096 Nov 11 15:32 nexmon

andrewbeard picture andrewbeard on 11 Nov 2019

Oh, that's he problem.
Is kalipi-kernel-headers package installed? if so, i think you need to download re4son kernel source and make the header pkg for your arch.
@evilsocket are there any other reasons beyond built-in nexmon for using kalipi-kernel?

DrSchottky picture DrSchottky on 11 Nov 2019

Yup, the package is installed. For some reason it sticks everything in linux-headers-4.19.66-Re4son-v7+ and not linux-headers-4.19.66-Re4son-v7l+, though. It looks like that might just be an endianess flag, so I'd be a little surprised if the source was different. I don't think I have an actual kernel config to tell it otherwise, though.

andrewbeard picture andrewbeard on 11 Nov 2019

Just to add another complication is looks like was just a new kalipi kernel update pushed to the apt repos. Info looks the same, so I would be shocked if the problem didn't continue:

abeard@pwnpi4:/usr/src/nexmon $ uname -a
Linux pwnpi4 4.19.81-Re4son-v7l+ #1 SMP Wed Nov 6 13:10:35 AEDT 2019 armv7l GNU/Linux
abeard@pwnpi4:/usr/src/nexmon $ dmesg | grep 7.45
[ 2.736258] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Oct 8 2018 09:18:41 version 7.45.154 (nexmon.org: 2.2.2-195-g01e0-1) FWID 01-4fbe0b04

andrewbeard picture andrewbeard on 11 Nov 2019

What happen if you try ignore vermagic by adding -f to insmod?

DrSchottky picture DrSchottky on 11 Nov 2019

No luck:

root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# insmod -f brcmfmac_4.19.y-nexmon/brcmfmac.ko
insmod: ERROR: could not insert module brcmfmac_4.19.y-nexmon/brcmfmac.ko: Invalid module format

andrewbeard picture andrewbeard on 11 Nov 2019

So from what I can tell the source code for the kernels should be identical. I need the kernel config for the v7l+ version, though. None of the config extraction options seem to be built in, so I'm kind of at a loss as to where I can get it from.

andrewbeard picture andrewbeard on 12 Nov 2019

It seems like this may be an issue specific to the pi4. From what I can tell it's the only system that uses the v7l+ kernel. The build issue is probably the same thing that happened here:
https://github.com/seemoo-lab/nexmon/issues/321

I'm kind of dead in the water at the moment without being able to find the right kernel source or config. It looks like the kernel is being installed from an apt repo here:
http://http.re4son-kernel.com/re4son/

But the re4son website is pretty out of date and so I'm failing miserably trying to find the source package that corresponds to the installed kernel so far. It doesn't not appear that there's a package for the kalipi full kernel source, just the headers (that I already have, and don't include v7l+).

andrewbeard picture andrewbeard on 12 Nov 2019

Yes, v7l+ (where l stands for lpae) targets Rpi4 beacuse of its larger RAM. The problem is that the kalipi-kernel-headers package is incomplete since it contains only headers for v7+.
If you look at re4son GH repo you can find their kernel and build script, so you should (never tried) to build a preper headers deb for your arch.
But imho switching to a 3rd party kernel is more a hassle than a benefit, that's why I asked Simone if it's needed.

DrSchottky picture DrSchottky on 12 Nov 2019

@andrewbeard I patched the buildscripts to include v7l+ headers in the package.
Try this
It's not tested (I don't have a kalipi setup available atm) and the builder has many bugs, so it's an attempt.

DrSchottky picture DrSchottky on 12 Nov 2019

COLLECTING STATISTICS read /usr/src/nexmon/STATISTICS.md for more information
COMPILING src/version.c => obj/version.o (details: log/compiler.log)
GENERATING LINKER FILE gen/nexmon.pre => gen/nexmon.ld
GENERATING LINKER FILE gen/nexmon.pre => gen/flashpatches.ld
LINKING OBJECTS => gen/patch.elf (details: log/linker.log, log/linker.err)
GENERATING MAKE FILE gen/nexmon.pre => gen/nexmon.mk
GENERATING MAKE FILE gen/nexmon.pre => gen/flashpatches.mk
APPLYING FLASHPATCHES gen/flashpatches.mk => brcmfmac43455-sdio.bin (details: log/flashpatches.log)
APPLYING PATCHES gen/nexmon.mk => brcmfmac43455-sdio.bin (details: log/patches.log)
BUILDING DRIVER for kernel 4.19 brcmfmac_4.19.y-nexmon/brcmfmac.ko (details: log/driver.log)
/bin/sh: 1: scripts/basic/fixdep: Exec format error
make[2]: * [scripts/Makefile.build:303: /usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon/brcmfmac_4.19.y-nexmon/chip.o] Error 2
make[2]: Waiting for unfinished jobs....
/bin/sh: 1: scripts/basic/fixdep: Exec format error
make[2]: [scripts/Makefile.build:303: /usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon/brcmfmac_4.19.y-nexmon/cfg80211.o] Error 2
make[1]: [Makefile:1519: _module_/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon/brcmfmac_4.19.y-nexmon] Error 2
make: ** [Makefile:52: brcmfmac.ko] Error 2

root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# file /usr/src/linux-headers-4.19.81-Re4son-v7l+/scripts/basic/fixdep
/usr/src/linux-headers-4.19.81-Re4son-v7l+/scripts/basic/fixdep: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=f96cf37e4ab3abdfa90880655b262c4ae72c937a, for GNU/Linux 3.2.0, not stripped

It looks like the scripts folder in the headers has locally compiled binaries, but since the headers were cross-compiled on an x64 system it's causing the nexmon build to shit the bed.

andrewbeard picture andrewbeard on 12 Nov 2019

Hey @DrSchottky I would like to help if I can as well.

Hardware:
Raspberry Pi 0w

Output from uname -a:
Linux pwnagotchi 4.19.66-Re4son+ #1 Sun Aug 18 13:42:02 AEST 2019 armv6l GNU/Linux

dmesg | grep 7.45:
[ 14.339006] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43430/1 wl0: Aug 7 2017 00:46:29 version 7.45.41.46 (r666254 CY) FWID 01-f8a78378

modinfo /lib/modules/4.19.66-Re4son+/kernel/drivers/net/wireless/broadcom/brcm80211/brcmfmac/brcmfmac.ko:

filename: /lib/modules/4.19.66-Re4son+/kernel/drivers/net/wireless/broadcom/brcm80211/brcmfmac/brcmfmac.ko
license: Dual BSD/GPL
description: Broadcom 802.11 wireless LAN fullmac driver.
author: Broadcom Corporation
firmware: brcm/brcmfmac4373-sdio.bin
firmware: brcm/brcmfmac4356-sdio.bin
firmware: brcm/brcmfmac4354-sdio.bin
firmware: brcm/brcmfmac43455-sdio.bin
firmware: brcm/brcmfmac43430-sdio.bin
firmware: brcm/brcmfmac43430a0-sdio.bin
firmware: brcm/brcmfmac4339-sdio.bin
firmware: brcm/brcmfmac43362-sdio.bin
firmware: brcm/brcmfmac4335-sdio.bin
firmware: brcm/brcmfmac43341-sdio.bin
firmware: brcm/brcmfmac43340-sdio.bin
firmware: brcm/brcmfmac4334-sdio.bin
firmware: brcm/brcmfmac4330-sdio.bin
firmware: brcm/brcmfmac4329-sdio.bin
firmware: brcm/brcmfmac43241b5-sdio.bin
firmware: brcm/brcmfmac43241b4-sdio.bin
firmware: brcm/brcmfmac43241b0-sdio.bin
firmware: brcm/brcmfmac43143-sdio.bin
firmware: brcm/brcmfmac4373.bin
firmware: brcm/brcmfmac43569.bin
firmware: brcm/brcmfmac43242a.bin
firmware: brcm/brcmfmac43236b.bin
firmware: brcm/brcmfmac43143.bin
srcversion: 389E0869FFB2AAD06E0D34E
alias: usb:v04B4p0BDCddcdscdpiciscipin
alias: usb:v0A5Cp0BDCddcdscdpiciscipin
alias: usb:v043Ep3101ddcdscdpiciscipin
alias: usb:v04B4pBD29ddcdscdpiciscipin
alias: usb:v13B1p0039ddcdscdpiciscipin
alias: usb:v0A5CpBD27ddcdscdpiciscipin
alias: usb:v0A5CpBD1Fddcdscdpiciscipin
alias: usb:v0A5CpBD17ddcdscdpiciscipin
alias: usb:v0A5CpBD1Eddcdscdpiciscipin
alias: sdio:cv02D0d4373
alias: sdio:cv02D0d4356
alias: sdio:cv02D0d4354
alias: sdio:cv02D0dA9BF
alias: sdio:cv02D0d4345
alias: sdio:cv02D0dA9A6
alias: sdio:cv02D0d4339
alias: sdio:cv02D0d4335
alias: sdio:cv02D0dA9A4
alias: sdio:cv02D0dA962
alias: sdio:cv02D0dA94D
alias: sdio:cv02D0dA94C
alias: sdio:cv02D0d4334
alias: sdio:cv02D0d4330
alias: sdio:cv02D0d4329
alias: sdio:cv02D0d4324
alias: sdio:cv02D0dA887
depends: brcmutil,cfg80211
intree: Y
name: brcmfmac
vermagic: 4.19.66-Re4son+ mod_unload modversions ARMv6 p2v8
parm: txglomsz:Maximum tx packet chain size [SDIO] (int)
parm: debug:Level of debug output (int)
parm: p2pon:Enable legacy p2p management functionality (int)
parm: feature_disable:Disable features (int)
parm: alternative_fw_path:Alternative firmware path (string)
parm: fcmode:Mode of firmware signalled flow control (int)
parm: roamoff:Do not use internal roaming engine (int)
parm: iapp:Enable partial support for the obsoleted Inter-Access Point Protocol (int)
parm: ignore_probe_fail:always succeed probe for debugging (int)

hackabean picture hackabean on 12 Nov 2019

I figured I'd give it one more jackass try and replace all the x64 script binaries with ARM ones from my local build tree. They shouldn't be dependent on the patchset, right?

Module built (yay!), but still wouldn't insert (boo):

  COLLECTING STATISTICS read /usr/src/nexmon/STATISTICS.md for more information
  COMPILING src/version.c => obj/version.o (details: log/compiler.log)
  GENERATING LINKER FILE gen/nexmon.pre => gen/nexmon.ld
  GENERATING LINKER FILE gen/nexmon.pre => gen/flashpatches.ld
  LINKING OBJECTS => gen/patch.elf (details: log/linker.log, log/linker.err)
  GENERATING MAKE FILE gen/nexmon.pre => gen/nexmon.mk
  GENERATING MAKE FILE gen/nexmon.pre => gen/flashpatches.mk
  APPLYING FLASHPATCHES gen/flashpatches.mk => brcmfmac43455-sdio.bin (details: log/flashpatches.log)
  APPLYING PATCHES gen/nexmon.mk => brcmfmac43455-sdio.bin (details: log/patches.log)
  BUILDING DRIVER for kernel 4.19 brcmfmac_4.19.y-nexmon/brcmfmac.ko (details: log/driver.log)
  COPYING brcmfmac43455-sdio.bin => /lib/firmware/brcm/brcmfmac43455-sdio.bin
  RELOADING brcmfmac
insmod: ERROR: could not insert module brcmfmac_4.19.y-nexmon/brcmfmac.ko: Invalid module format
make: *** [Makefile:190: install-firmware] Error 1
root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# insmod -f brcmfmac_4.19.y-nexmon/brcmfmac.ko 
insmod: ERROR: could not insert module brcmfmac_4.19.y-nexmon/brcmfmac.ko: Invalid module format

The modules themselves look pretty damn similar. I'm really at a loss what the difference could be:

root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# modinfo brcmfmac_4.19.y-nexmon/brcmfmac.ko > /tmp/newmod
root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# modinfo /lib/modules/4.19.81-Re4son-v7l+/kernel/drivers/net/wireless/broadcom/brcm80211/brcmfmac/brcmfmac.ko > /tmp/oldmod
root@pwnpi4:/usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon# diff /tmp/newmod /tmp/oldmod
1c1
< filename:       /usr/src/nexmon/patches/bcm43455c0/7_45_189/nexmon/brcmfmac_4.19.y-nexmon/brcmfmac.ko
---
> filename:       /lib/modules/4.19.81-Re4son-v7l+/kernel/drivers/net/wireless/broadcom/brcm80211/brcmfmac/brcmfmac.ko
13a14
> firmware:       brcm/brcmfmac43341-sdio.bin
27c28
< srcversion:     A208FF46FF4FDC4A48B10E7
---
> srcversion:     389E0869FFB2AAD06E0D34E
54a56
> intree:         Y
andrewbeard picture andrewbeard on 12 Nov 2019

This header thing is becoming a pain in the ass.
Since pwnagotchi can be installed on a working system why don't we burn a clean Raspbian Lite image, do a couple of nexmon tests and then install the software?

DrSchottky picture DrSchottky on 12 Nov 2019

Something I just noticed, that failedinstall process looks like it _did_ overwrite the firmware in /lib/firmware/brcm/brcmfmac43455-sdio.bin and when I rebooted the pi it did load the new firmware. 

[    2.664068] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    2.673602] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Nov 12 2019 09:40:03 version 7.45.189 (nexmon.org: 2.2.2-273-gaaa80e-dirty-7) FWID 01-e1db26e2

Unfortunately the channel change errors now start _immediately_, about 14 second after I boot:

[   14.299243] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[   15.312016] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53290, -52
[   16.325362] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53294, -52
[   30.549909] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[   31.567560] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53290, -52
[   32.584209] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53294, -52
[   62.261185] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[   63.278705] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53290, -52
[   64.299974] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53294, -52
[   78.550006] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[   79.568249] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53290, -52
[   80.586455] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53294, -52
[  128.297310] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[  131.399334] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[  134.503219] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[  137.631244] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52
[  140.753593] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53286, -52

It looks like new firmware + old module is even worse than old firmware + old module.

andrewbeard picture andrewbeard on 12 Nov 2019

@DrSchottky Sounds like a good plan. I'll see if I've got a spare sd card I can start from.

andrewbeard picture andrewbeard on 12 Nov 2019

OK, I've got a fresh Raspbian lite image + the nexmon driver installed with the 7.45.189 firmware. What do you need me to try?

andrewbeard picture andrewbeard on 13 Nov 2019

install aircrack-ng, remove wpasupplicant, activate mon0 and let airodump run for a while.
If it won't crash we can consider monitor mode working and go to the next step (you can also repeat the test with nexmon 7.45.154)

DrSchottky picture DrSchottky on 13 Nov 2019

So, I tried a couple things last night:

  • airodump seems to run without error for 6+ hours, but it picks up very little. Few stations detected.
  • I tried running bettercap but it produces an error when initializing the interface complaining that it couldn't set the channel
  • I tried to debug that a bit and boiled it down to this command bettercap is running:
root@pwnpi4-ng:/home/abeard# iwconfig mon0 channel 1
Error for wireless request "Set Frequency" (8B04) :
    SET failed on device mon0 ; Device or resource busy.

So, the good news is that it isn't locking up. The bad news is that I'm not really sure I'm getting to the fully functional point (even without trying injection).

andrewbeard picture andrewbeard on 13 Nov 2019

Ok, airodump can't find many stations because it can't do channel hopping.
So reboot your device and make sure wpa_supplicant isn't running/isn't installed and start mon0 this way

sudo ip link set dev wlan0 down
sudo iwconfig wlan0 channel 6
sudo iw phy `iw dev wlan0 info | gawk '/wiphy/ {printf "phy" $2}'` interface add mon0 type monitor
sudo ip link set dev mon0 up
sudo iwconfig mon0 channel 6

Now you should have mon0 set on channel 6 (and you should be able to change the channel with iwconfig), so start airdoump telling it to work on both 2.4 and 5.8 bands
sudo airodump-ng --band abg mon0

DrSchottky picture DrSchottky on 13 Nov 2019

MUCH better. It immediately popped up with 5x more networks than it detected all night. I'll let this run for a bit and see if it craps out.

andrewbeard picture andrewbeard on 13 Nov 2019

25 hours running airodump with no issues. Running bettercap right now as a test. What should I try next?

andrewbeard picture andrewbeard on 14 Nov 2019

Perfect.
I'd try to turn it into a complete pwnagotchi by following the manual install procedure

DrSchottky picture DrSchottky on 14 Nov 2019 
[95101.861954] brcmfmac: brcmf_sdio_hostmail: mailbox indicates firmware halted
[95123.800146] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[95126.360187] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[95128.920228] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110
[95131.480295] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[95134.520352] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[95137.560407] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[95140.600478] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4102, -110
[95143.640511] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4103, -110
[95146.680603] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4105, -110
[95149.720655] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4107, -110
[95152.760713] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4108, -110
[95155.800785] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4109, -110
[95158.840877] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53292, -110
[95161.880885] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53296, -110
[95164.920952] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53308, -110
[95167.960983] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53352, -110
[95171.001052] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53360, -110
[95174.041139] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53364, -110
[95177.081197] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53368, -110
[95180.121272] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53376, -110
[95183.161338] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=53384, -110
[95186.201382] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[95189.241458] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[95191.801500] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[95191.801517] brcmfmac: brcmf_cfg80211_get_channel: chanspec failed (-110)
[95194.361561] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4100, -110
[95207.321832] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[95207.321844] brcmfmac: brcmf_cfg80211_get_channel: chanspec failed (-110)
[95209.881888] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110
[95212.921901] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4098, -110
[95215.961962] brcmfmac: brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed w/status -110
[95215.961969] brcmfmac: brcmf_cfg80211_get_channel: chanspec failed (-110)
[95218.522018] brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4099, -110

I'm going to reboot it and try again from a fresh start, but it looks like it went blind after less than an hour.

andrewbeard picture andrewbeard on 14 Nov 2019

I tried disabling the advertise, deauth, and associate options just to see if removing all the packet injection stuff would make a difference. The short version is that it doesn't. airodump never causes it to lock up, but bettercap driven by pwnagochi causes it to die almost immediately.

andrewbeard picture andrewbeard on 15 Nov 2019

@andrewbeard is bettercap standalone (w/ injection enabled but not driven by pwnagotchi) causing any problem?
@evilsocket is bettercap doing any kind of injection under the hood or anything special?

DrSchottky picture DrSchottky on 15 Nov 2019

@DrSchottky yes, bettercap is the component that is:

evilsocket picture evilsocket on 15 Nov 2019

@evilsocket ok, but andrew told he turned off deauth/association, so I meant in addition to those.
I want to figure out what are the differences between airodump and bettercap (w/o injections) to understand why one works while the other crashes the fw.

DrSchottky picture DrSchottky on 15 Nov 2019

well there's also pwngrid that's injecting stuff on the same interface for the mesh protocol, but this bug happened way before pwngrid or the mesh were a thing ... maybe it might be the channel hopping? bettercap uses iwconfig to hop and that might be stressful for the rpi0w every few milliseconds ... it'd be interesting to try to disable hopping all together (by setting ai.enabled to false and personality channels to just 1 or whatever).

evilsocket picture evilsocket on 15 Nov 2019

By default airodump hops every 250ms, so the only difference that comes to my mind is that you do it through wext, while airodump uses (AFAIK) nl.
@andrewbeard could you try to disable hopping as evlisocket suggested?

DrSchottky picture DrSchottky on 15 Nov 2019

@DrSchottky yes i think it's not the timing but the methodology ... i also don't think airodump uses calls to the iwconfig binary as i do in bettercap, pretty sure there's a cleaner way to do it :D

evilsocket picture evilsocket on 15 Nov 2019

@evilsocket exactly, it uses a totally different software stack (wext vs nl80211).
If turning off the hopping solves the problem the easiest fix is to replace iwconfig with iw in bettercap.

DrSchottky picture DrSchottky on 15 Nov 2019
👍1

@andrewbeard any news?

DrSchottky picture DrSchottky on 18 Nov 2019

I get the "brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4109, -110" errors after about an hour when I have bluetooth enabled in the config, but bluetooth does not connect for whatever reason. pi0w, temp shows as 41c. pwnagotchi process locks and goes defunct when this happens. Is this the same bug?

Edit: Just saw the link to #494

neurovish picture neurovish on 22 Nov 2019

I got it "working" by writing a little script that reloads the driver everytime it crashes

TJFJefroy picture TJFJefroy on 9 Dec 2019

I got it "working" by writing a little script that reloads the driver everytime it crashes

do you mind posting your script? "modprobe -r brcmfmac && modprobe brcmfmac && sleep 2 && /usr/bin/monstart && systemctl restart pwnagotchi" for the driver reload seems to kinda work some of the time for me but not always. wondering if you're doing something better.

soaringswine picture soaringswine on 4 Jan 2020

echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/unbind
echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/unbind

echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/bind
echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/bind

this is working for me

TJFJefroy picture TJFJefroy on 5 Jan 2020

echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/unbind
echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/unbind

echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/bind
echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/bind

this is working for me

are you running this while pwnagotchi service is still running? are you doing anything more than the driver un/binding like trying to add mon0 back or bring it up? and are you monitoring dmesg for brcmf_cfg80211_nexmon_set_channel errors to trigger it?

because it doesn't look like mon0 is brought back up after the binds and trying to "/usr/bin/monstart" or "iw phy phy0 interface add mon0 type monitor && ifconfig mon0 up" gives a "command failed: No such file or directory (-2)" error for me.

soaringswine picture soaringswine on 6 Jan 2020

echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/unbind
echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/unbind
echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/bind
echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/bind
this is working for me

are you running this while pwnagotchi service is still running? are you doing anything more than the driver un/binding like trying to add mon0 back or bring it up? and are you monitoring dmesg for brcmf_cfg80211_nexmon_set_channel errors to trigger it?

because it doesn't look like mon0 is brought back up after the binds and trying to "/usr/bin/monstart" or "iw phy phy0 interface add mon0 type monitor && ifconfig mon0 up" gives a "command failed: No such file or directory (-2)" error for me.

ah, the error is because phy0 doesn't exist. when you rebind the driver, it increments the phy instance, so phy1, phy2 instead of phy0. this command seems to be more consistent (though I don't know if functionally it's any different): "iw dev wlan0 interface add mon0 type monitor && ifconfig mon0 up"

soaringswine picture soaringswine on 6 Jan 2020

here's my nexmon blindness error watchdog script I whipped together that seems to work pretty well until the bug is fixed. needs to be run as root. seems to do its job from the testing I've done:

#!/usr/bin/env bash
set -euo pipefail
red=$(tput setaf 1)
green=$(tput setaf 2)
yellow=$(tput setaf 3)
reset=$(tput sgr0)

if ! [ $(id -u) = 0 ]; then
   echo ${red}"need to run this as root!${reset}"
   exit 1
fi

echo "${yellow}patiently waiting for nexmon blindness error (https://github.com/evilsocket/pwnagotchi/issues/267)..${reset}"
tail -n 0 -f /var/log/kern.log | while read -r kernel_log_line; do
    if [[ $kernel_log_line == *"brcmf_cfg80211_nexmon_set_channel: Set Channel failed"* || $kernel_log_line == *"brcmf_cfg80211_get_tx_power: error"* || $kernel_log_line == *"brcmf_proto_bcdc_query_dcmd: brcmf_proto_bcdc_msg failed"* ]]; then
        echo "${red}caught blindness error, reloading nexmon driver..${reset}"
        set -x
        echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/unbind
        sleep 1
        echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/unbind
        sleep 3
        echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/bind
        sleep 3
        echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/bind
        sleep 3
        modprobe brcmfmac #this modprobe may not be needed, but it seems to help sometimes.
        sleep 3
        #running a couple extra binds here in case the modprobe made a difference. it can't hurt!
        echo -n "mmc1:0001:1" > /sys/bus/sdio/drivers/brcmfmac/bind || true
        sleep 3
        echo -n "mmc1:0001:2" > /sys/bus/sdio/drivers/brcmfmac/bind || true
        sleep 3
        iw dev wlan0 interface add mon0 type monitor
        sleep 3
        ifconfig mon0 up
        set +x
        date
        echo "${green}nexmon driver reloaded and mon0 brought back up! looping until next error..${reset}"
    fi
done
soaringswine picture soaringswine on 6 Jan 2020
👍3

I just read through this thread I was confident @DrSchottky and @andrewbeard where on to something.

myusuf3 picture myusuf3 on 29 Jan 2020

@myusuf3 I never heard anything back.
What should be tried is written a few posts above, so feel free to try and give a feedback.

DrSchottky picture DrSchottky on 29 Jan 2020

Seems like there is a fix available (https://github.com/raspberrypi/linux/issues/2453) ...trying this right now

dadav picture dadav on 12 Apr 2020

not working, but there've been some kernel updates. I'll try them

dadav picture dadav on 12 Apr 2020

any updates guys? facing similar issues on RPi0w.

"brcmfmac: brcmf_cfg80211_nexmon_set_channel: Set Channel failed: chspec=4097, -110"

airodum has not picked up anything so far, and the bug happens right away for me.
Although iwlist scan does work properly.
The way I bring the interface up does not seem to make a difference.
I haven't tried disabling hopping or increasing delay between two channels yet

The people from pwnagotchi managed to get it working, I should be able to get better results, there must be something I'm missing here

I switched to the firmware ("latest"?) provided by cypress (sdio.bin..) from the .tar.gz someone uploaded here, did not improve things, did not make it worse either

I'm using the module from DrShottsky's fork (4.19), here's the log when module is registered or manually inserted

dmesg | grep brcm
[ 3.593952] brcmfmac: loading out-of-tree module taints kernel.
[ 3.631407] brcmfmac: F1 signature read @0x18000000=0x1541a9a6
[ 3.647217] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[ 3.677400] usbcore: registered new interface driver brcmfmac
[ 4.013755] brcmfmac: brcmf_sdio_bus_preinit: before brcmf_sdio_debugfs_create
[ 4.029060] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43430-sdio for chip BCM43430/1
[ 4.060116] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM43430/1 wl0: Oct 22 2019 01:57:42 version 7.45.98.94 (r723000 CY) FWID 01-73a5ed62
[ 4.285213] brcmfmac: brcmf_bus_started: before brcmf_debugfs_add_entry

iwconfig wlan0

wlan0 IEEE 802.11 Mode:Monitor Frequency:2.412 GHz Tx-Power=31 dBm
Retry short limit:7 RTS thr:off Fragment thr:off
Power Management:on

iwconfig wlan0 power off
Error for wireless request "Set Power Management" (8B2C) :
SET failed on device wlan0 ; Invalid argument.

gwbres picture gwbres on 26 May 2020

For the Pi 4, could the latest nexmon firmware fix this issue?
bcm43455c0 | 7_45_206
I am trying to test it myself but I am stuck trying to figure out why airodump-ng does not find anything.

meliodasren picture meliodasren on 20 Sep 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

evilsocket picture evilsocket  路  8Comments
pingywon picture pingywon  路  6Comments
Shadyman picture Shadyman  路  7Comments
miotislucifugis picture miotislucifugis  路  6Comments
sayak-brm picture sayak-brm  路  3Comments