Norns: Move wifi.sh functionality into norns code

[ranch-verdin]

https://github.com/tehn/norns-image/blob/master/scripts/wifi.sh could be done a lot better in lua.

@artfwo & I discussed issues a bit on slack (and a bug report) but in summary.... Any piece of code containing 5 second sleeps is always going to cause trouble down the line!

[artfwo]

There are several options to manage wifi from lua:

• https://github.com/koreader/lj-wpaclient, which depends on #31 though.
• using lua for shell scripting
• anything else?

[simonvanderveldt]

One thing to keep in mind is that this will depend on how wlan/wifi is managed, there are several options from just handling ifup/ifdown yourself to using something large like NetworkManager.

[tehn]

i moved wifi.sh into the norns repo in the meantime.

it seems lua shell scripting (with coroutines? instead of all the sleep) is a good path. would be good to know what facilities will be available with the buildroot install

[tehn]

breakthrough idea! (of minor consequence)

ssid and psk could be stored in state.lua, in the user folder. hence editable via the web-editor, providing a method to get onto a router (provided you're first connected via hotspot.)

yeah? yeah?

[simonvanderveldt]

@tehn Why do you want to store the credentials anywhere else than in /etc/wpa_supplicant.conf?

Btw you can just read the wlan status from /sys/class/net/wlan0/operstate

# cat /sys/class/net/wlan0/operstate 
down
# ifup wlan0
wpa_supplicant v2.6
...
... some noise from wpa_supplicant and setting up the wlan
...
# cat /sys/class/net/wlan0/operstate 
up

[tehn]

you are 100% correct. i drank too much decaf tea (not joking). was attempting to figure out a UI solution to entering the ssid/psk. what we should do is just have a function in the REPL, ie

wifiwizard(ssid,psk)

and then that will store it in wpa_supplicant.

so, let's redo the whole system and have it live in lua. (sorry @ranch-verdin!)

[ranch-verdin]

no probs - the wifi.sh approach seemed reasonable enough at the outset. However it does makes sense to move that functionality into lua now.

[ranch-verdin]

@simonvanderveldt digging back into wifi.sh I remembered why I'm not putting credentials directly into wpa_supplicant.conf . The janky-looking way I'm doing it makes it easy to cleanly switch between passwordless & password-protected networks. See here https://github.com/ranch-verdin/norns/blob/dev/wifi.sh#L85-L89 .

Also I felt like the script shouldn't try to associate to a network if you just want to run wifi.sh scan. This consideration may be wrongheaded/inconsequential.

wifi.sh is far from beautiful, and I would definitely like to know if there's a better way. Is lua definitely a suitable tool for monitoring/managing a few daemons & some light text-munging? Can't believe I'm saying this, but maybe the cleanest way would be a perl script!?

[tehn]

i'd prefer not to introduce another language at this point (perl) but i'm glad to hear the script is getting more robust. thank you for keeping on it!

if it's working well at this point (seems to, for me) i'm actually more concerned about getting together a better wifi selection UI (i'll do this), which will indeed require a "wrong password" status, and also employ the scan. https://github.com/catfact/norns/issues/193

[tehn]

consensus has been to move over to NetworkManager.

here are my notes on early results:

sudo apt-get install network-manager
sudo nano /etc/NetworkManager/NetworkManager.conf

change the line managed=false to managed=true

sudo nano /etc/network/interfaces
Delete everything in that file and copy and paste the following:

auto lo
iface lo inet loopback

connect:
nmcli device wifi connect nnnnnnnn+ password "22222222"
hotspot:
nmcli dev wifi hotspot ifname wlan0 ssid norns password "22222222"

hotspot is busted:

• hotspot kicks out after connection, asks for PW again
• weird self-assigned IP (we need 172.24.1.1)
• after disabling hotspot, normal network won't come up until wifi dongle is pulled/inserted (edited)
• something is up with dnsmasq

---

norns menu UI needs some refinement.

router name / hotspot / off ----- ip @ strength

• off
• hotspot
• reconnect
• new connection

"new connection" should show a list of detected networks (from scan) and then allow a password to be entered. if successful, save to list.

"reconnect" should display a list of known networks.

on boot, resume the last known network state (off, hotspot, or found network)

wifi plug hotswap?

wifi.lua needs to be updated for new methods for wifi information extracting from nmcli (etc). ie, scan or current status, signal strength

[simonvanderveldt]

Looking at this https://github.com/mariospr/NetworkManager/tree/master/examples/lua/lgi it seems that it's possible to interface with NetworkManager from lua directly instead of shelling out.

[tehn]

wow, thanks for finding this!

[ngwese]

i just took a quick peak at the above examples; it looks like they are all based around lgi which afaik is this: https://github.com/pavouk/lgi

lgi appears to be a generic interface for exposing anything based on GObject (GLib Object System) to lua. ...my naive assumption is that using lgi would end up dragging in portions of gnome and is a fairly big hammer.

EDIT: shelling out to nmcli seems like a more direct path (despite the potential hackiness)

[ngwese]

@tehn - above you mentioned:

weird self-assigned IP (we need 172.24.1.1)

why do we need that to be the hotspot address? is it just because that is what we have in documentation?

[ngwese]

progress: https://github.com/monome/norns/commits/feature/network-manager

there are some ui glitches still but for existing connections things seem to be working reasonably well. ui stuff which is probably needed:

• detail mode on HOME page won't show ip addresses until one navigates to the wifi menu and back
• when enabling a network it would be good to have a spinner or some ui to show that it is working

i'm going to start experimenting w/ a new connection wizard.

[tehn]

@tehn - above you mentioned:

weird self-assigned IP (we need 172.24.1.1)

why do we need that to be the hotspot address? is it just because that is what we have in documentation?

you're correct, it doesn't actually need to be that address. it just needs to be the same every time, and i can update the documentation.

[tehn]

looking forward to checking everything out!

what steps did you take to disable existing network infrastructure? did you need to disable some services, and does dnsmasq etc work as expected?

when enabling a network it would be good to have a spinner or some ui to show that it is working

good call. i can do that.

detail mode on HOME page won't show ip addresses until one navigates to the wifi menu and back

this can be fixed, basically the IP address needs to be scraped at the right moment to show it on the screen. it's probably doing it at the wrong time presently.

i'm going to start experimenting w/ a new connection wizard.

i can help implement any UI changes, just let me know

[ngwese]

@tehn some ui changes are already in place on the branch referenced above. there is enough there to add and delete network connections. the wifi page feels cramped to me, it’s also the page that could use a spinner when connecting to a router. the wifi page continuously polls the “wlan0” device so even if the activation of a connection isn’t successful it will eventually update the status and menu to reflect that - if can take up to ~30 senconds for activation of the router connections to fail.

one potential option to consider is designating the hotspot connection as autoconnect which means that it could be made to fall into hotspot mode if the router network activation fails. if that seems desirable i can experiment to connection priorities (setting router based connections higher). what i don’t see immediate support for is choosing a connection based on wifi strength.

to get things setup i believe all i did was install the network manager package and comment out everything in /etc/network/interfaces.

if you have existing networks defined they should be selectable as networks on the wifi page. when matron first starts it looks for a connection called HOTSPOT, if that doesn’t exist a new hotspot connection is defined. the hotspot mode hasn’t been tested yet beyond enabling it. i will look into setting additional properties like the IP address.

[simonvanderveldt]

lgi appears to be a generic interface for exposing anything based on GObject (GLib Object System) to lua. ...my naive assumption is that using lgi would end up dragging in portions of gnome and is a fairly big hammer.

EDIT: shelling out to nmcli seems like a more direct path (despite the potential hackiness)

I haven't tried on norns/an RPi, but the deps seem pretty minimal/limited.

root@33f31c431882:/# apt-get install lua-lgi
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  gir1.2-glib-2.0 libffi6 libgirepository-1.0-1 libglib2.0-0 libglib2.0-data libicu57 libxml2 sgml-base shared-mime-info xdg-user-dirs xml-core
Suggested packages:
  sgml-base-doc debhelper
The following NEW packages will be installed:
  gir1.2-glib-2.0 libffi6 libgirepository-1.0-1 libglib2.0-0 libglib2.0-data libicu57 libxml2 lua-lgi sgml-base shared-mime-info xdg-user-dirs xml-core
0 upgraded, 12 newly installed, 0 to remove and 0 not upgraded.
Need to get 15.0 MB of archives.
After this operation, 54.5 MB of additional disk space will be used.

On the other hand you've already made some progress with the shell-based stuff, so maybe quicker to go that way, not sure how well both work compared to each other.

[ngwese]

good to know. that’s fewer deps than i would have expected. early emphasis on trying to minimize the os footprint probably has me over overcompensating when it comes to dependencies.

apologies for jumping on this if you really had your heart set on knocking something out. testing the new menu stuff got me inspired to take a second look. constantly enabling wifi after each reboot and the inability to restart matron from maiden have contributed to a lot of my frustration when trying to do something ambitious script/lib wise in the past so when i saw light i drove quickly at it.

parsing the output of nmcli is actually fairly robust since the —terse and —fields flags seem intended to facilitate just that.

would be more that happy to see that choice revisited should there be interest. one aspect of using nmcli which is handy is that it is easy to get async behavior by backgrounding commands when needed, not sure is the lgi route would require threading to keep the menu ui responsive.

[ngwese]

i believe i have a line on controlling the IP address such that it lands in the 172.24.1.1/24 range. generally its has always come up as 172.24.1.1 but there is no guarantee

hotspot is busted:

hotspot kicks out after connection, asks for PW again
after disabling hotspot, normal network won't come up until wifi dongle is pulled/inserted (edited)
something is up with dnsmasq

hotspot is causing me to loose my hair. i can get anything to connect to it - same symptoms like getting kicked back and asked for a PW. i have another small arm machine running with the exact same hotspot configuration (via nmcli) and it works fine. i've tried to revert any changes we've made to dnsmasq, dncpd, wpa_supplicant.conf, re-enable services, but no dice.

EDIT: i think i might try the wifi numb in hotspot mode on a normal rpi as a test case.

[ngwese]

possible interesting datapoint. norns wifi nub on a different ubuntu arm 16.04 LTS system in hotspot mode yielded the same PW failure as above. ....but a newer norns wifi nub doesn't seem to exhibit the same behavior.

so many gremlins

[tehn]

checking this out now, looking good!

i'm thinking i'll move the network selection screen to a scroller (like script select) rather than ENC3 selection.

we could likewise just have HOTSPOT be in this list rather than a separate HOTSPOT option? (this happened to me because i manually created a HOTSPOT network in my earlier explorations)

but an explicit hotspot option might be better UI