Pmbootstrap: [rx51] integrate ofono

[pavelmachek]

ofonod v1.6 from Debian works with Nokia N900, given right kernel. It should be possible to integrate it to pmos, too.

@craftyguy

[craftyguy]

Thanks @pavelmachek

I have created a branch with a kernel config we can play around with: feature/rx51_ofono

Notes on running ofono test scripts:

1) install ofono, python2, git, py-dbus: apk add ofono python2 git py-dbus

or

pmbootstrap install --no-fde --add ofono,git,py-dbus --sdcard=/dev/sdN

2) Grab ofono source: git clone https://git.kernel.org/pub/scm/network/ofono/ofono.git

3) scripts can be run by invoking python2: python2 ofono/test/enable-modem

[pavelmachek]

I get failure at step 2):

localhost:/my# git clone http://git.kernel.org/?p=network/ofono/ofono.git
Cloning into 'ofono'...
fatal: repository 'http://git.kernel.org/?p=network/ofono/ofono.git/' not found

date -s '2017-09-05'
git clone https://git.kernel.org/pub/scm/network/ofono/ofono.git

Does the trick.
Pavel

[craftyguy]

Yea, I assume you have the right date/time. You can also start the ntpd service: rc-service ntpd start

[PabloCastellano]

ofono/test/enable-modem has python3 shebang. Why do you need python2?

[craftyguy]

@PabloCastellano

The scripts need py-dbus, which isn't built in Alpine for python3 (only python2). The scripts don't seem to be using anything specific to python3, and any attempts I have made to build/package py-dbus for python3 have failed horribly (yes, even copying the APKBUILD for python2 py-dbus and modifying).

[pavelmachek]

Ok, if someone wants to debug it.

This is strace ofonod from debian:

open("/run/udev/data/+platform:48058000.ssi-controller",
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or
directory)
open("/sys/devices/platform/68000000.ocp/48058000.ssi-controller/ssi0/port0/ssi-protocol/net/phonet0/uevent",
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 10
open("/run/udev/data/n2", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 10
open("/sys/devices/platform/68000000.ocp/48058000.ssi-controller/ssi0/port0/ssi-protocol/net/phonet0/type",
O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 10

OFONO is only mentioned in data/n2 file:

pavel@n900:~/g/tui$ cat /run/udev/data/n2
I:5712363
E:OFONO_DRIVER=n900
E:OFONO_ISI_ADDRESS=108
pavel@n900:~/g/tui$ grep OFONO /run/udev/data/*
/run/udev/data/n2:E:OFONO_DRIVER=n900
/run/udev/data/n2:E:OFONO_ISI_ADDRESS=108
pavel@n900:~/g/tui$

And there's no /run/udev/data on pmos. I can't believe that's a
coincidence.

I tried to create /run/udev/data/n2 file on pmos, but that did not
help. (perhaps numbers need to be different?)

[craftyguy]

I pushed some updates to the ofono branch that allow building/booting the N900 kernel and sysfs nodes are populated:

localhost:/home/user# ls -l /sys/bus/hsi/devices/n900-modem/
total 0
lrwxrwxrwx    1 root     root             0 Jan  1 00:00 cmt_apeslpx -> ../../../../49052000.gpio/gpiochip2/gpio/gpio70
lrwxrwxrwx    1 root     root             0 Jan  1 00:00 cmt_bsi -> ../../../../49056000.gpio/gpiochip4/gpio/gpio157
lrwxrwxrwx    1 root     root             0 Jan  1 00:00 cmt_en -> ../../../../49052000.gpio/gpiochip2/gpio/gpio74
lrwxrwxrwx    1 root     root             0 Jan  1 00:00 cmt_rst -> ../../../../49052000.gpio/gpiochip2/gpio/gpio75
lrwxrwxrwx    1 root     root             0 Jan  1 00:00 cmt_rst_rq -> ../../../../49052000.gpio/gpiochip2/gpio/gpio73
lrwxrwxrwx    1 root     root             0 Jan  1 00:00 driver -> ../../../../../../../bus/hsi/drivers/nokia-modem
-r--r--r--    1 root     root          4096 Jan  1 00:00 modalias
lrwxrwxrwx    1 root     root             0 Jan  1 00:00 of_node -> ../../../../../../../firmware/devicetree/base/ocp@68000000/ssi-controller@48058000/ssi-port@4805a000/hsi-client
drwxr-xr-x    2 root     root             0 Jan  1 00:00 power
lrwxrwxrwx    1 root     root             0 Jan  1 00:00 subsystem -> ../../../../../../../bus/hsi
-rw-r--r--    1 root     root          4096 Jan  1 00:00 uevent

Ofono still prints a bunch of these errors on start, then does nothing:

ofonod[1593]: plugins/udevng.c:add_serial_device() Device is missing required OFONO_DRIVER property                                                                              

Regarding the /run/udev thing, that directory is totally populated on my N900 running pmos. I don't see anything relating to ofono though.

@pavelmachek Can you post your debian N900 rootfs somewhere so I can dig into it?

[craftyguy]

@pali

Sorry to bother you, but we could really use any wisdom you are willing to bestow on this situation!

Kernel config, for 4.13, can be found here. Ofono is 1.20 (through Alpine Linux repo).

Kernel cmdline does not have nokia-modem.pm=0.

[pavelmachek]

On pmos, we get bunch of "ofonod... Device is missing required
OFONO_DRIVER property" errors. We /run/udev/data contains that
property; but we do not have that file on pmos. Alternative place for
that data should be /dev/.udev/db , but we do not have that either.

https://unix.stackexchange.com/questions/206806/files-in-folder-run-udev-data

ofonod seems to rely on udev, and we do not have required database --
it does not know we have the modem.

https://github.com/systemd/systemd/issues/6713

I'm pretty sure we have udev problem here.

[craftyguy]

 /run/udev/data does exist on pmos, but none of the files in it mention ofono or the modem.

On September 6, 2017 1:48:46 AM PDT, pavelmachek [e-mail] wrote:

On pmos, we get bunch of "ofonod... Device is missing required
OFONO_DRIVER property" errors. We /run/udev/data contains that
property; but we do not have that file on pmos. Alternative place for
that data should be /dev/.udev/db , but we do not have that either.

https://unix.stackexchange.com/questions/206806/files-in-folder-run-udev-data

ofonod seems to rely on udev, and we do not have required database --
it does not know we have the modem.

https://github.com/systemd/systemd/issues/6713

I'm pretty sure we have udev problem here.

--
You are receiving this because you were assigned.
Reply to this email directly or view it on GitHub:
https://github.com/postmarketOS/pmbootstrap/issues/520#issuecomment-327418342

[pavelmachek]

I don't have /run/udev/data on pmos:

localhost:/home/user# ls /run/udev/
control rules.d
localhost:/home/user# ps aux | grep udevd
1318 root 0:00 /sbin/udevd --daemon
1541 root 0:00 grep udevd
localhost:/home/user#

[craftyguy]

Strange. It's totally there for me on pmos:

localhost:/home/user# ls /run/udev/
control  data     rules.d
localhost:/home/user# ls /run/udev/data/
c7:130  c7:131  c7:132  c7:133  c7:134  c7:2    c7:3    c7:4    c7:5    c7:6

None of those files mention the modem or ofono.

I also tried adding the ofono.rules file (from the ofono repo) to /etc/udev/rules.d, and re-triggering udev, that didn't result in any noticeable change

[pavelmachek]

I did udevadm trigger, and /run/udev/data/ appeared.

We are missing /lib/udev/rules.d/97-ofono.rules file on pmos. I added it. After a reboot, and

udevadm trigger
udevadm settle
modprobe nokia_modem
modprobe phonet
modprobe hsi
modprobe hsi_char
modprobe omap_ssi
modprobe cmt_speech
udevadm trigger

I now have n8 file:

localhost:/run/udev/data# cat n8
I:262900666
E:OFONO_DRIVER=n900
E:OFONO_ISI_ADDRESS=108
localhost:/run/udev/data#

[craftyguy]

I built the pmOS 4.13 kernel with the gpio-switch functionality from Pali's kernel, and enabled it (incl. loading modprobe nokia-modem pm=0, and without) but it didn't make any difference to the output of ofonod.

[craftyguy]

In case anyone is interested, here's the patch I used to add gpio-switch support to the kernel (based on @Pali's patch): https://gist.github.com/craftyguy/e87a1eb9f5fe4ecf1ac343ec5ae4a676

As mentioned previously, this didn't help the situation, ofono still does not see the modem.

[craftyguy]

I started doing git bisect on ofono's source, and identified that things started breaking very badly at this commit: https://git.kernel.org/pub/scm/network/ofono/ofono.git/commit/?id=3ac449e25fc8cb9880c48b8f309189c6a644f200

Using the parent commit, the modem is detected (list-modems shows it, ofono log shows nokia-gpio toggling it), but enable-modem script eventually segfaults:

      ofonod[13658]: drivers/isimodem/sim.c:isi_read_file_info() Fileid 6FB7 not implemented 
      ofonod[13658]: src/voicecall.c:ecc_g2_read_cb() 0 
      ofonod[13658]: drivers/isimodem/sim.c:isi_read_file_info() Fileid 6FB7 not implemented
      ofonod[13658]: src/voicecall.c:ecc_g3_read_cb() 0 
      ofonod[13658]: drivers/isimodem/sim.c:check_sec_response() SEC_CODE_STATE_OK_RESP
      Segmentation fault

[pavelmachek]

Could you put compiled .apk (or ofonod binary?) somewhere? Preferably
as old version as you can easily build?

[craftyguy]

Looks like the issue is that the nokia-modem driver does not create any device nodes under /dev, this causes the udevng plugin to bail when setting up the modem. Specifically, the call to udev_device_get_devnode returns NULL. Udev reports that no 'DEVNODE' property exists for this device, hence the reason this method returns NULL:

UDEV  [17082.548148] change   /devices/platform/68000000.ocp/48058000.ssi-controller/ssi0/port0/n900-modem (hsi)                                                                 
ACTION=change                                                                          
DEVPATH=/devices/platform/68000000.ocp/48058000.ssi-controller/ssi0/port0/n900-modem    
DRIVER=nokia-modem                                                                     
MODALIAS=hsi:n900-modem                                                                
OF_COMPATIBLE_0=nokia,n900-modem                                                       
OF_COMPATIBLE_N=1                          
OF_FULLNAME=/ocp@68000000/ssi-controller@48058000/ssi-port@4805a000/hsi-client         
OF_NAME=hsi-client   
SEQNUM=4798                                                                            
SUBSYSTEM=hsi                              
SYNTH_UUID=0                               
USEC_INITIALIZED=82203452  

I reached out to the ofono developers to report this:
https://lists.ofono.org/pipermail/ofono/2017-September/017449.html

I think either ofono needs to be changed to work with device drivers that don't create a device node, or the nokia-modem driver should be patched in some way to create a working device node. Both of these options are currently above my level of understanding.

[craftyguy]

@sre, maybe you can help us understand what may need to change here to get the N900's cellular modem working with the latest ofono version?

[pavelmachek]

I'm pretty sure ofonod needs to be fixed, kernel is working as
designed, even if it may be slightly unusual not to have /dev/ entry.

For testing, you may want to point ofonod at /dev/zero. ofonod should
not be able to do much damage there, and it may help you get further.

[sre]

This does look like a issue introduced by ofono. The kernel only provides a network device for the modem, so there is no DEVNODE. Nokia's N9 kernel provided /dev/cmt/, but that is not the modem controller. It's a second "device" providing the control GPIOs (and a huge hack). For reference this is how I used ofono with Debian's and mainline kernel, no gpio-switch stuff involved:

echo nokia-modem >> /etc/modules
echo "options nokia-modem pm=1" >> /etc/modprobe.d/nokia-modem.conf
ln -s /sys/bus/hsi/devices-modem /dev/cmt

Actually this is still quite the hack. IMHO all the gpios should be handled by the kernel based on the phonet0 interface status, but that does need changes in ofono. A patchset implementing the kernel part has been sent by me some time ago:

https://www.mail-archive.com/[email protected]/msg1064760.html

[craftyguy]

@pavelmachek @sre

Thank you both for the clarifications. I'm in contact with some folks in the ofono project to try to get to the bottom of this regression 😄

[craftyguy]

Well, I have hit a dead end that I'm not sure we can easily overcome: https://lists.ofono.org/pipermail/ofono/2017-September/017491.html

Looks like the isi modem driver is pretty much completely broken in ofono, in the sense that at least one critical code path was not originally tested when it was merged, and now it's segfaulting when trying to enable the modem.

[ollieparanoid]

@sre, do you have any recommendation on how to continue here? Thanks for throwing in your expertise earlier!

[sre]

First of all: I don't have the ISI specs either. I suggest to start with a git bisect to find the first broken commit and then try to fix things from there. That way it's much easier to analyze issues, since you can easily test (works / works not VS crashes / crashes a bit later). As a starting point: When I worked on the kernel PM patches I did use 5e4ce76bba92 as base (current master back then). That should work and is much newer (somwhere between 1.17 and 1.18) than the 1.6 @pavelmachek mentioned. You know 3ac449e25fc8~ has enable issues, so I suggest: git bisect 5e4ce76bba92 3ac449e25fc8~.

[craftyguy]

I actually had some success late last night, and able to get past the issue I saw and enable the modem. I need to think about how to patch the issue I saw, since I don't completely understand what is going on with in.

On September 10, 2017 7:39:21 AM PDT, Sebastian Reichel [e-mail] wrote:

First of all: I don't have the ISI specs either. I suggest to start
with a git bisect to find the first broken commit and then try to fix
things from there. That way it's much easier to analyze issues, since
you can easily test (works / works not VS crashes / crashes a bit
later). As a starting point: When I worked on the kernel PM patches I
did use 5e4ce76bba92 as base (current master back then). That should
work and is much newer (somwhere between 1.17 and 1.18) than the 1.6
@pavelmachek mentioned. You know 3ac449e25fc8~ has enable issues, so
I suggest: git bisect 5e4ce76bba92 3ac449e25fc8~.

--
You are receiving this because you were assigned.
Reply to this email directly or view it on GitHub:
https://github.com/postmarketOS/pmbootstrap/issues/520#issuecomment-328346891

[craftyguy]

Ok, these 4 patches are necessary to get you to the point where the modem is initialized, enabled, and (I think) online:

https://lists.ofono.org/pipermail/ofono/2017-September/017463.html
https://lists.ofono.org/pipermail/ofono/2017-September/017465.html
https://lists.ofono.org/pipermail/ofono/2017-September/017483.html
https://lists.ofono.org/pipermail/ofono/2017-September/017500.html

Registration and/or scanning for network operators still fails.. I'm not sure why yet.
Here are the results after executing the scan-for-operators test script: https://gist.github.com/craftyguy/0f28ab03be54697817883ee302298a13

Edit: Modem is not enabled or online, but it's closer to be than before..

[craftyguy]

Made some good progress yesterday! I was able to use 5e4ce76bba92 that sre mentioned to successfully send a SMS message from the N900. GPRS is 'up' but I cannot ping anything and dns doesn't work, but I have an IP assigned.

I'll now begin the long process of doing git bisect between this commit and the latest..

[ollieparanoid]

Woo! That is some major progress, big congratulations \o/

[pavelmachek]

Congratulations :-).

I use this code in connection setup:

def property_changed(name, value, path, interface):
...
if iface == "ConnectionContext" and name == "Settings":
interface = str(value["Interface"])
addr = str(value["Address"])
DNS = value["DomainNameServers"]
print("Connected to the internet, my IP address is "+addr+".")
subprocess.check_call(["/usr/bin/sudo", "/sbin/ifconfig", interface\
, addr, "up"])
time.sleep(1)
subprocess.call( ["/usr/bin/sudo", "/sbin/route", "del", "defa\
ult"])
subprocess.check_call(["/usr/bin/sudo", "/sbin/route", "add", "defa\
ult", "gw", addr])
print("dns says", DNS, str(DNS[0]))
#dns1, dns2 = DNS.split(" ")
open("/etc/resolv.conf", "w").write("nameserver "+str(DNS[0]))

..maybe that can serve as inspiration? :-)

[craftyguy]

I have no idea why master branch was failing earlier, but it seems to be working now with just one patch applied. The one patch is a combo of three of the previous 4 mentioned, the last one listed previously was merged this morning.

I successfully enabled gprs (using GSM) and have successfully brought up the data connection, including DNS resolution (thank @pavelmachek for the hint on setting default gw!).

This was all on the master ofono branch!

I will next work on distilling my findings into a coherent PR of some sorts to get this shit enabled on the N900. We will have to package ofono upstream from Alpine until they cut a 1.21 release with the patches necessary for enabling the N900's modem.

Edit: As of today, everything necessary to get ofono to properly set up the N900 modem for reproducing my success is now merged into upstream ofono.

[craftyguy]

Instructions for reproducing my progress thus far are on the wiki now: https://wiki.postmarketos.org/wiki/Nokia_N900_(nokia-rx51)#GPRS_.2F_SMS

I have also created a PR to set up ofono 1.20 and the N900 with everything necessary to get to this stage (#567)