Linux: Gadget serial (ttyGS0) hangs when no host is present; prevents reboot & shutdown

Created on 29 Mar 2017 · 22Comments · Source: raspberrypi/linux

If you configure a Pi Zero or Zero W with the gadget serial configuration (dtoverlay dwc2, modules dwc2 and g_serial) and provision a getty on ttyGS0, the pi won't be able to be rebooted or shut down cleanly.

If you attach a host, this doesn't happen.

This is probably because the getty's attempt to open ttyGS0 hangs until a host shows up, and the process becomes unkillable.

Source

nsayer

Most helpful comment

@nsayer : Thank you! It didn't help though I already tested it like this.

I actually realized that systemd was stuck trying to clear the TTY, so it never even got around to starting agetty. The only way to unstuck the process was to set the TTY directives to no :

TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no

You can quickly override the existing service file like this :
$ sudo systemctl edit getty@ttyGS0
And paste :

[Service]
TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no

This can also be done manually by creating an override.conf in /etc/systemd/system/[email protected] :

$ sudo mkdir -p /etc/systemd/system/[email protected]
$ sudo vi /etc/systemd/system/[email protected]/override.conf
$ sudo systemctl daemon-reload

This looks like a bug in systemd to be honest. I can connect using the serial connection from my computer now :D

popsUlfr on 20 Apr 2017

👍3

All 22 comments

I would like to confirm the behavior on my raspberry pi zero W. Disabling the g_serial module seems to solve the shutdown/reboot hang.

Have you managed to figure out a workaround, or how to do a clean shutdown once you've started up with g_serial enabled?

ScheissSchiesser on 11 Apr 2017

Plugging in a host is the only workaround I know. You can do so while the reboot is happening and it will finish as usual.

nsayer on 11 Apr 2017

I cannot replicate this issue with the latest kernel, and Busybox getty. With

ttyGS0::respawn:/sbin/getty -n -l /bin/sh ttyGS0 9600 linux

in /etc/inittab I can happily connect to my ZeroW through wifi and reboot, although there may still be an issue with Raspbian version of getty. Using the "-w" option for getty should stop it writing to ttyGS0 before a host connects, so there should be no reason for it to hang.

ED6E0F17 on 15 Apr 2017

Raspbian Jessie doesn't have an inittab.

The step to enable the getty was

systemctl enable [email protected]

I copied /lib/sytemd/system/[email protected] to [email protected] and modified it to make it
look a little more like a cross between [email protected] and [email protected]. Included with that,
I added a -w before --no-clear and it does, indeed, seem to be behaving slightly better. The getty
process shows up in ps as

1352 ttyGS0 Ss+ 0:00 /sbin/agetty -w --noclear ttyGS0 vt102

which is definitively less zombie-like than before. I can't test plugging in a host and actually attempting to use it at the moment, however.

nsayer on 15 Apr 2017

@nsayer Hi, would it be possible to share your service file? I'm also encountering this issue and any changes always geive me the same results. It leaves an agetty process in some weird state :
root 263 0.0 0.2 9164 1116 ? Ss 20:44 0:00 (agetty)

The service is impossible to stop and the system is unable to poweroff :(

Connecting to a computer doesn't solve the issue either even when booting while it is plugged into a computer.

popsUlfr on 20 Apr 2017

Here's what I came up with, but I'm not 100% sure it's best-of-breed. :)

#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Gadget Getty on %I
Documentation=man:agetty(8) man:systemd-getty-generator(8)
Documentation=http://0pointer.de/blog/projects/serial-console.html
After=systemd-user-sessions.service plymouth-quit-wait.service
After=rc-local.service

# If additional gettys are spawned during boot then we should make
# sure that this is synchronized before getty.target, even though
# getty.target didn't actually pull it in.
Before=getty.target
IgnoreOnIsolate=yes

[Service]
ExecStart=-/sbin/agetty -w --noclear %I $TERM
Type=idle
Restart=always
UtmpIdentifier=%I
TTYPath=/dev/%I
TTYReset=yes
TTYVHangup=yes
KillMode=process
IgnoreSIGPIPE=no
SendSIGHUP=yes

[Install]
WantedBy=getty.target

nsayer on 20 Apr 2017

@nsayer : Thank you! It didn't help though I already tested it like this.

I actually realized that systemd was stuck trying to clear the TTY, so it never even got around to starting agetty. The only way to unstuck the process was to set the TTY directives to no :

TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no

You can quickly override the existing service file like this :
$ sudo systemctl edit getty@ttyGS0
And paste :

[Service]
TTYReset=no
TTYVHangup=no
TTYVTDisallocate=no

This can also be done manually by creating an override.conf in /etc/systemd/system/[email protected] :

$ sudo mkdir -p /etc/systemd/system/[email protected]
$ sudo vi /etc/systemd/system/[email protected]/override.conf
$ sudo systemctl daemon-reload

This looks like a bug in systemd to be honest. I can connect using the serial connection from my computer now :D

popsUlfr on 20 Apr 2017

👍3

Closing this issue as questions answered/issue resolved.

Please reopen if you feel more Raspian kernel work needs to be done.

JamesH65 on 19 May 2017

just to known, reboot -f -n works like power cycle

sherpya on 9 Apr 2019

This issue still seems to exist as of 2019-09-16 in Raspbian 10. The changes suggested by @popsUlfr appear to fix the issue easily.

Why not incorporate these changes into Raspbian so this functionality works out-of-the-box?

kolbe on 17 Sep 2019

@XECDesign Any thoughts?

JamesH65 on 17 Sep 2019

Are there any downsides or potential issues?

XECDesign on 17 Sep 2019

If you run the following code on a Raspberry Zero with the latest Raspbian rebooting will hang. Reboot and shutdown are no longer possible without hanging after running the following script:

if grep -q 'Zero' /proc/device-tree/model; then
  echo '>>> Configuring Serial Login for Raspberry Zero'
  echo ' dtoverlay=dwc2' >> /boot/config.txt
  echo ' modules-load=dwc2,g_serial' >> /boot/cmdline.txt
  echo 'g_serial' >> /etc/modules
  systemctl enable [email protected]
else
    echo 'This is not a Raspberry Zero, skip this step.'
fi

Please fix this issue.

I came up with the following workaround from @popsUlfr:

# enable serial login on Raspberry Pi zero
if grep -q 'Zero' /proc/device-tree/model; then
  echo '>>> Configuring Serial Login for Raspberry Zero'
  echo ' dtoverlay=dwc2' >> /boot/config.txt
  echo ' modules-load=dwc2,g_serial' >> /boot/cmdline.txt
  echo 'g_serial' >> /etc/modules
  # fix /raspberrypi/linux/issues/1929
  mkdir -p /etc/systemd/system/[email protected]
  bash -c 'echo -e "[Service]\nTTYReset=no\nTTYVHangup=no\nTTYVTDisallocate=no" > /etc/systemd/system/[email protected]/override.conf'
  systemctl enable [email protected]
else
    echo 'This is not a Raspberry Zero, skip this step.'
fi

JavanXD on 5 Mar 2020

I can reproduce this

pelwell on 6 Mar 2020

A less invasive way to test is:

Add "dtoverlay=dwc2" to config.txt.
sudo modprobe g_serial
sudo systemctl start getty@GS0
sudo systemctl stop getty@GS0 then stalls, but leaves you with a working Pi.

pelwell on 6 Mar 2020

All that is required to make it reboot is changing TTYVTDisallocate from yes to no.

pelwell on 6 Mar 2020

@XECDesign I can't see any reason why we'd want TTYVTDisallocate=yes set - it's not enabled for any similar services (console-getty, container-getty, debug-shell, serial-getty@ and the autovt@ alias), and only seems to clear the terminal at shutdown (which I wouldn't want anyway).

Is this a systemd default setting? Can we patch this file, or add a local override?

pelwell on 6 Mar 2020

TTYVTDisallocate comes from systemd, but the problem is that the [email protected] unit is meant for virtual consoles. For serial consoles the serial-getty@service should be used.

See also the comment in the unit file https://github.com/systemd/systemd/blob/master/units/getty%40.service.m4#L31-L34

# On systems without virtual consoles, don't start any getty. Note
# that serial gettys are covered by [email protected], not this
# unit.
ConditionPathExists=/dev/tty0

HiassofT on 6 Mar 2020

Thanks, @HiassofT.

@JavanXD Use systemctl enable serial-getty@ttyGS0 (or, since you are running it from an init script, systemctl start serial-getty@ttyGS0 might make more sense).

pelwell on 6 Mar 2020

Thank you very much for clarifying!

JavanXD on 6 Mar 2020

Just wanted to thank you for this. I spent literally months searching for why my Rpis were not rebooting correctly, and some random serendipity walked me to your bug one night, and I fixed this nagging blocking issue for my next firmware update.

Gui13 on 27 Apr 2020

+1 Saved me hours! Use [email protected] not [email protected]

TTYVTDisallocate comes from systemd, but the problem is that the [email protected] unit is meant for virtual consoles. For serial consoles the serial-getty@service should be used.

See also the comment in the unit file https://github.com/systemd/systemd/blob/master/units/getty%40.service.m4#L31-L34
# On systems without virtual consoles, don't start any getty. Note
# that serial gettys are covered by [email protected], not this
# unit.
ConditionPathExists=/dev/tty0