Octoprint: comm port autodetection unreliable/broken

[bzed]

Hi,

printer auto connection is still broken on staging/devel. Connection times out instead of working as expected.

  File "/home/bzed/tools/OctoPrint/src/octoprint/util/comm.py", line 2693, in _openSerial
    serial_obj = factory(self, self._port, self._baudrate, settings().getFloat(["serial", "timeout", "connection"]))
  File "/home/bzed/tools/OctoPrint/src/octoprint/util/comm.py", line 2646, in default
    port = self._detect_port()
  File "/home/bzed/tools/OctoPrint/src/octoprint/util/comm.py", line 2627, in _detect_port
    programmer.connect(p)
  File "/home/bzed/tools/OctoPrint/src/octoprint/util/avr_isp/stk500v2.py", line 36, in connect
    self.sendMessage([1])
  File "/home/bzed/tools/OctoPrint/src/octoprint/util/avr_isp/stk500v2.py", line 110, in sendMessage
    return self.recvMessage()
  File "/home/bzed/tools/OctoPrint/src/octoprint/util/avr_isp/stk500v2.py", line 121, in recvMessage
    raise ispBase.IspError("Timeout")
octoprint.util.avr_isp.ispBase.IspError: 'Timeout'

I'll keep poking into it tonight, more to come (hopefully...).

[foosel]

So I've tried reproducing this and while at least against a creality board (which should be similar or even the same to what I know you have) it indeed doesn't work under py2 or 3 under 1.4.0rc3, neither does it work under 1.3.12.

So right now this doesn't look like a regression to me, and it actually autodetects another board (arduino mega) just fine.

Would you happen to remember when you last saw this working for you?

I'll fix this in any case by finally throwing out the stk500/programmer mode stuff because that was error prone AF to begin with, but since that will require some rewiring of quite sensitive parts of the comm layer that is not something I want to risk on an RC.

[foosel]

Considering this not a regression for now since I still haven't been able to make stuff that doesn't autodetect under 1.4.0 to autodetect under 1.3.x either. Going to move this to the 1.4.1 milestone and to be solved by finally getting rid of this horribly messy programmer mode bs altogether (as I said, that's been a thorn in my side for way too long anyhow)

[MatejSpindler]

I have a similar issue on 1.4.0rc5.
Versions 1.3 worked OK.

Manually selecting /dev/ttyACM0 and connecting is also working OK.

Octoprint is running on a Server/NAS type computer with a good power supply (not RPI).
My printer is a DIY Frankenstein running Bigtreetech BIQU SKR 1.3 32bit motherboard with Marlin 2.0.3 (latest tag on GitHub, at the time of writing this).

log files:
serial.log
octoprint.log

[foosel]

@MatejSpindler can you provide a serial.log from 1.3.x as well, for comparison in your specific case? I'm a bit surprised by the sheer ton of serial ports found on your system there and am wondering whether that's a difference to 1.3.x.

[MatejSpindler]

@MatejSpindler can you provide a serial.log from 1.3.x as well, for comparison in your specific case? I'm a bit surprised by the sheer ton of serial ports found on your system there and am wondering whether that's a difference to 1.3.x.

Here are requested logs. 1.3.11 connects normally.

I also updated firmware to Marlin bugfix-2.0.x (GitHub) branch just to be safe, same behavior, 1.3 connect normally, 1.4.0rc5 can't detect serial port.

octoprint.log
serial.log

Regarding a ton of ports, there really are that many tty devices :)
serial_ports.txt

[foosel]

So the difference is as suspected - 1.3.x doesn't list the ttyS* devices and hence will fall back on the dumb detection approach of "Only one serial port? Must be the printer", which it can't under 1.4.x since that added ttyS* to the serial port glob pattern and now we have a ton of additional potential devices to actually probe - and none of them, not even the printer, are detectable using the attempt to enter programming mode.

I'm pretty sure you'll see the same behaviour if you add /dev/ttyS* as additional port pattern under 1.3.x on your setup.

[MatejSpindler]

I'm pretty sure you'll see the same behaviour if you add /dev/ttyS* as additional port pattern under 1.3.x on your setup.

Correct!
Adding /dev/ttyS* to additional ports in settings on 1.3.11 has the same behavior as 1.4.0rc5 -> port detection doesn't work.

[foosel]

Not a regression then, just the actual port detection being a mess as it's always been (and the "only one serial port, let's use that" workaround for that working well in most cases).

Gotta throw that out and find a better solution for it ASAP but for now it'll be a case of having to select the port manually if detection isn't working.

[GitIssueBot]

This issue has been mentioned on OctoPrint Community Forum. There might be relevant details there:

https://community.octoprint.org/t/octoprint-no-longer-connects-if-port-set-to-auto/16795/7

[ChaosBlades]

Ever since I updated from 1.3 auto detect serial port has been broken. It worked for me on my stock Ender 3 Pro Board, Creality Silent board, and EZBoard Lite under 1.3. Now with 1.4 I get the following...

Changing monitoring state from "Offline" to "Detecting serial port"
Serial port list: [u'/dev/ttyACM0', u'/dev/ttyS0']
Trying /dev/ttyACM0
Could not connect to or enter programming mode on /dev/ttyACM0, might not be a printer or just not allow programming mode
Trying /dev/ttyS0
Could not connect to or enter programming mode on /dev/ttyS0, might not be a printer or just not allow programming mode
Changing monitoring state from "Detecting serial port" to "Error: Failed to autodetect serial port, please set it manually."
Failed to autodetect serial port, please set it manually.

I have to refresh the connection list, manually select /dev/ttyACM0 every time, and click Connect. Since the device disappears when the printer is turned off saving the connection settings is useless as it always reverts to Auto. There are no other USB devices connected to my Pi.

Also tried to define a static name to this USB port as mentioned in the below thread and several other guides as a work around until this is fixed but I can't manage to get it to work.

https://community.octoprint.org/t/cannot-connect-to-autodetected-serial-port/17252/2

If auto detecting the serial port has been broken even in 1.3 then something else is going on here.

[foosel]

If auto detecting the serial port has been broken even in 1.3 then something else is going on here.

Yes. /dev/ttyS* was added as a serial port pattern to support some setups out there. For some people that gets found on their machines as well as their printer (it doesn't on mine, I haven't yet figured out a pattern here). If more than one serial port is found, autodetection tries to figure out if the port is a printer in a way that's broken. If only one serial port is found, autodetection assumes that must be the printer.

I'm going to remove the broken autodetection-if-multiple-ports-found approach in 1.4.1 to work around this. I'll also look into providing a workaround for those affected by the /dev/ttyS* detection in the meantime since it seems to be more people than testing suggested.

Just out of curiousity - what devices do you have connected to your OctoPrint server (must not only be USB, maybe some kind of display, picam etc)? And on what kind of hardware is it running, Raspberry Pi? If so which model and what operating system version?

[ChaosBlades]

Yes, I just read through the previously linked thread. I just asked in that thread if someone would create a guide tested on OctoPi to create a "udev rule to match their printer board and manually set this alias up in OctoPrint". I believe that is what mungbean tried to do in the thread I linked but I couldn't get it to work on OctoPi. Appreciate the background on the issue. I didn't realize the pi camera uses USB.

I have:
-Ender 3 Pro w/ EZBoard Lite w/ TH3D Firmware
-Raspberry Pi 3B+ w/ Pi NoIR Camera V2 w/ Adafruit PiTFT Plus running OctoPi 0.16.0

The PiTFT was just added yesterday and running OctoScreen. That is when I really started digging into this issue because every time I turn the printer on I have to open the WebUI to manually connect because OctoScreen can't do it.

If I unplug the PiCam and turn on the printer on it automatically switches the Serial Port to the correct /dev/ttyACM0. Before it stayed on Auto. If I have 'Auto-connect on server startup' enabled when I power on the printer it will auto connect. On 1.3 it only seemed to try to connect when I first booted OctoPrint and not when Octoprint was already loaded and I turned on the printer.

[foosel]

So that sounds like the /dev/ttyS* matches something provided by the camera. That explains why I don't see it here, I use USB webcams only. Can you confirm that with unplugged camera it is only seeing /dev/ttyACM0 in the port drop down list (apart from AUTO)?

[foosel]

Can those of you running into this issue try if it gets solved by this plugin?

https://github.com/OctoPrint/OctoPrint-Remove-ttyS-Ports

It monkey patches the serial list to remove /dev/ttyS entries from the port list again and thus should work around this issue on what seems to be most of the system that so far ran into issues.

If this works I'll push it to the repository as a workaround until I have pushed a proper solution with 1.4.1 (hopefully).

[GitIssueBot]

This issue has been mentioned on OctoPrint Community Forum. There might be relevant details there:

https://community.octoprint.org/t/octoprint-no-longer-connects-if-port-set-to-auto/16795/22

[MatejSpindler]

With the plugin installed auto detection is working for me.

[ChaosBlades]

I completely re-installed everything on my OctoPi from scratch and I no longer have this issue. All my plug-ins and settings. I notice I don't have an extra ttyS port like I had before. I have no idea what that extra port was. lsusb is showing the same things it had before. Before we thought that port was the Pi Cam but that is working as it was and port is nowhere to be found.

[foosel]

@ChaosBlades what version of OctoPi did you have previously and what do you have now?

[ChaosBlades]

0.16 to 0.17. Both were apt get updated.

[taker218]

Had the same problem (Ender 3, Rpi 3b+, PiCam and USB LEDs attached).

With the plugin everything works fine like it was before.
Strangely for me this worked with 1.4.0 up to the point when I shutdown my Rpi because I had to move the printer. After that (everything was plugged in like before) I had to manually set the port to connect.

Now autodetect works fine.

[foosel]

@kantlivelong I remember you working on the reimplementation needed here. I want to look into solving this ASAP, but don't want to do the work twice - are you on the way to be able to create a PR, or want me to take over or something?

[kantlivelong]

@foosel I stepped away from it after making changes post this commit that didn't play nicely. It does mostly work but needs clean up. Also am not sure about the potential issue you brought up where the printer may send information right away. Might give it another stab this week if you're alright with it. Otherwise you're welcome to use any of that if it's helpful.

[foosel]

So that should be fixed now through a rewrite of the whole auto detection. Quoting the commit:

Now [port and baudrate detecation] no longer are separate processes but one merged one.

Port detection now will only test the two most common baudrates plus
any additionally configured ones if there is more than one port to test,
that way the additional overhead of having to cycle through both port
and baudrate should be greatly reduced.

As a consequence all avr_isp references have been finally removed.

I did a bunch of tests with a couple of serial devices and the printer was detected correctly in all cases. Still happy for additional tests of the maintenance branch! Merged anyhow, to be released with 1.4.1.

[bzed]

@foosel auto-detection works for my skr board, and printing and everything is fine as usual. Thanks!