Prusa-firmware: Feature Request: Power panic recovery when printing via USB (Octoprint/RepetierServer)

[mrcg]

The Prusa MK3 is really awesome machine with very features. This is what your customers like.
Lot of makers are printing via USB using Octoprint or Repetier server.
Unfortunately the power panic feature is only working when printing from SD.

One workaround would be to use the Octoprint feature "print from SD".
Repetier Server does not support this feature and some of the benefit using external printing server are lost. Beside this the gcode needs to be saved on the local SD and cannot be transferred via network. (The print to SD is really no useful option as this takes ages!).

Because of this i would like to ask for power panic recovery support when printing via USB.
Solution could be:
1.) Prusa MK3 FW to store the line number which is sent from printing server for every command. The line number should be displayed on the LCD after power recovery. User to start print from line number xxx manually.
2.) If line number is not provided from printing server than Prusa MK3 should count every line of code received after specific code command (indicating print started). This line number should be displayed on the LCD after power recovery including the last gcode line executed. With this information user can identify the line in the gcode for manual recovery (exclude all blank lines and comment lines).

WDYT?
Any other better idea to support power panic feature when printing via USB?

[tbedore]

Can this actually become something? Dealing with power outages is just an unexpected circumstance and I primary run my printer through the Raspberry Pi. 90% of the power outages happen and I don't know they are and then I lose my prints.

[rhys100]

just a followup, will this ever be possible?

[cmcoffman]

I've been trying to figure out what's possible with the gpio exposed on the raspberry pi port. I don't have one plugged into my.mk3 at the moment but as far as I can tell it's the power, ground, serial Rx/Tx, and two GPIO pins, which I'm not sure are currently being used.

If the printer could tell the Pi that power panic has started it could try and remember where it is and resume. Alternatively, the pi could listen for a message that power panic happened at X line, the pi could recover as well.

I think the real problem though is a hardware one. The pi is currently connected to the Einsy board for power, and the pi uses a lot of power in this context. Since power panic depends on residual current from the capacitors in the power supply, a connected pi is likely to drain this too rapidly for the Einsy board to do the power panic.

Also, while hard shutdowns are often not a problem on a raspberry pi, it's possible that in this scenario you'd really be rolling the dice on getting your SD card corrupted.

On this architecture the perfect solution would probably involve a little software and a small battery backup for the pi.

So it's a tricky problem, and I suspect the next gen board will be 32-bit with built-in wifi anyways.

It would actually be very informative if someone had a pi zero on their mk3 and looked at how quickly the machine loses power when you unplug it.

[maximlevitsky]

Here is the quick and dirty summary (pending upload to some git repo, when I have time for that) that summarizes the pins and connectors of the einsy.

The RPi connector has the power panic input directly exposed (that is it will toggle without main MCU help), and thus indeed the question is if the RPi has enough power to save the point where it was to recover.

connectors.txt
pins.txt

[mrcg]

Just as an idea:
MK3S could count the amount of commands received from the code (only movement commands, except temp polling of course). In case of power panic the MK3S could store this number to memory like it's done when printing from SD. After power is back MK3S should ask if print from USB should be resumed. If user confirms than he get's instructed to start the print again. MK3S should than simply ignore all the number of lines quickly until stored number of code is reached. I think this procedure is quite similar to the "print to SD" card feature.

[3d-gussner]

Like the idea and here my thoughts:

• As the Einsy board has on Port J19 /RPi port at pin 3 the nAC_FAULT signal the Octoprint board could get notified if a power panic has been triggered
• The Octoprint board needs itself a power outage protection like a battery backup or UPS as you don't want to crash your Linux system and Octoprint
• I am not sure how Octoprint tracks if lines have been printed or not and if the Prusa firmware has a cache or not. But if i am not wrong the Prusa firmware acknowledge the printed line back to Octoprint.

Assuming the Octoprint instance stays alive (battery backup/UPS) during power outage on the Einsy board

• Prusa firmware stores xyz last positions to EEPROM and moves the hotend aside
• An Octoprint Plugin could be triggered by the nAC_FAULT signal
• Octoprint should know which lines have been acknowledged and store that information into the Plugin
• The Octoprint plugin could set the interrupted job into "to be recovered" mode

During power outage:

• The Octoprint plugin could check if the Einsy board is back online

After power is reestablished:

• The Octoprint plugin knows that the Einsy board is powered
• It could trigger Octoprint to reconnect to the printer
• It could request the stored xyz positions from the printer
• then compare the results with the "to be recovered" gcode
• Depending on settings
  
  
  
  • If bed is still hot enough
  
  
  • Octoprint Plugin rehomes only X and Y
  
  
  • and reheats Bed and Hotend to desired temps
  
  
  • continues the print at last know xyz coorninates
  
  
  • If bed temp too low
    
    same procedure as the SD card restore print.
  
  

So at the end the only Prusa firmware part would be:

• Store xyz position to EEPROM also for USB prints
• Move the hotend away from print
• Make it possible to read these values via gcode

The rest would be an Octoprint Plugin issue and connecting the nAC_FAULT to the board.

Here some Power Panic related EEPROM settings in the firmware

#define EEPROM_UVLO                     (EEPROM_CALIBRATION_STATUS_PINDA - 1) //1 - uvlo during print
#define EEPROM_UVLO_CURRENT_POSITION    (EEPROM_UVLO-2*4) // 2 x float for current_position in X and Y axes
#define EEPROM_FILENAME (EEPROM_UVLO_CURRENT_POSITION - 8) //8chars to store filename without extension
#define EEPROM_FILE_POSITION (EEPROM_FILENAME - 4) //32 bit for uint32_t file position 
#define EEPROM_UVLO_CURRENT_POSITION_Z  (EEPROM_FILE_POSITION - 4) //float for current position in Z
#define EEPROM_UVLO_TARGET_HOTEND       (EEPROM_UVLO_CURRENT_POSITION_Z - 1)
#define EEPROM_UVLO_TARGET_BED          (EEPROM_UVLO_TARGET_HOTEND - 1)
#define EEPROM_UVLO_FEEDRATE            (EEPROM_UVLO_TARGET_BED - 2)
#define EEPROM_UVLO_FAN_SPEED           (EEPROM_UVLO_FEEDRATE - 1) 
#define EEPROM_FAN_CHECK_ENABLED        (EEPROM_UVLO_FAN_SPEED - 1)
#define EEPROM_UVLO_MESH_BED_LEVELING     (EEPROM_FAN_CHECK_ENABLED - 9*2)

#define EEPROM_UVLO_Z_MICROSTEPS     (EEPROM_UVLO_MESH_BED_LEVELING - 2)
#define EEPROM_UVLO_E_ABS            (EEPROM_UVLO_Z_MICROSTEPS - 1)
#define EEPROM_UVLO_CURRENT_POSITION_E  (EEPROM_UVLO_E_ABS - 4)                 //float for current position in E