Marlin: [FR] Resurrection System/Power Panic integration

[morsyfadl]

Hello,
I noticed this has been discussed here a couple of years ago but you never went back to it. But that was before the M600 command was created (I think?)
And alot of new features have been created in Marlin but not this
So, is it possible we see this in upcoming releases? especially now with it being in MK3 and such?

Marlin could use a similar approach by using a home etched PCB like prusa's power panic circuit or like WASP's hardware option
They're both open source and easy to do. And even WASP's version of marlin is open source and imo not hard to integrate and actually can be done better by the Marlin dev's

http://www.personalfab.it/download/25251/
https://github.com/prusa3d/MK3_Power_Panic
http://www.wasproject.it/w/en/resurrection-system-a-3d-printing-revolution/

[Spawn32]

I am all for it, already have the port / hardware on-board on my STM32F7 board :)

[Spawn32]

This is the module needed, it's more or less identical to the MK3 solution:

https://www.ebay.com/itm/3D-Printer-MKS-DET-Power-Outage-Detection-Module-W2P2-I3Q9/322849565686?epid=9008006283&hash=item4b2b556bf6:g:yUMAAOSwWZpZ8rXP

The "Hot" side of it will carry 110/220v, be careful and use a shrink tube on it ;)

[Spawn32]

The ider is that there is enough juice left in the power supply cap's to save your position and / or move a stepper out of the way (hotend up) or even trigger a UPS

MKS tft 2.8/3.2 has support built in for this module, never tested it..

I have this power outage module, so if a dev wants to test something just let me know and i will test it :)

[revilor]

The power outage detector is the simple part in the equation. Prusa also has some clever tricks on the Einsy board to immediately cut power to the heated bed and hotend as soon as power outage is detected. You also need very efficient voltage regulators on the board etc. THen most of the juice left in the capacitors is for the ATmega to write its state to the EEPROM and maybe move the Z axis up a little bit. Joe Prusa can try to do this trick because he has complete control over the hardware used in his printers. Without that I suppose it won't be possible hardware-wise without some kind of UPS added to the equation.

What Marlin probably could provide is a way to trigger "Pause SD print and save current state" via an unused end stop pin for example. Or "Pause SD print and go into power savings mode (shut down display, heaters and all possible steppers)" so a UPS battery would be able to keep the printer and its internal state alive as long as possible until power is back.

[Spawn32]

I studied the new Einsy board (MK3) board when i designed my F7 board, didn't find anything special hardware wise except the port for the power outage module (also connected to the octopi port).

With the PC psu i use on my setup it looks like both my own designed board and my MKS Sbase actually survives at least 2 seconds after i turn of the psu.

That's along time in the software world and Marlin should have no problem doing what we want,
at least not on a fast 32bit 😄

That being said, i dont think Prusa has it working like the want to yet, read about some problem over at the Prusa forum...

[Grogyan]

The loss of power detection is by far the easiest with multiple solutions from as simple as a 230V relay to using an opto isolator with a PNP output. (Harder to find than NPN output variants)

[Grogyan]

Trick is to find a solution that backs up the last line to somewhere on the SD card, not in EEPROM.
And to resume printing from that line.
Keeping in mind that the line that had the power loss detected, may still be in the buffer, and therefore
must be dumped somehow.
I reckon that this should be an implemented feature soon, as this is like the 5th thread on the subject.

[morsyfadl]

I couldnt agree more; the act of detecting an outage is not an issue, its what the firmware does after that.
@Grogyan Well Prusa implemented the EEPROM method .
On the other hand WASP implemented the SD method. the firmware saves the current line to a file called "resurr.gcode" and once the printer is powered up again a prompot appears asking to resume. The firmware then loads that file and start a new print from where you left off. You can even see the lines they added to implement this features but I'm not qualified to fully understand it.

Keep in mind, BOTH approaches use the same 8-bit processor. So I'd assume its nothing too fancy, in WASP's case at least.

[Grogyan]

Looking through the code for WASP, it shouldn't be too difficult to do a pull on those changes to the current base of Marlin 1.x.
Testing of the feature can be done with using an emergency stop button.

[revilor]

You also have to consider that even if you manage to save the state of the printer: if the nozzle gets stuck where it is on power outage the print probably is ruined anyway. Therefore move up Z and retract filament should be part of the solution. But this is really hard to do just out of the power supply capacitors.

[revilor]

@Spawn32 By "clever tricks on the Einsy" I'm referring to the AND gates in the PWM lines of the heater MOSFETs for example. nAC_FAULT to LOW and the heaters are dead immediately. Otherwise all the remaining juice in the capacitors will be converted into heat before the MCU even has a chance to save the printer state.

[Grogyan]

I will emphasize caution if using those power loss detection boards from AliExpress, as they might cause electric shocks, electrocution or fire.
On the ones listed above, the opto isolator is just a plain opto coupler, and may introduce false triggers if AC is used.

From OP
Power Panic is a (relatively) safe design, when used in conjuction with a case over the IEC connector and SMPS connections, as it also integrates an Air Gap, as the dielectric of the circuit board alone might not be enough. Would be neat to see this available for purchase.

[Spawn32]

@Grogyan totally agree with you, but for some strange reason i cant find any complaints on the thing in the reviews, will fire one up tomorrow and look at what's feed into the hot side of the opto...

can look like they are rectifying and doing some stuff before it reach the opto...

[morsyfadl]

@Grogyan well prusa did post the KiCad files for it. The components are all passive and a single IC. We can send it to EasyEDA for example and order 10 PCBs +components for less than $5 -ish. I'm willing to donate that to devs if they're up to it.

[fiveangle]

Since the MK3 came out, it seems this is the hot topic of discussion.

https://github.com/MarlinFirmware/Marlin/issues/8037
https://github.com/MarlinFirmware/Marlin/issues/8042
https://github.com/MarlinFirmware/Marlin/issues/6028
https://github.com/MarlinFirmware/Marlin/issues/3429
https://github.com/MarlinFirmware/Marlin/issues/2085
https://github.com/MarlinFirmware/Marlin/issues/2659
https://github.com/MarlinFirmware/Marlin/pull/2084

[morsyfadl]

@fiveangle What is the dev's take on this so far?

[p3p]

I do agree this could be an amazing feature for areas with less stable power infrastructure but I don't quite understand why it is so requested, I would hope people in areas like that would have some form of UPS for their sensitive electronics.

That aside, with extra power fail interrupt hardware and a guaranteed amount of mAh before brownout it would be possible to continue printing with some caveats, (I know of prusas implementation and the difficulties it has)

We need the guaranteed up-time after power failure to manage a controlled stop and move the Z axis out the way, save position, gcode line, and how far through that command was executed, (posibley inferred from last known position) without battery backup this will be very variable between power supplies.

You will lose position, microsteps cannot survive stepper disable never mind power loss, and depending on your printer the axis may move when unpowered, this can be restored for X,Y by rehoming, Z cannot be homed unless it homes +ve on your machine.

Part bed adhesion, if the bed cools before restart it may become loose, parts can also warp slightly when cooling so the head may hit it even with absolute position restoration.

This feature would be unreliable at the best of times I think, with a small battery backup or super cap setup, it would possibly be usable, probably be a support nightmare.

[Grogyan]

Well, UltiMachine are just in the process of shipping out Einsy Rambo for general 3d printer use.
So I seeing that this FR is just going to get louder as the boards land in users hands.

[fiveangle]

I tend to agree with Chris that I don't see the appeal, given the effort to implement and the manifold of potential pitfalls. But then I am not constantly plagued with power delivery issues. If I were, I would take the approach of installation of a small UPS (since if I have constant power problems, I would already have a UPS anyway), run the UPS power monitoring daemon client on my Octoprint server, and when power-out was detected, script Octoprint to pause the print, save the resulting position. When power was restored, I would hand-restart the print using the last saved position (and after confirming all the other requirements were met, such as print hasn't dislodged, head did not fall, gantry had not moved, etc).

I believe that attempting to do all of this in firmware automatically would be more effort than it would be worth to me, as it could never allow 100% success due to potential dislodge when the heated bed were cooled. But again, if I had power-outages every other day, I may have a different take.

Some info on this approach (which is ready-to-go immediately): https://github.com/foosel/OctoPrint/issues/366#issuecomment-275662447

[Grogyan]

A look for a cheap UPS where I am is upwards of NZ$115, and for the average user, this is a pretty high added cost. Not many would be doing a 10+ hour long print every day, so it seems unreasonable to expect everyone to buy a UPS.

And then that depends on the length of the power outage, without saving last known location, the battery in those cheap UPS won't last as long as many might expect. Even in areas where there is a reliable power supply, outages occur, people driving into power poles, natural disasters etc.
The saving of last known location in octoprint, is just the last line of gcode. Which for a firmware implementation of such has already been proven possible, through WASP and Original Prusa I3.

[fiveangle]

Thinking more on this, I can envision a large improvement in Marlin for this scenario where it could write a marker file to the to the SD when issued M25. Then an option added to M24 so that it will execute whatever commands were necessary in the file (turn on extruder heater, heat bed, move to last known XY postion when M25 was invoked, etc) then resume printing. It would remove the need for Octoprint in situatoins where a user could manually invoke the pause/park-head command from the control panel prior to shutting down power to the printer (which would be running on UPS).

[p3p]

I tried to not sound like I was just offhandedly rejecting the feature I may have failed, I was just trying to get the point across that it will never be perfect, be extremely hardware dependant and without some form of battery/super cap supply backup unreliable.

yes prusa has the feature, and it even works some of the time (as with everything, people it doesn't work for shout the loudest so perception of usability may be skewed), they have to support 1 power supply, they can tune everything exactly to that hardware configuration, I like the idea of having a, pause, save position and shutdown command on interrupt, I'd hook it up to a low power warning on a battery backup for a "chance" at resuming.

[morsyfadl]

@fiveangle THANK you. This is actually exactly what im trying to get to but much better phrased. Just the ability to perform those actions when an external input or via even manually via LCD. Or both. The hardware should be up to the user.
I actually do have a backup UPS but it its not enough to run a 360watt printer for couple of hours. And/or sometimes the power is out while im charging it, which is kinda ironic.
Also, a manual method through LCD would be very helpful for those 40h prints when I'm trying to get some sleep.

[dosono]

Thanks for the conversation. For many places around the world like India and such power supply is always up and running. And especially for long prints it is risky also in other places. Think about 2 days prints... You dont want to lose it 30 minutes before the end.
I think we can:

1. Assume there is a UPS that signals a pause command to a predefined pins
2. Initiate M600
3. Save last Gcode line position.
4. On power on, allow resume of print as it was a normal pause/resume routine.

[siana]

@Grogyan, i honestly don't see the issue with the Makerbase MKS-DET board linked here. To me this looks like it uses a capacitive dropper first, then a full bridge rectifier, then a beefy 5V Zener to drop the voltage down to supply the optocoupler LED via a 330 Ohm resistor, so it is expressly intended to be used AC side and not somewhere else, and will work with a variety of AC voltages and should react fairly swiftly by switching the output. On a glance looks like a completely reasonable design. These kinds of things tend to fail on the Zener, here 1n5338b and it's beefy, it weighs like a gram and is designed for 5W sustained, something that will only be reached if the voltage on its input reaches 500V, and i don't see how it could ever reach more than 1/10th of that. I think they went all out on this.

[thinkyhead]

Dupe of #2085

[morsyfadl]

@thinkyhead So, what's the verdict on this? I was hoping for a dev's input

[siana]

I don't see that this needs a verdict, the issue is open on the duplicate.

[fiveangle]

@siana - I believe @morsyfadl is asking about the discussion above for an improvement to the "pause" feature, which would then allow power-down after pause, and then manual interaction to resume, which has nothing to do with "power-panic recovery".

@morsyfadl - Your request is definitely different than what this discussion was opened for. I suggest you review the discussion above, review how it would, and would-not work for your situation, think about all the pitfalls, then once you have a good handle on what you think is required, submit a new [FR] issue clearly detailing your request. Something like "_[FR] Ability to remove printer power after 'Pause' (M25) then manually 'Resume' at a later time after re-applying power_". I would pull the parts of the above discussion you think are relevant and include it your detailed description. Be sure to include expected caveats (okay that it will only work if Z axis remains stationary in power-off, okay that SD card is required, etc)

[thinkyhead]

My verdict is I welcome anyone's really good implementation of any useful feature, regardless of sex, race, or creed. I am especially excited to see this get implemented. Have at it!

[Hotendz]

@Spawn32 - Do you think this detection module could work with the Einsy Rambo? Any ideas how to wire it up on the low voltage side?

https://www.ebay.com/itm/3D-Printer-MKS-DET-Power-Outage-Detection-Module-W2P2-I3Q9/322849565686?epid=9008006283&hash=item4b2b556bf6:g:yUMAAOSwWZpZ8rXP

[Vexatuspl]

Anyone checked how it works with the MKS TFT? There are two options for it:

1. Without UPS: extruder stays where it was, after it warms up, the print continues.
2. With UPS: it only needs couple of minutes (or even less) to finish current command, move extruder away and store the progress.

I mean - it works with Marlin! So why can't it be done internally with the same approach?

[dosono]

This is not working well with MKS.
Maybe with their new version 3.
So far I could see it work

[Vexatuspl]

@thinkyhead gave a description of how should it work in the post from 22 Jan in this thread:
https://github.com/MarlinFirmware/Marlin/issues/6397#issuecomment-359412356
and in this one:
https://github.com/MarlinFirmware/Marlin/issues/2085
@Grogyan mentioned a ready function for an older version.

@dosono I know that MKS TFT is kind of 3rd party to Marlin (similar to OctoPrint) - a host and for a host, it's even harder to recognize where the printer actually is (due to buffering and so on) then for the actual controller.

I mean - it does work for Prusa i3 MK3. It also does work for example on the Anycubic I3 Mega and its UI seems to me to be the MKS TFT.