Klipper: INIT_TMC does not fully reset TMC2209 but FIRMWARE_RESET does

[AshIzat]

I'm not sure if this is an issue with klipper or TMC2209s.
When motor power is lost the stepper drivers stop functioning correctly until FIRMWARE_RESET is issued. INIT_TMC does not fix the issue, which presents as the motors running at what appears to be half speed, I believe this is caused by the motors defaulting to 32x microtepping my setup is for 16x.

output from DUMP_TMC when motors working correctly:

Send: dump_tmc stepper=stepper_y
Recv: // ========== Write-only registers ==========
Recv: // IHOLD_IRUN: 0008150c IHOLD=12 IRUN=21 IHOLDDELAY=8
Recv: // TPWMTHRS:   00000026 TPWMTHRS=38
Recv: // TPOWERDOWN: 00000014 TPOWERDOWN=20
Recv: // SGTHRS:     00000000
Recv: // ========== Queried registers ==========
Recv: // GCONF:      000001c0 pdn_disable=1 mstep_reg_select=1 multistep_filt=1
Recv: // GSTAT:      00000001 reset=1
Recv: // IFCNT:      00000024 IFCNT=36
Recv: // OTP_READ:   0000000d OTP_FCLKTRIM=13
Recv: // IOIN:       21000045 ENN=1 MS1=1 PDN_UART=1 VERSION=0x21
Recv: // FACTORY_CONF: 0000000d FCLKTRIM=13
Recv: // TSTEP:      000fffff TSTEP=1048575
Recv: // MSCNT:      00000008 MSCNT=8
Recv: // MSCURACT:   00f7000c CUR_A=12 CUR_B=247
Recv: // CHOPCONF:   14030053 toff=3 hstrt=5 TBL=2 vsense=1 MRES=4(16usteps) intpol=1
Recv: // DRV_STATUS: c00c0000 CS_ACTUAL=12 stealth=1 stst=1
Recv: // PWMCONF:    c80d0e24 PWM_OFS=36 PWM_GRAD=14 pwm_freq=1 pwm_autoscale=1 pwm_autograd=1 PWM_REG=8 PWM_LIM=12
Recv: // PWM_SCALE:  0000000e PWM_SCALE_SUM=14
Recv: // PWM_AUTO:   000e0024 PWM_OFS_AUTO=36 PWM_GRAD_AUTO=14
Recv: // SG_RESULT:  00000000

output after power lost (and restored) but before INIT_TMC:

Send: dump_tmc stepper=stepper_y
Recv: // ========== Write-only registers ==========
Recv: // IHOLD_IRUN: 0008150c IHOLD=12 IRUN=21 IHOLDDELAY=8
Recv: // TPWMTHRS:   00000026 TPWMTHRS=38
Recv: // TPOWERDOWN: 00000014 TPOWERDOWN=20
Recv: // SGTHRS:     00000000
Recv: // ========== Queried registers ==========
Recv: // GCONF:      00000101 I_scale_analog=1(ExtVREF) multistep_filt=1
Recv: // GSTAT:      00000001 reset=1
Recv: // IFCNT:      00000000
Recv: // OTP_READ:   0000000d OTP_FCLKTRIM=13
Recv: // IOIN:       21000045 ENN=1 MS1=1 PDN_UART=1 VERSION=0x21
Recv: // FACTORY_CONF: 0000000d FCLKTRIM=13
Recv: // TSTEP:      000fffff TSTEP=1048575
Recv: // MSCNT:      00000004 MSCNT=4
Recv: // MSCURACT:   00f70006 CUR_A=6 CUR_B=247
Recv: // CHOPCONF:   13010053 toff=3 hstrt=5 TBL=2 MRES=3(32usteps) intpol=1
Recv: // DRV_STATUS: c01f0000 CS_ACTUAL=31 stealth=1 stst=1
Recv: // PWMCONF:    c80d0e24 PWM_OFS=36 PWM_GRAD=14 pwm_freq=1 pwm_autoscale=1 pwm_autograd=1 PWM_REG=8 PWM_LIM=12
Recv: // PWM_SCALE:  00000021 PWM_SCALE_SUM=33
Recv: // PWM_AUTO:   000e0024 PWM_OFS_AUTO=36 PWM_GRAD_AUTO=14
Recv: // SG_RESULT:  00000000

once INIT_TMC is run the output is identical to the first:

Send: dump_tmc stepper=stepper_y
Recv: // ========== Write-only registers ==========
Recv: // IHOLD_IRUN: 0008150c IHOLD=12 IRUN=21 IHOLDDELAY=8
Recv: // TPWMTHRS:   00000026 TPWMTHRS=38
Recv: // TPOWERDOWN: 00000014 TPOWERDOWN=20
Recv: // SGTHRS:     00000000
Recv: // ========== Queried registers ==========
Recv: // GCONF:      000001c0 pdn_disable=1 mstep_reg_select=1 multistep_filt=1
Recv: // GSTAT:      00000001 reset=1
Recv: // IFCNT:      00000024 IFCNT=36
Recv: // OTP_READ:   0000000d OTP_FCLKTRIM=13
Recv: // IOIN:       21000045 ENN=1 MS1=1 PDN_UART=1 VERSION=0x21
Recv: // FACTORY_CONF: 0000000d FCLKTRIM=13
Recv: // TSTEP:      000fffff TSTEP=1048575
Recv: // MSCNT:      00000008 MSCNT=8
Recv: // MSCURACT:   00f7000c CUR_A=12 CUR_B=247
Recv: // CHOPCONF:   14030053 toff=3 hstrt=5 TBL=2 vsense=1 MRES=4(16usteps) intpol=1
Recv: // DRV_STATUS: c00c0000 CS_ACTUAL=12 stealth=1 stst=1
Recv: // PWMCONF:    c80d0e24 PWM_OFS=36 PWM_GRAD=14 pwm_freq=1 pwm_autoscale=1 pwm_autograd=1 PWM_REG=8 PWM_LIM=12
Recv: // PWM_SCALE:  0000000e PWM_SCALE_SUM=14
Recv: // PWM_AUTO:   000e0024 PWM_OFS_AUTO=36 PWM_GRAD_AUTO=14
Recv: // SG_RESULT:  00000000

However the driver appears to still be using 32x microstepping. With a FIRMWARE_RESET everything then works as normal

klippy.log

[KevinOConnor]

Well, that's certainly odd. A FIRMWARE_RESTART just internally calls INIT_TMC, so I don't know why there would be any differences.

If the driver is reporting correct values, but using different values then it sounds like something odd with the tmc2209 hardware. In general, the recommendation is to not remove motor power from the tmc drivers though.

-Kevin

[AshIzat]

I can't explain it either, I'll have a dig through the documentation for the tmc2209 and see if I can find anything.

The advice from TMC to not remove power is unfortunate. I like having the ability to turn main power on/off using a relay controlled by the microcontroller, it also acts as an additional layer of protection should a heater MOSFET fail shorted.

Also unfortunate is the design of the skr mini e3, which powers the mcu via usb, meaning you need to unplug the usb cable every time you want to turn the machine off if you don't want to run into this issue of powered mcu but unpowered TMC drivers.

For now, a firmware_reset after power is turned on is a viable workaround. But I'll see if I can come up with anything better.

[Hywelmartin]

@Ashlzat Watterot:s stepstick protector v2 would probably fix the power issue(power on sequence)

[jakep82]

@Ashlzat Watterot:s stepstick protector v2 would probably fix the power issue(power on sequence)

The SKR mini e3 is not a stepstick board. The drivers are soldered to the board itself.

[Hywelmartin]

@jakep82 du didn't realize that this thread was E3 specific..

[AshIzat]

@Hywelmartin It may or may not be, I just mentioned that's the board I'm using, my current plan is to use a separate MCU (just a sparkfun pro micro) to control the PSU and only power the SKR mini E3 from the PSU.

In regards to the issue itself, the only difference I can think of between a firmware reset and INIT_TMC is that a firmware reset could be resetting communications with the tmc2209s,
Page 15 of the datasheet says :

The communication becomes reset if a pause time of longer than 63 bit times between the start bits
of two successive bytes occurs

If i'm understanding that correctly then when the board is reset, it would also reset the comms of the tmc2209s, which looking at the code, INIT_TMC doesn't do. In theory we could do the same by pulling the uart line low for 63 bit times, but I think that's something that would require extra mcu code, also in the case of multiple 2209s on one uart, it would reset all of their comms, but I don't think that should be a problem as I don't believe it's a full reset, just the comms.

[KevinOConnor]

Any further updates on this?

-Kevin

[KevinOConnor]

I'm going to close this issue as it looks like the conversation has concluded. If anyone does find out the root cause for this odd behaviour then do let us know.

Thanks,
-Kevin

[lenne0815]

Now like usually i cant add anything really substatial here unfortunately but i ran into the same issue while tuning my TMC5160s. It lead to hours and hours wasted in utter confusion while tuning them. The only ! reliable way i found to completely reset them is cutting power to both mcu and raspi. Somewhere between mcu / klipper core reboots the tmcs become unresponsive, sometimes as far as not reporting anything from a spi dump.

For example i had heavy mid band resonances for about 50 normal reboots ( nearly stalling the steppers ) and then it suddently disappeared completely which i can only attribute to a default setting finally being correctly written.

My best guess is that theres a bug in the spi implementation somewhere which only shows its ugly head after some firmware resets or similar.
Maybe for some reason the spi frequency isnt being adhered to ? I dont have a scope unfortunately either so i cant check that.

[jalanjarosz]

@lenne0815 I don’t know your hardware configuration, but Watterott has documented some interesting things with TMC drivers, link below.

Things to consider:

• Turn on the high voltage (VM) first before turning on MCU which applies (VIO) logic voltage. They documented a power on and power off sequence to use.
• There are some subtle differences when using 3v3 logic over 5v logic.
• they suggest adding a Schottky diode or the v2 silent step-stick protectors under specific logic voltage conditions.

https://learn.watterott.com/silentstepstick/faq/

[lenne0815]

@lenne0815 I don’t know your hardware configuration, but Watterott has documented some interesting things with TMC drivers, link below.

Actually this could very well be the case, my sticks are the more sensible 3,3 - 5v version and i didnt control how they see ( and especially not how they are switched off ) Vmot / IO.

They do work perfectly now so i might not see immidiate benefits from it but nonetheless i bought the protectors, ill report back if i see the mentioned irregularities still.

[lenne0815]

So, i installed stepstick protectors ( even shelled out for the watterot ones ) But nothing has changed, the sticks still behave erratically, right now heavy mid band resonances are back which dissappear with a firmware / host reset.

[lenne0815]

So i did some more testing and the only reliable sequence i found to get them fully reset and working is.

• make sure the tmc power on sequence is obeyed at all times
• disconnect mcu via octoprint
• hard reset mcu / controller board
• connect via octoprint
• issue FIRMWARE_RESTART
• issue RESTART
• G28 ( Now i can tell if it worked by listening for the resonances, if it didnt stick the first time: )
• issue FIRMWARE_RESTART
• issue RESTART
• G28
• Ready

I took over G28 which does tmc_init now for all steppers before homing, so i believe that plays no role whatsoever:

[homing_override]
axes: x,y,z
set_position_x:450
set_position_y:450
set_position_z:450
gcode:
INIT_TMC STEPPER=stepper_y
INIT_TMC STEPPER=stepper_x
INIT_TMC STEPPER=stepper_x1
etc...
etc...

Sometimes Octoprint gets stuck in between aswell ( not connecting to mcu anymore ) so i have to reboot it aswell ( soft reboot )

Unfortunately i have 0 clues why it behaves like that.

I would expect a complete spi reinitialisation on both Firmware and host reset ( I think the Firmware reset already resets spi, but doesnt write the config values, so one always needs a host restart afterwards aswell )
And i believe something fishy starts to happen once spi is activated in general, before it never happend that the klipper octoprint host is unable to connect to a freshly hard resetted mcu.

And important ! Im on 5160s which exibit the resonance problem, but i firmly believe its the same for the 2209s, its just exposing diferent faults.

I added my log which should contain the aforementioned problems ( I just ran such a sequence ) if they are logged.

klippy.zip

[KevinOConnor]

Be aware that some of the 5160 step sticks end up bridging the spi and "external clock" lines - which can cause the chips to do odd things.

You may also wish to run DUMP_TMC on the steppers before and after each stage to see if DUMP_TMC reports any differences.

-Kevin

[lenne0815]

I made absolutely sure beforehand that this plays no role -

Im using Bigtreetech 1.2 tmc5160 which are supposed to remedy this problem but i didnt trust them so i clipped off the pin to make sure my rumba board doesnt interfere somehow ( that was unecessary as that pin is isolated, still better to be 100% sure )

And furthermore i soldered all clk pins directly to gnd which once again made no difference.

Dump TMC does indeed report different values each time, only sightly different most of the time but still different. I think the major settings leading to my heavy mid band resonances arent reported back by dump tmc, probably some obscure chopper settings that get corrupted / not initialized.

Some kind of spi sanity check would be great, something that compares various dump parameters vs actual config set values. Or even some snippet that test writes something via spi and reads it back out ?

[KevinOConnor]

Yeah - a number of the registers on the trinamic drivers are write-only. If they get corrupted, it's not easy to find out. That said, if you post the info that is available, someone may spot something interesting in the data.

-Kevin

[lenne0815]

So this is after a fresh cold boot with tmc protector boards and multiple init_tmcs ( as ive got one in my homing code ) in place, heavy mid band resonances:

Send: dump_tmc stepper=stepper_x
Recv: // ========== Write-only registers ==========
Recv: // COOLCONF: 00040000 sgt=4
Recv: // IHOLD_IRUN: 00061207 IHOLD=7 IRUN=18 IHOLDDELAY=6
Recv: // TPOWERDOWN: 00000080 TPOWERDOWN=128
Recv: // ========== Queried registers ==========
Recv: // GCONF: 00000004 en_pwm_mode=1
Recv: // CHOPCONF: 14010014 toff=4 hstrt=1 tbl=2 MRES=4(16usteps) intpol=1
Recv: // GSTAT: 00000005 reset=1(reset) uv_cp=1(Undervoltage!)
Recv: // DRV_STATUS: 80074000 stealth=1 CSACTUAL=7 stst=1
Recv: // FACTORY_CONF: 0000000d FACTORY_CONF=13
Recv: // IOIN: 300000d0 DRV_ENN=1 SD_MODE=1 SWCOMP_IN=1 VERSION=0x30
Recv: // LOST_STEPS: 00000000
Recv: // MSCNT: 00000008 MSCNT=8
Recv: // MSCURACT: 00f7000c CUR_A=12 CUR_B=247
Recv: // OTP_READ: 0000000d OTP_FCLKTRIM=13
Recv: // PWMCONF: 00000000
Recv: // PWM_SCALE: 00000007 PWM_SCALE_SUM=7
Recv: // PWM_AUTO: 0000001d PWM_OFS_AUTO=29
Recv: // TSTEP: 00000000
Recv: ok

This is after a firmware and host reboot cycle:

Send: dump_tmc stepper=stepper_x
Recv: // ========== Write-only registers ==========
Recv: // COOLCONF: 00040000 sgt=4
Recv: // IHOLD_IRUN: 00061207 IHOLD=7 IRUN=18 IHOLDDELAY=6
Recv: // TPOWERDOWN: 00000080 TPOWERDOWN=128
Recv: // ========== Queried registers ==========
Recv: // GCONF: 00000004 en_pwm_mode=1
Recv: // CHOPCONF: 14010014 toff=4 hstrt=1 tbl=2 MRES=4(16usteps) intpol=1
Recv: // GSTAT: 00000005 reset=1(reset) uv_cp=1(Undervoltage!)
Recv: // DRV_STATUS: 80074000 stealth=1 CSACTUAL=7 stst=1
Recv: // FACTORY_CONF: 0000000d FACTORY_CONF=13
Recv: // IOIN: 300000c0 SD_MODE=1 SWCOMP_IN=1 VERSION=0x30
Recv: // LOST_STEPS: 00000000
Recv: // MSCNT: 000002e8 MSCNT=744
Recv: // MSCURACT: 01dc010b CUR_A=-245 CUR_B=-36
Recv: // OTP_READ: 0000000d OTP_FCLKTRIM=13
Recv: // PWMCONF: 00000000
Recv: // PWM_SCALE: 00000014 PWM_SCALE_SUM=20
Recv: // PWM_AUTO: 00180053 PWM_OFS_AUTO=83 PWM_GRAD_AUTO=24
Recv: // TSTEP: 00000000
Recv: ok

[KevinOConnor]

That's odd - it shows stealthchop enabled (en_pwm_mode=1), but your previous config showed it as disabled..

-Kevin

[lenne0815]

Jeah, thats just one of many oddities - in fact though stealthchop is disabled, i proud myself to be able to tell by now ;) So im unsure whats going on there aswell.

[KevinOConnor]

I would try to figure out why that is occurring. It should not be reporting that.

-Kevin

[lenne0815]

Hey Kevin, i finally had time to look into it again, tried this and that - nothing.

So i went back to the drawing board and reflashed the mcu. Lo and behold the drivers reported the correct pwm state at first boot and dont exibit any of the issues mentioned before.

So my conclusion is:

• Im stupid
• Im stupid

Thank you for narrowing it down to investigating the wrong chopper mode, this made testing a lot easier.