Px4-autopilot: Check magnetic field vector norm

[Roman-]

I am experiencing magnetometer glitches described there.

П-shape glitches are followed by the message Mag#0 fail: STALE!, but "spikes" do not.

Problems are likely to be hardware and have nothing to do with PX4 Firmware. However, when these pikes take place, PX4 does not deal with them in any way and feed this obviously wrong data into EKF. This led to multiple UAV crushes.

I would assume running an extra check isn't hard to implement: if the instant change of magfield vector exceeds some threshold, assume this is a fail. This way user can at least get an information that there's something wrong with the magnetometer.

[CarlOlsson]

However, when these pikes take place, PX4 does not deal with them in any way and feed this obviously wrong data into EKF.

The EKF has internal checks for sanity of measurements so depending on your tuning it will most likely reject the mag spikes you describe

This led to multiple UAV crushes.

Can share some of the logfiles where the mag spikes have led to crashes?

[Roman-]

Can share some of the logfiles where the mag spikes have led to crashes?

https://logs.px4.io/plot_app?log=b6d7df79-7457-449f-87dd-227afc70e44c

[Roman-]

Sorry, I've forgotten to specify:
I'm using Firmware v1.10.
I have disabled the internal mag (mag#1) in config, so after external mag#0 failover there were no valid magnetometers remained. Surely adding extra checks won't fix that, but I would like to see warning messages before taking off.

[CarlOlsson]

The logging rate is very low so it is a bit hard to say what happened. But from the logfile it isn't obvious to me that the reason of the crash are the mag spikes. These should be rejected by the EKF but you the estimator still diverges right after takeoff.

[CarlOlsson]

Not only the mag norm is bad for a split second but the mag data is continuously bad for the majority of the logfile

Yeah, some detection for this bad data sounds like a good idea

[bresch]

@Roman- I added that check in November 2019 but it's unfortunately not in your version of the code.
https://github.com/PX4/ecl/blob/e4b44f704bb641fa93bedc81734c5249fcd9ed42/EKF/mag_control.cpp#L299-L335
from the PR: https://github.com/PX4/ecl/pull/662

[bresch]

Also note that in your log, the mag fusion is rejected during most of the flight

[Roman-]

Thanks!
I am a little confused, the changes you've made in November 2019 weren't included in the release of December 2019 that I use.

[bresch]

@Roman- Yes, this is because we branched out before for intensive testing. This was the last ECL update for v1.10 https://github.com/PX4/Firmware/commit/5f040fe24c1e488c6b7aadefa37c168bfa96b190