Px4-autopilot: vehicle status flags user interaction

[dagar]

A common source of confusion I see is the disconnect between the vehicle state (more or less commander's vehicle_status_flags) and what the user is able to see and do.

For example the vehicle might not have a valid global position estimate (preventing the user from entering AUTO). The user can still try to switch to a mode requiring a valid global position, but they receive a generic rejection message. In QGC users make assumptions based on the displayed GPS satellite count or gdop, but it's not what the system is directly using.

How can we improve this?

Should a subset of these flags be exposed over mavlink and displayed in some form in QGC (almost like an automated checklist)? Could we take the next step and expose to the GCS which modes are actually available based on the current vehicle state, rather than waiting for an attempted change + rejection. Likewise for arming.

Thoughts?

@LorenzMeier @DonLakeFlyer @priseborough @antiheavy

[dagar]

Main wish list

1. If I try to enter a mode or arm and it's denied, I should know exactly why. From the ground control station I need to be able to see if those conditions are now satisfied or not.
2. QGC should have sufficient information to not allow me to request the mode change or arming in the first place. Again, with specific reasons.

[LorenzMeier]

Since the manned aviation and our Firmware implementation both essentially follow checklists and abort the flight under certain fail conditions (MMEL, ETOPS), it sounds like we should expose a similar concept in the GCS.

The simple thing to do would be a very long bitfield (512 bit or so) and map everything out. I'm not sure that's the right thing. @DonLakeFlyer @tumbili?

[Antiheavy]

A lot of this will be on the UI side, but the UI obviously needs status from the vehicle. The DJI user model is pretty good here (there are other good models too). I realize this isn't the firmware side, but illustrates how firmware status information can map through to the UI.

DJI shows a big green bar at the top of the screen saying "Ready to Go (GPS)" or "Ready to Fly (GPS)".

If there is a problem the top bar is red with some sort of indicator message.



etc...
Yellow bar is used in some situations too:

Clicking on the bar takes you to the detailed setup screens where the problem lies (e.g. poor GPS or bad compass calibration). Options are not greyed out or hidden, they are always available to press, but there may be pop-up windows with detailed messages explaining why you are not allowed to do certain things depending on the situation.

QGC doesn't really have a big giant status bar that is always in your face. An approach that might work withing the existing QGC UI might be to gray out (or red text) modes and commands (e.g. Arm, Takeoff, etc) if unavailable and then if the user tries to select them there would be an appropriate warning message with clear reason given for unavailability. Mission feasibility would also factor into this and feedback would be nice on both the overall mission feasibility as well as individual invalid mission items (waypoints etc).

[Antiheavy]

Since this is the firmware repository, I'll add to my previous UI-focused post with some firmware thoughts.

A list of all the various warning messages currently in the system might be a guide to help identify what types and categories of status information could be sent.

Other thoughts:

• Mission feasibility status (overall)
• Mission feasibility status (individual mission items)
• Global Position status (and/or Local Position status) is an obvious one.
• Mode availability
• "Arm" availability
• Calibration availability
• Various types of sensor status (I think there already might be some of this in place, but might it be extended or reworked based on whatever new status message approach develops here?)
• Calibration status (again, already there to some degree, but could it be reworked?)
• Battery status (already there, can current approach be improved)
• Datalink status (anything here?)

[priseborough]

The information on which of the estimators GPS checks are failing is contained within the estimator_status.gps_check_fail_flags message. However at the moment this cannot be used directly by the UI because the message also shows the results for inactive checks. Checks are activated by the EKF2_GPS_CHECK parameter.

It makes sense to modify the reporting of estimator_status.gps_check_fail_flags so that only the bits for checks being used by the EKF are reported.

[hamishwillee]

My 2 bits FWIW

If I try to enter a mode or arm and it's denied, I should know exactly why.

Yes.

Lots of good UI suggestions there, but even if we could just provide an accurate error message on why arming is not able to happen that would sort out 80% of questions.

[dagar]

At the moment something exposed in QGroundControl that corresponds to global position valid (within commander) would probably solve a large portion of the pain users are experiencing.

In fact that would be the thing to emphasize in the status bar over things like satellite count. @dogmaphobic @DonLakeFlyer

Any ideas?

We currently have ESTIMATOR_STATUS in Mavlink. https://mavlink.io/en/messages/common.html#ESTIMATOR_STATUS

The ESTIMATOR_STATUS_FLAGS look close enough. https://mavlink.io/en/messages/common.html#ESTIMATOR_STATUS_FLAGS

[dogmaphobic]

ESTIMATOR_STATUS_FLAGS is a start. Let's sort out the messages, protocol and consequences first and I will deal with the UI once this is done. With that said, I would need some sort of a table describing to the user what each of flags mean and what they should do when the flag is set.

What this does not tell me is what level of importance or what to do/tell the user. For all flags, those in ESTIMATOR_STATUS_FLAGS and others yet to be decided, I need to know if:

• Can't do anything until fixed
  
  
  
  • What needs to be done to fix it
  
  
• Can do but watch out for this or that
  
  
  
  • List of things to watch out for
  
  
• Can do but with limited functionality
  
  
  
  • list of what is limited
  
  
• Something else I may be missing

Where would all this information be? Another XML file? How to handle custom firmwares? How to handle localization?

[dagar]

ESTIMATOR_STATUS_FLAGS don't give the whole picture as PX4 commander has a small amount of additional logic logic on top of these to decide if the estimated accuracy is acceptable for the current situation. However, using them immediately instead of the nothing we're currently doing would be a relatively easy, but very valuable usability improvement. We could use ESTIMATOR_POS_HORIZ_ABS & ESTIMATOR_POS_VERT_ABS as the high level green "ready to go", "GPS good" state that the vast majority of users actually need, and get annoyed by when they think they have good GPS, but the system can't enter a position mode.

Actual GPS readings should be deemphasized, and possibly not shown to the user at all by default. The system doesn't directly care about your GPS fix, number of satellites, etc, it cares about estimator status.

The other easy yet valuable thing to display would be these 2 flags.

By default we don't need to show anything, but if either of them are set they need to be shown immediately. Short term a critical message (QGC yellow box) would be okay and better than nothing. I was wondering if something more along the lines of a check engine light on the QGC toolbar would make sense. If you tap it to expand it could show the specific problematic thing.

[Antiheavy]

how to the internal preflight checks factor into this?
how about potentially new system status flags, e.g. if something like this issue post gets implemented? https://github.com/PX4/Firmware/issues/9030

[Antiheavy]

I was wondering if something more along the lines of a check engine light on the QGC toolbar would make sense. If you tap it to expand it could show the specific problematic thing.

Lately I've been thinking about this idea more. I like the idea of popping up a whole list of good/bad/ignore status items with details when you click on the "check engine light" (UI details TBD).

The pop-up list may contain:

• "Sensors" - aka Estimator status (question: would COM_ARM_* params be respected here?)
• "GPS" - probably worth breaking this one out explicitly
• "Home" - aka presence of a valid home position
• "Battery" - aka above threshold where it prevents arming
• "Mission" - aka mission feasibility checker status
• "Datalink" - presence, maybe someone might add RSSI or packet% thresholds someday
• "RC link" - presence, if required
• ...others?

would a new MAVlink message/structure make sense to summarize this, or is there enough info in existing messages?

[sam-weiss]

I run into this issue on a semi-regular bases while I am trying to fly surveying missions. It's very frustrating to select mission mode and not have the vehicle accept but have no feedback about why. I hope some progress on this issue can be made.

[DonLakeFlyer]

QGC has a sensor health page in the instrument panel based on the SYS_STATUS message.

[dagar]

@DonLakeFlyer that helps a bit, but it's still quite disconnected from what's happening vehicle side.

Think of it as multiple layers.

• raw sensor health
• sensor voting and timeout checks
• estimator checks and health
• commander (PX4 state machine) checking all the lower levels and making the final decision (including things like probation times)

If we wanted to finally close this nasty discrepancy we'd need to share the final high level status flags for things like global position estimate validity.

We'd get most of the total benefit by doing something like the red/yellow/green status bar status mentioned above (https://github.com/PX4/Firmware/issues/8697#issuecomment-358148569) based on home position + global position validity.

[Antiheavy]

still a highly applicable topic.

[davids5]

@dagar - This may be overkill but reminds me of human factor indications

https://inis.iaea.org/collection/NCLCollectionStore/_Public/27/002/27002051.pdf?r=1&r=1

See ALARMS AND ANNUNCIATORS section.

[Antiheavy]

still relevant.

[svpcom]

still relevant

[LorenzMeier]

The MVP for this is this PR which gets enough information across to at least prevent surprises when arming: https://github.com/PX4/Firmware/pull/14322

I will work with Don / Gus on the QGC side.