In response to medic/medic-projects#4329
Full details and additional screenshots can be found in the design spec: https://docs.google.com/document/d/1we_Qu1B4x8PuAi9pxiIu4MeXtdH1kY9W29kMidHThsc/edit?usp=sharing
Summary of muting in general

Muting households

UI changes to main People page list

UI changes to profiles
Given that some of the above will be covered by configuration (eg the mute form, not generating Tasks for muted contacts), I'll try to summarize the changes/features needed, along with some questions before we implement. Once outstanding questions are answered we spawn new issues for individual parts of this issues, or just leave them all part of this one.
An field in app_settings that can contain an array of form IDs that mute a contact, and another field for unmuting contacts. We already have a similar fields for muting SMS notifications:
"notifications": {
"off_form": "OFF",
"on_form": "ON",
It makes sense to combine this new muting with the existing muting of SMS notifications since the reasons for muting via SMS should also mean there shouldn't be Tasks in the app for that contact. We should first verify though that this is compatible with how projects use the existing SMS version. If that is the case, this field would need to change from accepting only a single form to _also_ accepting an array of form fields.
If a submitted report's form ID is on the mute list then the contact the report is about should get a muted flag (eg muted: true). Conversely when a report on the unmute list is submitted, the contact has the muted flag unset (eg muted: false, or remove the field). _Note that we've often entertained a similar feature, that of a submitted report updating fields on the contact. Eg setting/updating a person's birthday via a SMS form. It would be a bonus if the way we implement this issue is generic enough or extendable to satisfy other doc updates._
As mentioned in the issue, we cannot have an unmuted person/place in a muted place. We can avoid this situation in the webapp by setting form context accordingly. By SMS however, we'd need to be able to validate the unmute form by checking that no parent of the contact is muted. We likely either need to allow validation on notification reports, and ensure that fields for all levels of the lineage can be accessed, or perhaps just an event_type like patient_not_found eg muted_in_lineage.
Amanda has clearly laid out what a muted contact looks like in the list. Curious if this is necessary for the first version of this feature, or if we are better off with a simpler implementation (eg just graying out the contact in the contact list) until we get real project use and feedback of the feature.
Done via configuration by checking for any muted parent in lineage.
Done via pre-exisiting feature to turn notifications off, if we can combine with that feature.
When opening an action form for a muted contact, if the form is not an unmute form then show a modal before continuing eg “This person belongs to a family that is currently muted. Are you sure you want to proceed? Yes/No".
Are muted contacts excluded from Targets? (This is dealt with in configuration, but good to know here)
In regards to Targets, I think we will want the muted contacts included in some but not included in others. Perhaps @michaelkohn could provide some suggestions here.
Agreed... I think this'll need to be decided on a target-by-target basis. For simple form counts, I think you will want to include them because the Mute feature should prevent submission of forms while the contact is muted anyway... and if they weren't muted at the time of form submission, they should be counted. However, I can definitely imagine some other scenarios where you wouldn't want to include them... perhaps in some denominators for expected home visits, for example.
I don't know exactly how Targets are configured, but I imagine some of the processing could also be pretty heavy if you have to look for muting. For example.... if you needed to calculate something like % visits within 24h and wanted to excluded certain muted contacts, you'll probably need to JOIN the relevant forms not only with the contact but with the contact's parent to see if the family has been muted. Just something to consider.
Ok, thanks for the info. Since the muting will be on the place it will be easy to include or exclude it on a per widget basis. It means that we still need to process all contacts, and shouldn't exclude them from Nools just because they are muted. We'll be able to handle the muted contacts in config of targets pretty easily.
Since the muting will be on the place
I assume you mean 'in addition to' the contact, not just the place
Also... sorry I just noticed this comment
If a user begins any new form for a muted person/place that is not the "Unmute" form, they will first get a warning modal saying “This person belongs to a family that is currently muted. Are you sure you want to proceed?”
Which means my comment
...because the Mute feature should prevent submission of forms while the contact is muted anyway...
Is not necessarily true. Totally fine, just means we'll need to be more diligent in configuring targets and determining when to include/exclude things.
Since the muting will be on the place
I assume you mean 'in addition to' the contact, not just the place
I wouldn't assume that, but gather that is what you want?
When muting a place (aka family) adding the muted flag to every person within place seems problematic as it would mean changes to more docs. I was thinking a person would be muted if they themselves, or any of their parents have the muted flag.
This allows us to:
The muted flag would be available in the person' lineage docs.
Since the muting will be on the place
I assume you mean 'in addition to' the contact, not just the place
I wouldn't assume that, but gather that is what you want?
Sorry... I'm OK with family muting only updating the clinic and not cascading those updates to the person docs. I just meant that you'd have to look at the clinic _and_ the person to know whether or not the person was muted, and therefore whether or not to include a given person specific form in a count.
When you say _any of their parents_, I take that to mean this includes muting health_center, district_hospital, etc... I was thinking we were only addressing muting person and clinics. It would be awesome if we were allowing muting of health_centers.
If a submitted report's form ID is on the mute list then the contact the report is about should get a muted flag (eg muted: true). Conversely when a report on the unmute list is submitted, the contact has the muted flag unset (eg muted: false, or remove the field).
muted field on the "Place" (clinic) as well, since you can mute at that level. muted: false rather than removing the field.muted field (defaulted to false) on all _new_ places / contacts.muted field to every existing place/contact when they upgrade to the version that includes this feature.Some additional questions/comments:
P form (for example) for a muted contact, I assume we won't have the "are you sure" workflow. Will we simply accept the P and create the schedule?data_record.Collection of responses below:
you'd have to look at the clinic and the person to know whether or not the person was muted
Correct
would be awesome if we were allowing muting of health_centers.
I don't see a good reason why we wouldn't make it generic enough to mute any place level.
add the muted field to every existing place/contact when they upgrade to the version that includes this feature.
We are unlikely to add a migration for this due to the performance costs (eg rebuilding views, re-downloading all contacts). Given that the absence of a muted flag means that the place is not muted, what is the advantage of adding muted: false to places by default?
Answers to the additional listed questions:
- For SMS projects, if you submit a P form (for example) for a muted contact, I assume we won't have the "are you sure" workflow. Will we simply accept the P and create the schedule?
Yes, I believe that is the best way to proceed.
- Will we differentiate between ON/OFF and muting when it comes to message states?
The reports that turn notifications OFF set the state of scheduled messages to muted. I don't see a reason to have different states... did you have something in mind?
- If a schedule-creating form is submitted (and accepted) for a muted contact, I assume we will still create the schedule but automatically place it in the new mute status?
That is a good point, and I would not have assumed that as it is not how it currently works -- but it is worth specifying as such. In config we should make sure that the response mentions that the patient is muted if that is the case.
- If possible/easy, I can imagine it might help some debugging if we stamp the contact/place's current mute status on the data_record.
Would this be only at doc creation, eg contact_mute_state_at_creation_time, or updated whenever the contact's mute state changes, eg contact_mute_state?
would be awesome if we were allowing muting of health_centers.
I don't see a good reason why we wouldn't make it generic enough to mute any place level.
Actually... I guess it's less about muting the _place_, but rather muting the _contact_ of that place. What I'm specifically looking for is the ability to know which CHW's are still with the program and therefore should show up in dropdowns in analytics, for example. This was discussed in https://github.com/medic/medic-projects/issues/1894 . This is different than muting the place (health_center), which would have the net effect of muting all schedules beneath it. That isn't what I'm looking for, I just want the ability to mute a specific CHW.
We are unlikely to add a migration for this due to the performance costs (eg rebuilding views, re-downloading all contacts). Given that the absence of a muted flag means that the place is not muted, what is the advantage of adding
muted: falseto places by default?
The data model (and all our queries) will be much cleaner, not to mention easier to understand for the community, if we can simply check the value of a field that is known to exist rather than having to always check to see if fields exist first, and then check the value of them. As far as I am aware, we don't have any other situations where the _existence_ of a field is conditional (other than fields being added later, but from that point on they always exist).
Based on the first part of your response, I recognize we'll have to first check for the existence of that field for projects that have upgraded from a previous version, but ideally all projects that start from this release onwards would have the field no matter what.
The reports that turn notifications OFF set the state of scheduled messages to muted. I don't see a reason to have different states... did you have something in mind?
Not necessarily... just wanted to understand the plan. This will make it pretty tricky to know _why_ a particular schedule was muted (OFF message, individual muted, family muted, etc...), but you can figure it out by looking at the timeline of potential mute messages and comparing to the schedule date/time.
Would this be only at doc creation, eg
contact_mute_state_at_creation_time, or updated whenever the contact's mute state changes, egcontact_mute_state?
I was specifically thinking of the first one (contact_mute_state_at_creation_time), but it's not a big deal because technically we can build the timeline of mute messages and I imagine it (creating schedules for muted people) will be pretty rare anyway.
@abbyad @michaelkohn I'm trying to follow the convo you're having. These are some assumptions I have about our muting workflow. Can you confirm which are still true? cc @amandacilek
-- muting a household mutes the people in that household
-- unmuting a person in a household unmutes the household. this is to prevent us from having "active" people in hidden households. you can then individually mute members of the household.
-- I had expected that if you reported for a muted person, particularly if it is a schedule-creating form, this would prompt you to unmute the person. It sounds like that is not a given, and I'm curious about that. If a CHW is actively treating a person but choosing not to receive notifications about them (and hiding them from their list, and some targets), what is the reason for this? This seems to signal a different motivation for mute than what was requested (if a person moved away or doesn't want to receive services anymore) and worth deeper consideration before we allow it.
-- OFF and mute are functionally the same, but we haven't had a "OFF" form for households yet (I think in part because it was created for SMS projects and we don't have households on SMS yet). Continuing to use "OFF" for muting individuals (or even naming it that way for households) works for me, as this is how our users have been trained and are used to it.
-- In order to keep the scope of this issue in check, we hadn't designed this for other levels of the hierarchy yet or taken into consider unique rules that might affect muting a place like a branch and applying that to all the people in that place, i.e. CHPs. It makes sense to build it to work generically, but we'll want to think through that before we broadcast to teams that it can be used for that.
First to note that family muting and individual muting have been written as separate issues. My understanding has been that family muting is the primary focus for right now (and for this ticket) and that enhancements to turn on/off into a true individual muting feature are nice to have but not required and not necessarily bundled together with this work.
Yes. Muting a household automatically mutes schedules attached to household-level workflows as well as all schedules for all people in the household.
Yes, if a user wants to unmute an individual in a muted household, the whole household will need to be unmuted. There is no such thing as an unmuted person in a muted household. BUT, I seem to recall discussing that the unmuting of the household needs to happen on the household profile, meaning I don't think you can do it directly from the person profile from the individual on/off form. So my understanding is the best we can do is _warn_ them that the individual is muted, and _recommend_ that the user exit, go back to the family profile, and unmute family before proceeding.
The plan I tried to describe is that: yes, whenever a user tries to start a form for someone who is muted, they will first get a warning dialogue message telling them that the person/family is currently muted. So the "prompt" is just a warning dialogue that requires the user to do additional actions. But we never consider Michael's point about SMS users...clearly they wouldn't get the pop-up warning, which does make me question our plan a bit...
I keep trying to work this out - it seems that we're already supporting individual muting. What individual muting features are left in the backlog beyond what we can already do? Are you referring to changes such as showing muted people differently in the household list or selectively muting schedules on a profile?
I believe this sentence is originally from Gareth: "we need to make an extension of that to also _mark_ the patient muted and make the UI changes to their profile to reflect the “Muted” status". My interpretation of this was that it's not just a matter of the UI changes to the profile. I guess we currently don't "mark" the patient as muted in the way that we need to.
Got it. That sounds like the field that Michael was suggesting. Muted: true/false.
((My above 3 answers are based on my original understanding of the plan, may not reflect what Michael and Marc have been discussing. Curious to hear their thoughts. Also, I am clearly not the expert in how this should be handled on the back end - I wrote the original issue but I'm relying on Michael and Marc to fill in the gaps and give recommendations or suggestions if something needs to be changed))
Noting that @michaelkohn and @Fatoufall should be included in the acceptance testing.
Also include @Enock1990 in the acceptance testing.
@michaelkohn @Fatoufall @Enock1990 gamma.dev has been setup with muting. The admin credentials are in one password. Please have a look to ensure it meets the expected requirements.
For the requirement below. Does the configuration of a task need to handle this or is it expected to happen by the webapp code? @dianabarsan @abbyad
If there was an existing open task for a now muted person/place on the Tasks tab, that open task should disappear from both the Tasks tab and the muted person/place’s profile
@newtewt Task configuration would need to handle that.
I believe these test cases cover muting and have gone over them in gamma.dev. I also have setup a meeting with Fatou and Micheal to give a quick demo and get them setup to run through muting on their own. Would appreciate any other areas I may have missed as part of this and other test cases if you can think of any.
Mute Person and no other areas muted.
Mute Area and all people under are muted
Mute District and all areas and people are muted
Unmute at any level will unmute top most and cascade down.
Condition cards remain. Ex: Pregnancy card, immunization
Tasks/Targets are muted based on config
User is prompted if they start new form on muted contact. New form is saved.
Mute/unmute VIA SMS
Add person to muted household ensure muting applies
Muted Styling
Different forms can be used to mute/unmute
@Enock1990 @joyannee please look over this list and see if you can think of additional test cases for the muting feature and/or correct the ones above.
By areas do we mean CHV areas? In that case mute households and all under the family are muted
Hey @joyannee I'm using the standard config and the Hierarchy looks like this. District > Health Center > Area. Does that answer your question?
To your second question. Muting the household would mute anyone associated under that household. You could also mute individuals within that household.
So in that case, "areas" as the lowest level of hierarchy maps to "households" in most of our app projects, so in your mind just swap out the word/concept.
Got it! Thanks
Got some feedback during the demo and going to share here.
Can contact cards disappear based on condition to the cards? Do we have access to contact in the card config.
Yes we can condition the contact cards/fields by using contact.muted in the appliesIF
Wanting to know when a person/place was muted/unmuted. If cascades are occurring then it might be hard to know which report unmuted.
Was hoping we could display the timestamp that's currently being set on the doc. However, the contact.muted property available in the contact-summary is always a boolean so displaying that as a field is either true or false. This request would need to be added as an addition to this feature in another ticket if we decided to do this.
Wanted to know if grouping of muted contacts on the left hand side was added.
Not in the first iteration but would like to be prioritized and added at some point.
Why are we using time stamp instead of true/false for muted. Would prefer not to have to mix datatypes. Probably null instead of false.
Diana and Michael discussed a change here to what's being stored. @michaelkohn would this be something worth holding up this ticket for?
https://medic.slack.com/archives/C024KTGRW/p1542293416241500
Where does the timestamp get set from. Form being submitted. Server time. Sentinel time?
The time stamp is set based on when sentinel processes the muting. Since this is when the person would actually become muted.
@dianabarsan looking to add the changes you and @michaelkohn had discussed.
Making the muted property look more like.
{ muted: true, date: 172893719, report_id: 'a' },
{ muted: false, date: 1953213131, report_id: 'b' }
Actually, we discussed saving a mute log in the sentinel info doc. Is that right, @michaelkohn ?
To be clearer:
The muted property remains as it is now, evt. removing it instead of setting it to false to avoid having to check for multiple data types.
When muting/unmuting a contact, an entry would be added to the sentinel info doc, which would look like:
{
muted: <true/false>,
date: <muting sentinel timestamp>,
report_id: <doc _id of report that triggered the muting/unmuting>
}
Would you also like the muting timestamp to be exposed to Contact Summary as an addition to this new feature request?
Yeah I think that would be good. So we could easily show when they were muted
Thanks!
Hey @newtewt & co. I did a few tests and I think this works well for me. Just an observation/bug; after muting a Health Facility, then I click on a CHV Area I'm able to unmute the CHV Area, other CHV Areas plus the Health Facility.
Merged into master, back ported to 3.3.x and 3.2.x.
This looks good to me. I'm waiting for responses on feedback and will then close this out.
From a data architecture perspective, this ended up being implemented as follows.
When a mute or unmute form (data_record) is submitted, this will update the appropriate place/person records in the medic DB and the corresponding -info record in the medic-sentinel database. The updates are done as follows:
muted field is added whenever a person or place record is mutedmuted field will be the timestamp of when the person/place was mutedmuted field will retain the original timestampmuted field is completely removedperson, clinic, health_center, district_hospitaldoc ? 'muted', this checks the existence of a key in JSON. If the key does not exist, doc ? 'muted' will evaluate to falsehealth_center{
"_id": "8d2e5384-0421-45ea-9cba-6adf09d5296f",
"_rev": "11-58ec8cb4a26de08bd12ea1ba80446a35",
"parent": {
"_id": "45cc55c5-302d-49ce-8edf-fe77a8e0f110"
},
"name": "Red Foreman Area",
"geolocation": "",
"contact": {
"_id": "ef0d26b8-bf27-41f3-a9ac-f7613238b0c3",
"parent": {
"_id": "8d2e5384-0421-45ea-9cba-6adf09d5296f",
"parent": {
"_id": "45cc55c5-302d-49ce-8edf-fe77a8e0f110"
}
}
},
"type": "health_center",
"reported_date": 1542228245476,
"muted": "2018-12-03T15:36:08.639Z"
}
The muting history for a person/place is accessible from the corresponding info document stored in the medic-sentinel DB.
muting_history key will be addeddata_record that caused this record to be muted or unmuted.medic DB, if the person/place is already muted, muting them again will not add or update anything in the muting_history. The same is true for unmuting.info doc{
"_id": "8d2e5384-0421-45ea-9cba-6adf09d5296f-info",
"_rev": "20-14c194aa7f039a72aee3367d4d778cbd",
"type": "info",
"doc_id": "8d2e5384-0421-45ea-9cba-6adf09d5296f",
"initial_replication_date": "2018-11-14T20:44:05.657Z",
"latest_replication_date": "2018-12-03T15:36:10.617Z",
"transitions": {},
"muting_history": [
{
"muted": true,
"date": "2018-11-29T14:31:57.250Z",
"report_id": "7e09d2fb-0561-4806-84f7-0d362b959827"
},
{
"muted": false,
"date": "2018-11-29T21:47:07.020Z",
"report_id": "749aaf7c-bee7-4fc1-bc49-7d0c7f445458"
},
{
"muted": true,
"date": "2018-12-03T15:27:08.287Z",
"report_id": "b30945da-19cd-44e6-bd74-7b56e55a917a"
},
{
"muted": false,
"date": "2018-12-03T15:27:26.628Z",
"report_id": "94634b6b-2054-4422-85cb-34f7a75883e5"
},
{
"muted": true,
"date": "2018-12-03T15:36:08.639Z",
"report_id": "526ee71b-4c90-499a-b8f7-2f4a9864aefb"
}
]
}
Nothing has changed. Review documentation about message states here
This is a really helpful summary @michaelkohn! We should make sure it is persisted as documentation too. Can you add it to this doc, or a separate one in medic-docs/configuration?
Most helpful comment
From a data architecture perspective, this ended up being implemented as follows.
When a mute or unmute form (
data_record) is submitted, this will update the appropriate place/person records in themedicDB and the corresponding-inforecord in themedic-sentineldatabase. The updates are done as follows:medic DB
mutedfield is added whenever a person or place record is mutedmutedfield will be the timestamp of when the person/place was mutedmutedfield will retain the original timestampmutedfield is completely removedperson,clinic,health_center,district_hospitaldoc ? 'muted', this checks the existence of a key in JSON. If the key does not exist,doc ? 'muted'will evaluate tofalseExample of a muted
health_centermedic-sentinel DB
The muting history for a person/place is accessible from the corresponding
infodocument stored in themedic-sentinelDB.muting_historykey will be addeddata_recordthat caused this record to be muted or unmuted.medicDB, if the person/place is already muted, muting them again will not add or update anything in themuting_history. The same is true for unmuting.Example muting history stored on the
infodocScheduled SMS messages
Nothing has changed. Review documentation about message states here