Cht-core: Ability to configure SMS sending to other contacts that aren't the primary contact

[benkags]

EDIT The description below is outdated, see this comment: https://github.com/medic/cht-core/issues/5817#issuecomment-607838321

---

Is your feature request related to a problem? Please describe.
As it is now we do not have a way of sending an sms to anyone else other than reporting_unit, parent, grandparent or a phone number (either hard-coded in the config or as a field in the report) in SMS reporting configuration.

User story:

A supervisor who is in an area above a CHV area in the hierarchy needs to get a notification when the CHV submits a report. The supervisor is not the primary contact of the area.

Describe the solution you'd like
1) Ability to broadcast an SMS nottification to everyone at a level (levels being: parent, grandparent, et al).
2) Ability to send an SMS notification to someone who is not the primary contact of their place.

Describe alternatives you've considered
There exists the option for the user to type in the phone number when sending in a report but that is just not user friendly.

Additional context
Discussed this a bit with @SCdF and this is a probable way to approach (2) above:
_In addition to existing ways of specifying the recipient, add another option that uses a JSON path that resolves to a UUID or shortcode, which is used to pull a contact, where we will look for their phone number in the same places that we'd look for the hard-coded relationships above (reporting_unit, parent ...)_

Also came across #333 & #427 and while they seem related, they don't seem to address this particular problem.

[garethbowen]

@benkags Great description, thanks! I'm just wondering how we would work out who to notify for a given CHV. Usually I'd expect the supervisor to be the primary contact of the parent of the CHV area. Why is this not the case here? Is there something in the database that links a CHV to their supervisor?

[benkags]

A good example of cases where the supervisor is not the primary contact is that of LG. The primary contact at the branch (highest level in the hierarchy) would be the branch manager. While that individual could be a supervisor for some of the CHVs there are other supervisors in that level and are linked to the CHVs by having a supervisor key in the CHV's place object that contains the id of the supervisor. The exercise of assigning supervisors to the CHV places is called supervisor mapping and is done via a script.

In the EBS project, hierarchy is the of Kenyan county governments starting with sub country as the highest level: sub country -> community health unit (CHU) -> chw. The CHW reports (events) via sms that are meant to go to someone at the CHU place. In most cases this would be the primary contact but there is a use case we are not able to support where some of the notifications need to go to a different person at that level based on skill set need to review the report. It get trickier here because we wouldn't be mapping the CHVs to the person the notification goes to, rather, the type of report submitted determines where the notification goes. Type of report here can be determined by either type of report or attribute in the field of report. _My idea for a possible way to approach this is have the contacts at a place, e.g CHU in this case, have an identifying attribute e.g {profession: 'nurse' }. We'd then specify recipient as byAttribute: {`profession`: `nurse`}. We'd then look for the phone number by first iterating through the contacts in the place to get one (or more) of qualifying one(s). A bit inefficient I know , so there is probably a different way to look at this._

[garethbowen]

So there is nothing in the database currently that links the CHV to their supervisor? Or conversely, links the supervisor to the CHV?

Could this be solved by adding more hierarchy levels, so under the branch would be some "supervisor areas", and under that would be the CHV areas? That way the supervisor could be the primary contact of the supervisor area, and you could send the message to "grandparent" (or similar) as per usual.

[benkags]

In the LG case, there exists a hard link in the database using the key supervisor and with three levels of the hierarchy (branch -> chv area -> family) there isn't room for additional level (unless in flexible hierarchy ??).
In the second case there isn't linkage.

[garethbowen]

Yes I was thinking about once flexible hierarchy ships in 3.7.0. Would additional levels of hierarchy be a workaround for this for those two projects?

[benkags]

For the EBS project, where this would be of use, flexible hierarchy wouldn't quite solve for it. Unless with flexible hierarchy one CHVs can belong to two places at the same time. The notification routing isn't based on the hierarchy but reports, which means that for the same CHV a notification could go to either of the contacts depending on what report they have submitted.

This is feature isn't required for LG at the moment as far as I can tell, I was just using it to demonstrate the a direct linkage of supervisor to chvs and a possible scenario where even with flexible hierarchy you would want to have the supervisors in the same level. Sorry about that confusion.

[benkags]

Could we consider prioritizing this for soon? cc @kennsippell

We aren't able to support a workflow in Event-Based Surveillance (EBS) that requires CHVS and Animal Disease Reporters(ADRs) all reporting to either a CHA(Community Health Assistant) or an AHA(Animal Health Assistant) depending on whether the signal they were reporting was animal-related or not and as a result, advised the partner to manage part of this workflow outside the system. Having the ability to send an SMS notification to both CHA & AHA would have been really useful. There is an expectation that we will be in a position to support this by the time we are completing the pilot, as it is critical for scale-up that animal-related events are triaged by the personnel from animal departments within the system.

cc @Maigua

[MaxDiz]

Hi @benkags re ^ above comment, is this contractually obligated or a loose expectation on feature availability? This is the first I've heard of a direct project deployment use case. It would be good to talk to other partners to understand if/how this feature would be used to help determine priority.

cc: @n-orlowski - issue is in your backlog

[benkags]

I'd say contractually obligated as that is part of the requirements wasn't fully fulfilled but @SMurithi can comment here.

[PhilipNgari]

Thanks team. Its a contractual one. We initially indicated to the CDC team that it was a product limitation as we entered into this,however they are expecting it. It is also largely driven by the their global thinking around one health (animal+human), a concept they are rolling out in all countries since Last month starting with Uganda.

[SCdF]

tl;dr; we should change the phone number resolution logic so it supports UUIDs, that we then load and we pull the phone from that (as opposed to having to resolve into concrete phone numbers).

--

I've discussed this with @benkags, as the use case and proposed solutions didn't completely align in my mind. Specifically, this is not about adding the support for sending the same SMS to multiple people, nor is it about adding support for sending different SMS to different people (you can do this already).

The current issue is that the way you specify a recipient is not flexible enough, as it presumes that the phone number you wish to send an SMS to is directly available in the given data structure. This is fine if you want to, say, send an SMS to your facilities primary contact because the primary contact is attached to the place and is so accessible. It is not fine if the person you wish to contact is not directly attached.

An example of where this might occur is if you have multiple specialised people in a facility: so you have a primary contact, but you also have a supervisor of all immunisations, or all pregnancies, or whatever. There are two logical ways to describe this relationship:

• On the record for the supervisor, have "immu_super": true to "tag" them as the immunisations supervisor
• On the record for the place, have `"immu_super": "uuid_of_super"

As long as these relationships are static and 1:1, the latter is more advantageous than the former:

• When creating or editing the place you can specify and search for the super and attach them
• The 1:1 relationship is maintained correctly (if it's against the super then multiple can be marked as the immu_super)
• It means that this relationship is available in the hierarchy

So now we have the UUID, the only piece of work needed is to update our phone resolution code to support taking UUIDs, loading them from CouchDB, and using their phone field.

(If those relationships are not static and 1:1 then the solution is much harder and more complex, because it's basically giving phone number resolution arbitrary code execution, including querying the DB).

[SCdF]

More context: other, more complicated solutions:

• as above, + adding expr support, so you can write stuff like contact.parent.parent['chw_super_' + contact.chw_region]. So more flexible, but more error prone
• all of the above, but support special functions that hit the DB, so you can write stuff like contact.parent.parent.children().find(c => c.is_immu). Even more flexible, but we'd have to add these functions, it's not clear how they'd work exactly, and you've gone from safe linear time (resolving a path) to arbitrary js, to arbitrary js that _hits the database_, so way more error prone

[MaxDiz]

de-prioritizing this issue for now to redirect our attention to covid work. The project is currently on hold

[benkags]

@MaxDiz the project is technically not on hold. There are users in Siaya and Nakuru counties using the system and those are the users we are not able to support fully without this feature. What is on hold is a deployment to a third county. Please reconsider its prioritization.

[MaxDiz]

ok, will bring it up at the next roadmap prioritization discussion

[kennsippell]

We are scaling our existing EBS systems to new counties and spinning up new EBS systems for our COVID response This issue was listed as the top issue to prioritise for COVID EBS scale-up response.

[SCdF]

@benkags hey so we realised that instead of attaching the UUID of the person we can just attach the phone number, and so no dev is required? We already support paths so you should be just be able to use that.

Can you confirm this works for you? So basically:

• Changing the place form so that you can use db-object to assign managers as contacts for certain things. Store their UUID and phone against the object (uuid so when you edit select2 can repopulate)
• Then configure the path appropriately

[SCdF]

To be clear, if that works it doesn't nullify this ticket just reduces it's priority, since directly linking the phone number means people can't change their numbers, and so isn't a long term solution.

[SCdF]

@benkags has said that he's going to confirm if this works today. If it does we don't need to do this (or at least not in 3.9.0)

[benkags]

Tested and confirmed:

• Can use 'db-object' to select an existing person to attach to a place as an object that includes the phone number
• Can edit place including the attached object
• Can use path contact.parent.parent.aha.phone as recipient to configure schedules. Documentation needs to be updated to include this.

Todo:

• [ ] Test that you can create the person object to attach to a place if it's a new person during that place creation.

[benkags]

@SCdF any ideas on how to test the last bit above. See this related question. Basically, how do I create this secondary contact and attach their partial object to the place contact within the place creation form?

edit
See a sample form that works for selecting an exiting person. See the aha group.

[SCdF]

@benkags I'm presuming you can't just literally link it using an xpath calculation? If that doesn't work you've my level of expertise and need to pull in @abbyad

[SCdF]

Anyway, the issue you're working through wouldn't be solved by this fix anyway. I've moved this to 3.10.0, let me know @benkags if that is not the right call.

[MaxDiz]

Current workaround is ok for immediate project needs. The workaround does not support people changing their phone number, so it is not suitable as a longer term solution. Bumping to v3.11

Additional conversation: https://medic.slack.com/archives/C3TJFFKK7/p1588686499015300

[dianabarsan]

I was thinking we could add a new property on contact docs, called "contacts".
This would be an example of how a contact could look like:

{
  ....
  "type": "clinic",
  "name": "my contact",
  "contact": { "_id": "<primary_contact_uuid>" }, 
  "contacts": {
    "<tag1>": { "_id": "<tag1_contact_uuid>" },
    "<tag2>": { "_id": "<tag2_contact_uuid>" },
    ....
  },
  ....
}

<tag1>, <tag2> can be any string you wish.

We can change our hydration code to hydrate every entry in contacts (at 1st we can do a shallow hydration, just the main contact docs and no parents/grandparents) and you would be able to access them with something like patient.contacts.<tag1>.phone or we could add a "smart" path like we use for "ancestor" like contact:<tag1>.

How does this sound?

[garethbowen]

I like it. It's basically exposing a property that configurers can use to get something hydrated (and minified) automatically. Some thoughts about expanding on your initial idea...

1. The ancestor: smart path is a limited version of what you're proposing, ie: it links to some contacts but not all. Can we deprecate that and use this new smart path with implicit tags for parent contact types? eg: contact:clinic would work just like ancestor:clinic and we can drop ancestor eventually.
2. The property name contacts feels a little overloaded.
3. Is it useful to still have an object with { _id: "xyz" } rather than just a string ID? Initially that was done for backwards compatibility on the contact property so minified and hydrated docs would be equivalent. On a new concept I'm not sure it's needed.
4. What about being able to include reports, messages, tasks, etc in this field? I'm not sure when/if that would be useful but I can't see any technical reason to restrict it to just contact type docs.

I'm happy to discuss any/all of these - they don't all have to be done, and they certainly don't all have to be done in this release.

[dianabarsan]

1. The ancestor: smart path is a limited version of what you're proposing, ie: it links to some contacts but not all. Can we deprecate that and use this new smart path with implicit tags for parent contact types? eg: contact:clinic would work just like ancestor:clinic and we can drop ancestor eventually.

So contact:<tag> would search the whole lineage for a linked contact with that tag and contact:<contact_type> would search the whole lineage for a parent contact with that type? or do you expect contact:<contact_type> to look into "tagged" contacts as well to match the type?
More to discuss here, I think, it's not 100% straight forward.

1. The property name contacts feels a little overloaded.

Not sure what you mean by "overloaded". Are you suggesting we should use another name?
It'd, maybe, be nice do drop "contact" and have "contacts.primary" (not saying we should, but this came along with the first thought).

1. Is it useful to still have an object with { _id: "xyz" } rather than just a string ID? Initially that was done for backwards compatibility on the contact property so minified and hydrated docs would be equivalent. On a new concept I'm not sure it's needed.

Sure, I don't mind that at all (to be honest I had no idea why the contact is an object until now). Everything should be null-checked pre-assignment anyway.

1. What about being able to include reports, messages, tasks, etc in this field? I'm not sure when/if that would be useful but I can't see any technical reason to restrict it to just contact type docs.

It _may_ make hydration code somewhat more complex, besides that no objections. But then the field should definitely not be named "contacts".
I don't oppose linking other doc types, but I like transparency, and when I see a uuid in a doc, it'd be nice to know at least what type of doc to expect there.
Maybe they could be separate.

[garethbowen]

So contact: would search the whole lineage for a linked contact with that tag and contact: would search the whole lineage for a parent contact with that type? or do you expect contact: to look into "tagged" contacts as well to match the type?

I was thinking that contact:abc would look in the tagged contacts for abc and then if none found would look in the hierarchy for an ancestor with type abc. It's not a big deal, just trying to simplify configuration of resolving recipients.

Are you suggesting we should use another name?

Yeah. It's weird to have contact and contacts.

Maybe they could be separate.

Yes I suppose so. If that's the case definitely don't do the work now - we can do it when a workflow that needs it comes up.

[dianabarsan]

Yeah. It's weird to have contact and contacts.

"linked_contacts"? I feel particularly uninspired this morning 😁

[garethbowen]

"linked_contacts"?

Yes that's about the best I could come up with too :/ For the generic solution I was thinking linked_docs but that's even more uninspired...

[dianabarsan]

Ready for review on 5817-linked-contacts along with this documentation PR: https://github.com/medic/cht-docs/pull/145

[brad1905]

@dianabarsan Passed AT, ok to merge
Tested default and customized hierarchy, each type of recipient for each level in hierarchy, validated correct recipient scheduled for SMS

[dianabarsan]

@benkags would you like to give this a go before I merge?

[dianabarsan]

Merged into master.

[benkags]

Sorry @dianabarsan I couldn't get to this before merging. I'll try get some time early next week and test it.

[ngaruko]

@brad1905

Tested default and customized hierarchy, each type of recipient for each level in hierarchy, validated correct recipient scheduled for SMS

Can you please share a little bit of testing steps, configuration and if possible, code samples for the tech lead training demo and future reference? Thanks