Cht-core: Reminders are not working

[dianabarsan]

Reminders are not sent because the clinic docs are not hydrated before passing them to generate the message, thus the recipient phone number will never be matched: all reminder tasks will have a recipient of "clinic". At least, if the message configuration is not templated to be unique, the tasks will get a duplicate state and not spam gateway.

Also:

• back in 2013 someone renamed a task field from code to form (https://github.com/medic/medic/commit/e626ba7acba951a139a6d176eec634e0cea0f448#diff-664792df5ba92ac7b86ac6a683087fdeL98) but the view which checks when last reminders were created still checks for code, so it's a useless view ATM https://github.com/medic/medic/blob/fc315d632da137105bf6563ae121f25007314f40/ddocs/medic/views/sent_reminders/map.js#L4

• reminders don't have a timestamp or due property - they have a ts property, so their position in admin message-queue will be dependent to doc.reported_date (in this case, the moment the clinic was created).

Looking into projects, there is only one project that has "reminders" configured.

Is this a feature that we should fix or deprecate?

cc @garethbowen @abbyad

[garethbowen]

Which project is the lucky one? Depending on which project it is I'd vote for removal - it's clearly very broken.

[dianabarsan]

The lucky project is ipd-nepal. I'm not sure what the state of the project is, there are recent reminders created by sentinel (last ones from yesterday), but it appears that none have been sent in a long time. It also appears to run an ancient version of the app (0.4.16).

[abbyad]

That project was meant to close out at the end of 2018. Moving the project-specific conversation to Slack.

It is not clear which reminders we are talking about here. Is it something from our current transitions?

[dianabarsan]

@abbyad These reminders

[abbyad]

I am not sure if the feature is being ignored because it isn't useful, or because it is undocumented. We implemented a similar feature in Medic Collect as Android notifications which had some limited use as well. They are useful for CHWs that need to submit periodic reports, which is something which is done infrequently in our SMS projects these days. I'll get some input from tech leads to see if they would use it.

[garethbowen]

@dianabarsan If it's a quick fix let's patch it. If Marc finds out it's useful but nobody knows about it let's call it a new feature!

[abbyad]

Ya, with limited feedback it already sounds like it is potentially useful for us and CHT partners, and worth doing now if it is a quick fix.

[dianabarsan]

It's not a lot of work, but also not a quick (20 min) fix per se.

I'm in disagreement with the way this feature is implemented: saving tasks (along with all their history) in the clinic document. It can, over time, create huge documents with data that is irrelevant to anyone except our backend stack (Sentinel, API & Gateway) as we do not display this information anywhere in the webapp.

I think it would be better to store these tasks separately, maybe even in individual docs, but at least detached from the clinic doc.
If we decide that they are not needed in Webapp (as they are now), we don't even need to replicate them to offline users and could even use a different doc type (fe reminder) to not pollute other views.

[garethbowen]

Oh man, you're right, that's horrible for smartphone users. It's so far out of sync with how we work now. Let's drop it and list it as a breaking change. If people want it we can rebuild it in a way that's consistent with the current workflows.

It may be that we run nools on the server and generate tasks which are SMSed to SMS users and displayed in the tasks tab to smartphone users or something like that.

@abbyad Does that sound ok to you? Have you had any response from people wanting this feature? It really does seem like a useful thing to have...

[abbyad]

We got two responses in favor of using it, but nothing immediately planned. Although it seems worthwhile to rebuild properly, the projects with SMS workflows do not typically have offline users so I don't think we'd encounter that problem yet.

Also, just a note that these messages should display in the outgoing messages section of the admin console now.

It may be that we run nools on the server and generate tasks which are SMSed to SMS users and displayed in the tasks tab to smartphone users or something like that.

I think this a great longer term vision for SMS/task reminders, and we may make some progress on it with supervisor features (eg writing tasks to disk). Curious if we can have a quick fix to have this operational until we get the more full featured reminders.

[garethbowen]

The quick fix is to patch it and leave it updating docs over and over again which is bad for anyone replicating. I don't think there's a quick fix for getting it to behave with decent performance.

The choice for now is to delete it completely, or patch it so it works poorly.

[garethbowen]

@abbyad What do you think?

[garethbowen]

@abbyad ?

[abbyad]

Given that folks are interested in using it and that we don't have projects that would encounter the performance issue I think that we should do the quick fix patch. As part of this issue we should also put a disclaimer on the documentation that this should only be used when there are no mobile app users.

cc @MaxDiz

[garethbowen]

Ok. Let's patch this to get it working, and update the docs as @abbyad suggests. I've raised a new issue #5475 to make it better.

[ranjuts]

Would potentially need this for MOH Nepal very soon - unclear whether this is the right feature for what we need. Wanted to check whether what we need is supported by any other existing/working feature instead.

• CHWs (FCHVs) receive SMSs on a regular basis (monthly or bi-monthly)
• Each SMS would have different content; a group of messages needed for a period of time (6 months or a year) would be defined and configured -- similar to what we have for ANC within a group of messages
• CHWs would be expected to respond with a pre-defined text form
• Needs to be triggered by CHW contact creation instead of a report. Would only need to mute or clear the schedule if the CHW is no longer active as a CHW.

@abbyad could you help us out here? cc @ashish-medic @daniellearon

[abbyad]

@ranjuts lets discuss in Slack to see if this feature would do what you need.

[dianabarsan]

Ready for AT on 5351-fix-reminders
Please do not merge post AT, a shared lib has to be published and dependencies updated.

[tookam]

@dianabarsan conducting the AT of this ticket .. couple questions:

These two messages were created few minutes ago but they are showing 5 months ago in the field Created .. am I doing something wrong?

Also, are these reminders supposed to show up in the webapp?

[dianabarsan]

@wtekeu
The "created" column shows when the document the reminder is attached to was created. Reminders are attached to clinics, so you would have the document reported_date field value there.
The "due" and "last update" columns refer to the reminder task itself.

The reminders are not displayed anywhere in the webapp.

[tookam]

@dianabarsan
LGTM
Did not merge.

[tookam]

By the way, tested using the following config:

    "reminders": [
        {
            "form": "P",
            "text_expression": "every 2 minutes",
            "cron": "",
            "mute_after_form_for": "3 days",
            "message": "Please submit last week's VPD report. Thank you!"
        },
        {
            "form": "P",
            "text_expression": "every 3 minutes",
            "cron": "",
            "mute_after_form_for": "3 days",
            "message": "Today is the last day to submit last week's VPD report. Please submit. Thank you!"
        }
    ]