Cht-core: Integration with Africa's Talking SMS aggregator

[garethbowen]

A new project EBS wants to use feature phones and SMS instead of smartphones and data. Usually we use medic-gateway running on android to send and receive SMS using medic-api APIs, but it has some issues including being dropped from the play store, the app going to sleep, low throughput, high latency, etc. It would be much better to use a specialized third party cloud hosted aggregator instead. For this project we've chosen Africa's Talking.

1. Create a new routine to send SMS via Africa's Talking (both immediate messages such as auto replies, and scheduled messages generated periodically).
2. Create a new API (potentially multiple endpoints) that Africa's Talking will call when an SMS is received.
3. Create a new API to update the status of outgoing messages as they are delivered.
4. This integration must be disabled by default, and all the account information must be configurable.

Africa's Talking provides a sandbox mode you can use during development and testing, and have support if required. The also have an npm SDK package which exposes a simple API to use, as well as tutorials on how to integrate.

There will likely be more integrations with other providers at a later date so consider refactoring the gateway and Africa's Talking integrations to use shared services where possible to make reuse easier for the next integration.

@newtewt prototyped an integration using expressjs which almost worked and may be useful as a starting point.

[newtewt]

I've added the prototype to a branch off the webapp here. @derickl or I can give access to the sandbox we setup on their site if need be. The sandbox API key is found in the index.js. Since we are using them we should create an org account and get rid of the personal one we were using.

https://github.com/medic/medic/tree/africas-talking-demo/scripts/africa-talk

[dianabarsan]

Ready for AT on 5604-africas-talking-integration. Please do not merge post AT.

[newtewt]

@garethbowen should we update this documentation as well?

https://github.com/medic/medic-docs/blob/master/user/message-states.md

I'm trying to find details about why something failed. It seems like anything outbound is failing for me but I'm able to receive from Africa Talking. I kinda assume the failure is expected since it's not a real message sent but I guess I also expected the sandbox to just return a success since it's a fake.

Maybe my lack of knowledge on the SMS process but it seems like we should have some more info or logging around this. Maybe that is another feature request an aside from this ticket.

[newtewt]

Also worth noting that this bug affects this issue.

https://github.com/medic/medic-os/issues/46

[garethbowen]

should we update this documentation as well?

Good spotting! Updated in the medic-docs PR.

I'm trying to find details about why something failed. It seems like anything outbound is failing for me but I'm able to receive from Africa Talking.

I had the same experience using the sandbox and also assumed the failure is expected in this case. We won't know until we have a production ready Africa's Talking account to test against.

[newtewt]

This is good as it can be testing against a sandbox environment. As we likely need a real account to really see the test go through and not just fail as both Gareth and I have seen.

Leaving this open for now as we were hoping to get a project with a shortcode setup to run through this.

[ngaruko]

I will move this to Done to kick off release testing. Will eventually test again once the ideal environment is available (during release testing hopefully)

[ngaruko]

One branch here has conflicts @dianabarsan

[garethbowen]

Conflicts resolved. I'll merge once the build passes.

[garethbowen]

Merged.

[garethbowen]

We weren't able to do a full AT on this issue as it requires a production account with Africa's Talking. We've decided to release as is and test it in the field. If any issues are found we can patch and release a service pack quickly.

[benkags]

@garethbowen

Began testing this with the prod short code today and here is some feedback:

• Incoming messages are working as expected but non instantaneous. There seems to be a delay going up to 2 mins. Not sure if this delay is from AT's end.
• Couldn't get outgoing messages delivered. Console output indicates an attempt at sending the messages but they never hit AT (nothing in outbox). Note that the API Key and app settings have been configured as per the documentation. Could we be missing something ?

[dianabarsan]

@benkags What are the local statuses of the messages that should have been delivered?

[derickl]

@garethbowen @dianabarsan I've also set this up locally. Outgoing messages are still pending due to a possible error from Afticastalking. I added some logging in api/src/services/africas-talking.js

This is what we get back from when we attempt to send

{"SMSMessageData":{"Message":"Sent to 0/1 Total Cost: 0","Recipients":[{"cost":"0","messageId":"None","number":"+254XXXXXX","status":"RejectedByGateway","statusCode":502}]}

I'll reach out to AT support tomorrow

[derickl]

According to their documentation - http://help.africastalking.com/en/articles/742491-why-did-my-messages-fail, that error occurs when using a short code not mapped to our account. We did receive confirmation of the mapping

[garethbowen]

@derickl It sounds like you've got this covered - let me know if I can help!

[derickl]

Waiting for feedback from Africastalking. The wrong type of shortcode (on-demand rather than toll-free) was provisioned on the Telco's end. I'll be reaching out today for an update.

[derickl]

I tested this today and was able to send out the queued replies