Cht-core: Import People doesn't work for large numbers of people

[garethbowen]

The Admin > Import & Export > People page needs to be improved. Firstly one request is made per uploaded person which is inefficient. Secondly the requests are made in parallel which is guaranteed to fail with a large enough import. Thirdly it's implementing the logic that's already implemented in the /api/v1/people api. Finally it would be better to be in medic-conf (https://github.com/medic/medic-conf/issues/193 maybe related).

1. Work out if this feature is needed in the UI. If not, implement it in medic-conf and delete it from the admin app.
2. If it is needed in the UI, create a new v2 API which takes an array of people to create or update, not just a single doc. Then update the UI to just call the API.

[garethbowen]

@abbyad Can you help work out if this feature is needed in the UI?

[abbyad]

I think we ultimately want a UI version to import people, but think that the way it is now is not ideal anyhow. The export is in CSV, and the import in JSON. That means that anyone wanting to import or bulk edit people can't just do an export, modify the file or add new rows, and upload the same file. Until we get that functionality I think it is fine to move this to medic-conf. We can then design an UI import that would be more functional.

cc @MaxDiz

[garethbowen]

Ok. I propose we leave the UI as is for now, remove this issue from 3.8.0, and schedule a broader issue for redesigning the UX altogether.

[MaxDiz]

@amandacilek could use your recommendations on how to pursue a more comprehensive upgrade to this feature

[amandacilek-zz]

My best recommendation is to ask a lot of questions :). Find out who the primary users are, figure out their needs, and then plan a solution accordingly.

• Who currently uses Import/Export of People?
• In the ideal future CHT world, who might be using Import/Export of People?
• What level of technical knowledge and background do these users have?
• What do these different users need to accomplish? Provide examples of their setup, workflow, and how they use the feature.
• In this context, what does "People" mean? What sorts of people can be imported and exported? App users? Patient contact lists? All/any of the above?
• Does this feature only get used at initial project setup? Or on an ongoing basis?
• Are they generally uploading a brand new contact list? Or making small edits to an existing one?
• Are they almost always checking "Overwrite existing records" or almost never?
• What is the preferred file format to work in (CSV, JSON or something else)?
• Should it ideally support multiple file formats?
• Would the same user both import and export, or would those potentially be different users?
• Would it be helpful to see some sort of preview of all of the data before or after importing/exporting?

[abbyad]

Just encountered a situation where this would be very helpful for a partner who wants to bulk upload and update contacts. I have directed them to csv-to-docs.

[newtewt]

There's another problem with this page. The export references that it would come down as json but the file is a CSV. The expected upload file is a json file.