Collect: Scan the instances folder for new instances and insert it into instances provider

[yanokwa]

This functionality is needed because sometimes you need to add a submission to a device. The use cases are...

1. An implementer who wants to programmatically generate pre-filled instances with folders. This is useful for follow-up studies.
2. An implementer who moves instances from one device (data collector) to another (manager) to implement a supervise-and-send workflow.
3. A developer who wants to test a fix (See #415 for an example).

Currently, the only way to add instances is to copy the /odk folder with the relevant databases. This overwrites any other existing ODK data. My proposed feature is more about being able to incrementally add, and eventually remove submissions.

[nribeka]

This probably not gonna fly easily.

tl;dr:

• You can only relate an instance record with the instance xml file using the path.

The instance record looks like this:
2|sample||true|/storage/sdcard/odk/instances/sample_2017-02-27_23-29-21/sample_2017-02-27_23-29-21.xml|sample||submitted|1488257479383|Sent on Mon, Feb 27, 2017 at 23:51|

The instance xml looks like this:

<?xml version='1.0' ?>
<sample_xlsform id="sample">
        ... data stuff goes here
    <metadata_note />
    <start>2017-02-27T23:29:21.843-05</start>
    <start_test_output />
    <end>2017-02-27T23:30:38.245-05</end>
    <end_test_output />
    <today>2017-02-27</today>
    <today_test_output />
    <deviceid>6395771f8f6c42f8</deviceid>
    <deviceid_test_output />
    <simserial>89014103211118510720</simserial>
    <simserial_test_output />
    <phonenumber>15555215554</phonenumber>
    <phonenumber_test_output />
    <meta>
        <instanceID>uuid:3a6b9f77-40b6-46ca-a687-f23eaec77d2f</instanceID>
    </meta>
</sample_xlsform>

One approach that can be done I think:

• Create a list of path that's already added in the instances table
• Loop through the instances folder and skipping path that in the already added list
• For each path that's not in the database yet:
  
  
  
  • Parse the instance xml file to get the id of the form
  
  
  • Query the form table to get the form info
  
  
  • Construct the instance record add it using the provider
  
  
• Cross your fingers this process will not consume all of your battery

I think this should be a manually triggered process instead of automatic one.

[lognaturel]

@yanokwa Can you please describe a use case in your initial issue description?

[nribeka]

This is simple shell script to duplicate one of your instance if the instances folder. This script will take the folder name of one instance and duplicate it 100 times and rename the xml file to match the folder name.

#!/bin/bash
for ((i = 1; i <= 100; i++)); do
    cp -R $1 $1_${i}
    mv $1_${i}/$1.xml $1_${i}/$1_${i}.xml
done

To run it, you'll need to be in an instances folder, then ./script.sh single_instance_folder_name