Collect: Can't send photos from collect to google spreadsheets anymore

[calcahope]

Hi,

I'm pretty new with the software, i'm developing a form for my employees to send field reports (with photos) and received them to my mail. I made a form, conect it to google spreadsheets and then made a script on google to send the mail. All was good and fine until I had to change some things to the form and now i receive this error everytime I want to send a picture:

I don't know what to do, this error only shows when there is a picture involved, the old version still works fine but nothing new I develop works now not even if i make a form with only a button to upload a picture.

Software and hardware versions

Collect v1.4.14, Android v5.0.2, LG Magna LG-H52g

Problem description

Steps to reproduce the problem

just create a form on build or kobotools to send an image to a google spreadsheet. My form attached.
IssueReport.zip

Expected behavior

received a link on a cell to the image stored in google photo, as it was before. This is a working example:
workingExample.zip

Other information

Create new forms with only the button to send the picture, new tools to make the forms like kobotools, nothing works

Please help.

Thanks,

[yanokwa]

Hi @calcahope! I have been able to reproduce the problem. It's because the Google APIs that we use have changed. We are in the middle of updating them (Issue #367) and I hope we can have a fix out in a few weeks.

Until then, one workaround is for you to use ODK Aggregate as the backend and then use the Aggregate publisher to stream data to Google Sheets.

02-22 11:27:24.773 28448 31072 W System.err: com.google.api.client.http.HttpResponseException: 501 Not Implemented
02-22 11:27:24.773 28448 31072 W System.err: Insert is no longer supported
02-22 11:27:24.775 28448 31072 W System.err:    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1061)
02-22 11:27:24.775 28448 31072 W System.err:    at org.odk.collect.android.picasa.GDataClient.execute(GDataClient.java:87)
02-22 11:27:24.775 28448 31072 W System.err:    at org.odk.collect.android.picasa.GDataClient.executePost(GDataClient.java:107)
02-22 11:27:24.775 28448 31072 W System.err:    at org.odk.collect.android.picasa.GDataXmlClient.executePost(GDataXmlClient.java:86)
02-22 11:27:24.775 28448 31072 W System.err:    at org.odk.collect.android.picasa.PicasaClient.executePost(PicasaClient.java:70)
02-22 11:27:24.775 28448 31072 W System.err:    at org.odk.collect.android.picasa.PicasaClient.executeInsert(PicasaClient.java:79)
02-22 11:27:24.775 28448 31072 W System.err:    at org.odk.collect.android.picasa.PicasaClient.executeInsert(PicasaClient.java:83)
02-22 11:27:24.775 28448 31072 W System.err:    at org.odk.collect.android.tasks.GoogleSheetsAbstractUploader.getOrCreatePicasaAlbum(GoogleSheetsAbstractUploader.java:762)
02-22 11:27:24.775 28448 31072 W System.err:    at org.odk.collect.android.tasks.GoogleSheetsAbstractUploader.uploadOneSubmission(GoogleSheetsAbstractUploader.java:267)
02-22 11:27:24.775 28448 31072 W System.err:    at org.odk.collect.android.tasks.GoogleSheetsAbstractUploader.uploadInstances(GoogleSheetsAbstractUploader.java:143)
02-22 11:27:24.775 28448 31072 W System.err:    at org.odk.collect.android.activities.GoogleSheetsUploaderActivity$GoogleSheetsInstanceUploaderTask.doInBackground(GoogleSheetsUploaderActivity.java:412)
02-22 11:27:24.775 28448 31072 W System.err:    at org.odk.collect.android.activities.GoogleSheetsUploaderActivity$GoogleSheetsInstanceUploaderTask.doInBackground(GoogleSheetsUploaderActivity.java:359)
02-22 11:27:24.775 28448 31072 W System.err:    at android.os.AsyncTask$2.call(AsyncTask.java:295)
02-22 11:27:24.775 28448 31072 W System.err:    at java.util.concurrent.FutureTask.run(FutureTask.java:237)
02-22 11:27:24.775 28448 31072 W System.err:    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
02-22 11:27:24.775 28448 31072 W System.err:    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
02-22 11:27:24.775 28448 31072 W System.err:    at java.lang.Thread.run(Thread.java:818)

[yanokwa]

Another interesting tidbit from the mailing list "The error is also Google Account specific. From a test account from which I had done an upload previously (more than 10 days back), the new uploads are successful but failing from a Google account which is using this upload feature for the 1st time."

[shobhitagarwal1612]

Working on this issue

[lognaturel]

@shobhitagarwal1612 and I had a great conversation about this in #collect-code. From what we can tell, the only way to upload images to Google Photos (formerly Picasa) programmatically is through its gdata API. Unfortunately, gdata seems like its on its way to complete deprecation. The latest version doesn't seem to have any convenience wrappers that can be used from Android.

Another option we discussed is to upload attachments to Google drive using the Google Drive API. This is an active API and would allow for arbitrary files (not just images) to be uploaded. This seems more promising.

[shobhitagarwal1612]

I have created a folder odk-uploads in case the folder doesn't exists, as suggested by @lognaturel to upload the photos with required access permissions. It's working perfectly. Now since the drive v3 api is used. So, the support for code written to Get Blank Forms from google drive is also removed. Now working on that part of this issue. I have also submitted a pull request so that the upload feature may be tested by you all.
I'll make a separate pull request for updating the drive api code for Get Blank Forms section. Thanks :smile:

[lognaturel]

That's great, @shobhitagarwal1612!

Does anyone have additional opinions on the placement of the photos? Is having them all in odk-uploads as a flat directory ok? Another option would be to have folders for each form by ID within that odk-uploads folder. With the old Picasa code, albums for each jrformID used to be created. The advantage is that it does group images somewhat more sensibly if a user looks at them.

Another possible challenge is that I think users may be more likely to look at their drive than they were to look at their Google Photos albums. Should we do anything to encourage them not to delete the folder? One thing we could do is drop a README document in odk-uploads to explain what the folder is. @yanokwa @nribeka

[shobhitagarwal1612]

Should the media type folder also be considered? For now we are only uploading the images. But since the scope has been broadened by using drive-api, so in future we can also upload sound, video, etc. So keeping that in mind, should the directory structure also include media type along with the jrformId?
@lognaturel @yanokwa @nribeka Any thoughts?

[lognaturel]

Good point about enabling more media types, @shobhitagarwal1612. I don't think we necessarily need to separate them, though.

@yanokwa and I discussed a bit offline and would like to propose Open Data Kit/Submissions/jrformId/. That way we have a nice, clear, top-level folder that we could eventually put more things in. It matches the convention used by other Google tools that write to drive such as Google Photos. The submissions folder makes it clear what the contents is. And then organizing it by form id means a user can see related media (of any type).

[shobhitagarwal1612]

Sounds great!
I'll make the suggested changes and send a PR as soon as I'm done making these changes

[shobhitagarwal1612]

Done :+1:

[shobhitagarwal1612]

Fixed in #722

[yanokwa]

@calcahope You should see this fix in the next release of ODK Collect (~March 25th). Thanks so much for reporting the issue and for your patience!