Android: Autoupload implementation doesn't sort older images into correct directories.

[jasonbayton]

From #630

What I have noticed is it's not respecting the date of the whatsapp media. There are images from 03/2016 being uploaded to 02/2017 for example. Is it not set to interrogate image data on non-camera images?

Is this simply a case of it finding the images today (yesterday) for the first time and uploading them to the corresponding directory for today's date? Is it supposed to be able to pick up the correct year/month based on file name or meta?

[tobiasKaminsky]

Yes. Currently the upload date is the date which is used for subfolders.
But there is #451 which should solve this and use the creation date instead of the upload date.
So it is nothing with new new autoupload.

[mario]

You mean last_modified, not creation date, right @tobiasKaminsky ? :)

[tobiasKaminsky]

I am uncertain what the user would expect, if he takes an image on 31.1.2017, modifies it on 3.2.2017 (like cropping) and uploads it on 05.02.2017 via autoupload.
I personally would expect that is in january 2017 as it is the date where the image was taken.

[mario]

I don't think you can get creation date on Android, except from metadata, and again only JPGs will have that. So usually, last_modified date is used. (also, if auto upload is enabled on 5.2., images created/modified before that won't be uploaded).

[tobiasKaminsky]

Then, yes. This is also the way it is done in the PR on library https://github.com/nextcloud/android-library/pull/36

(in my scenario autoupload was enabled all the time, but only at 5.2. the file was uploaded via wlan; I know it is a strange scenario, but just wanted to create an artificial delay)

[mario]

@jasonbayton try the newest beta please.

[jasonbayton]

Oh do we have a possible implementation? 🥇
Downloading now.

First look:

• Keeps storage path in settings
• Any previously synced folders are deselected again
• Previously set custom folders are reset to default

Let me know what I should be seeing differently here (if anything)

[tobiasKaminsky]

@jasonbayton try the newest beta please

There is no new beta from 10.02 as the build was not working due to a wrong library merge, do not know who merged). So the latest ist 20170209, thus @jasonbayton you should not be able to see any difference ;-)

@mario do you have an idea why the settings are not stored / reset for @jasonbayton ?

[jasonbayton]

Indeed I didn't see anything different.

[mario]

What settings tobias?

On Fri, 10 Feb 2017 at 11:15, Tobias Kaminsky [e-mail]
wrote:

@jasonbayton https://github.com/jasonbayton try the newest beta please

There is no new beta from 10.02 as the build was not working due to a
wrong library merge, do not know who merged). So the latest ist 20170209,
thus @jasonbayton https://github.com/jasonbayton you should not be able
to see any difference ;-)

@mario https://github.com/mario do you have an idea why the settings
are not stored / reset for @jasonbayton https://github.com/jasonbayton ?

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/nextcloud/android/issues/631#issuecomment-278906944,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAWspjwBqW11p-dPtM86tBj2tP5GOHPks5rbDjRgaJpZM4L5eSx
.

[tobiasKaminsky]

In https://github.com/nextcloud/android/issues/631#issuecomment-278899808 @jasonbayton wrote:

Any previously synced folders are deselected again
Previously set custom folders are reset to default

[mario]

Yes, I think I fixed that bug :)

On Fri, Feb 10, 2017 at 11:34 AM, Tobias Kaminsky [e-mail]
wrote:

In #631 (comment)
https://github.com/nextcloud/android/issues/631#issuecomment-278899808
@jasonbayton https://github.com/jasonbayton wrote:

Any previously synced folders are deselected again
Previously set custom folders are reset to default

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nextcloud/android/issues/631#issuecomment-278910935,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAWsjFkaHau8aRbxFNDXIvBmXfl_dqqks5rbD04gaJpZM4L5eSx
.

[jasonbayton]

Sorry @mario @tobiasKaminsky I have a habit of editing and naturally these aren't re-sent to inboxes :)

[mario]

Ah, so custom locations indeed reset? :-/

On Fri, Feb 10, 2017 at 12:07 PM, Jason Bayton [e-mail]
wrote:

Sorry @mario https://github.com/mario @tobiasKaminsky
https://github.com/tobiasKaminsky I have a habit of editing and
naturally these aren't resent to inboxes :)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nextcloud/android/issues/631#issuecomment-278917605,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAWsqjbD0xuAgo9xQFQi7s2EW8ViABLks5rbEUOgaJpZM4L5eSx
.

[mario]

Also, do images upload for deselected folders?

On Fri, Feb 10, 2017 at 12:19 PM, Mario Đanić [e-mail] wrote:

Ah, so custom locations indeed reset? :-/

On Fri, Feb 10, 2017 at 12:07 PM, Jason Bayton [e-mail]
wrote:

Sorry @mario https://github.com/mario @tobiasKaminsky
https://github.com/tobiasKaminsky I have a habit of editing and
naturally these aren't resent to inboxes :)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nextcloud/android/issues/631#issuecomment-278917605,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAWsqjbD0xuAgo9xQFQi7s2EW8ViABLks5rbEUOgaJpZM4L5eSx
.

[jasonbayton]

Ah, so custom locations indeed reset? :-/

On every update recently, yup.

Also, do images upload for deselected folders?

No, I haven't seen that as yet.

[mario]

Did you, by any chance, remove the previous beta before installing the new one?
Or do you just install the new one over the old one?

[jasonbayton]

I generally only ever update directly over the previous version. That shouldn't cause issues though.

[mario]

The implementation uses preferences entry in order to check if it should clean "old" auto upload entries. If you remove the app, that preferences entry disappears too, so it would clean again.

But this doesn't look like to be the case with you :)

[mario]

OK, I just saw that my changes regarding this (and duplicate uploads) didn't go in the build. Should be there tomorrow.

[jasonbayton]

I'll pick it up in the morning, thank you :)

[tobiasKaminsky]

Beta is now up to date: https://github.com/nextcloud/android/blob/beta/apks/latest.apk

[jasonbayton]

I've had it installed most of today.

• Backup folders remained selected
• App path has remained
• No wild uploads so far

Is there anything else I need to look out for?

[mario]

upload based on date?
On Sat, 11 Feb 2017 at 18:03, Jason Bayton [e-mail] wrote:

I've had it installed most of today.

• Backup folders remained selected
• App path has remained
• No wild uploads so far

Is there anything else I need to look out for?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nextcloud/android/issues/631#issuecomment-279159861,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAWsrXDVrWNycQwm8FkOzbrsxhlOTGmks5rbentgaJpZM4L5eSx
.

[jasonbayton]

How does it work, scan and upload a folder and all its contents or just new files?

[tobiasKaminsky]

Upload based on date means that not the current upload date is used but the date when the file was last modified, e.g. you take an image on 31.01. and it gets uploaded on 01.02. Prior to the latest change it would be uploaded to 2017/02 as this was the upload date, but now it will get uploaded to 2017/01 as this is the creation date of the image (assuming you have enabled "use subfolder").

[tobiasKaminsky]

Rescanning an upload folder and upload the files that are missing is another feature which is wished, but not yet implemented.

[jasonbayton]

Perfect, just so I know what to expect. This sounds perfect :)

[mario]

I can do this easily, but that is one way sync - do we want that right niw?
On Sat, 11 Feb 2017 at 18:52, Tobias Kaminsky [e-mail]
wrote:

Rescanning an upload folder and upload the files that are missing is
another feature which is wished, but not yet implemented.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nextcloud/android/issues/631#issuecomment-279163317,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAWsmgH9ZVnoBEA-aObcYj8Ocwc55x_ks5rbfVogaJpZM4L5eSx
.

[tobiasKaminsky]

I vote to keep the autoUpload as simple as possible and implement a real 2way sync soon, which then can also be limited by the user to a 1way sync (at least how I imagine it ;-) )

[jasonbayton]

I'm ok with that also.

[jasonbayton]

OK small update - it just tried to upload my entire watched folder, again, to the 2017/02 directory on the server. Luckily another recurring issue "App was terminated" stopped it.

So

• it's still trying to push everything to this month's folder [1]
• it's still trying to upload duplicates of files that exist [1]
• whenever a new single file drops into the watched folder, it attempts to upload it multiple times [2]

[1]

[2]

It's from a few days ago, as that's the current, up-to-date output when I click on logs in settings.

Edit, it's going mad again:

[mario]

You're free to delete the logs now - I have them. Gotta see what's happening, as I can't reproduce this with my branch atm :((

[mario]

Also, @tobiasKaminsky this sounds strange - didn't the timestamp fix fix the uploading to proper folders issue? Seems like not...

[jasonbayton]

If I can force it to generate logs I'll happily supply more.

[tobiasKaminsky]

@jasonbayton it just tried to upload my entire watched folder

How do you do this?
The timestamp fix is not for auto upload :/ I had it wrongly remembered. This fix "just" sets the upload date on the server to the last modification date and not to the current one.
Respecting and using the last modification date for "use subfolder" is something new :/
Sorry, my fault, I had a knot in my brain.
@mario I can develop this tomorrow if you like.

[mario]

Sure @tobiasKaminsky, but please do it in this branch.

[jasonbayton]

All I've done is set a directory to be uploaded via auto upload and chosen a custom directory. Everything else happening it's doing by itself.

[tobiasKaminsky]

All I've done is set a directory to be uploaded via auto upload and chosen a custom directory. Everything else happening it's doing by itself.

But this does not led to uploading already uploaded files, or? (maybe I am just a little bit confused)

[tobiasKaminsky]

Sure @tobiasKaminsky, but please do it in this branch.

@mario ok, I'll do it :)

[mario]

No it does not
On Sun, 12 Feb 2017 at 13:58, Tobias Kaminsky [e-mail]
wrote:

All I've done is set a directory to be uploaded via auto upload and chosen
a custom directory. Everything else happening it's doing by itself.

But this does not led to uploading already uploaded files, or? (maybe I am
just a little bit confused)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/nextcloud/android/issues/631#issuecomment-279216965,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAWssiSDV2--pPUZutRbY1SY-wAzG1Mks5rbwHugaJpZM4L5eSx
.

[tobiasKaminsky]

But on https://github.com/nextcloud/android/issues/631#issuecomment-279209877 it seems as if there lots of lots of old images trying to be uploaded? (that is my confusion)

[mario]

That issue is unrelated. Please ignore and make sure uploads happen to the directory based on date modified. :)

[jasonbayton]

Oops, I've crossed issues.
The other one should probably be reopened.

[jasonbayton]

I'd say this _basically_ works now. Perhaps with the build today it'll stop confusing some images with the months they should be organised into, but for the most part it's working. I'll test the new build when it drops

[mario]

@jasonbayton if I understood our auto-build system properly, this should already be there so please try it :)