Client: Allow multiple accounts to sync the same local folder

[icewind1991]

Currently when using the multi-account feature it's not possible to sync the same folder to multiple servers.

It would be nice to have the options to sync a folder to multiple servers.

[guruz]

@icewind1991 You mean sync the same local folder? The problem is we use exactly one .csync_journal.db per local folder and server..

[icewind1991]

You mean sync the same local folder

Yes

The problem is we use exactly one .csync_journal.db per local folder and server..

Yes, you would have to change the csync.db filename to be based on the server

[RalfWW]

I run this scenario like this under Win7:

folder-owncloud1
Symlink-to_folder_ABC >---|
|----> folder_ABC
folder-owncloud2 |
Symlink-to_folder_ABC >---|

Having two local folders synced to a dedicated owncloud server (owncloud1, owncloud2) each.
Adding a symbolicLink to the "shared" folder (folder_ABC).

The only drawback: changes within folder_ABC will not be detected by the client "in time". It can take a long time (up to one hour, I guess) until the client performs a "check over all" run to get notice about the changes.
Stop and start syncing at the client will start a "check over all" manually in case you are in a hurry :-)

regards

[gebhard73]

I've just created the workaround RalfWW mentioned - hopefully it will work :-)

Nevertheless I'd like to "vote" for this feature, too.

cheers,
Gebhard

[RalfWW]

The actual clients don't support symlinks anymore.
The upcoming release 2.1.1 might perform it again!?

I'm staying with Version 2.0.1, the last that worked with symlinks (directories only!)

Ralf

[gebhard73]

Ah - thanks for the feedback, I'll have a look into it next weekend latest.
Have you tried junctions?

Cheers,
Gebhard

[gebhard73]

Any information if 2.1.1. works with this workaround?

Thanks!

[RalfWW]

It seems that Version 2.2.2 is back with the old functionality to handle symlinked directories.

[gebhard73]

Thanks, seems indeed to work again ... :-)

[guruz]

@ckamm @dragotin We seem to have a conflict here with regards to https://github.com/owncloud/client/pull/5036
@gebhard73 and @RalfWW want a behaviour that we're disallowing (again).

[gebhard73]

My problem is: I _really_ need the symlink feature because it's a workaround for the missing server 2 server sync (https://github.com/owncloud/core/issues/1190). Perhaps it can be made configurable?

[dragotin]

@gebhard73 if you love adventures, you could go ahead and test http://download.owncloud.com/desktop/daily/ownCloud-2.3.0.6239-nightly20160710-setup.exe which includes the changes from the pull request mentioned above. Be careful please, as it builds on master and is completely untested (well, almost).

Please try if that solves what you intend to do. This client will create a journal file independent for each server it connects to. That is probably a start to solve the problem, if that is already enough, I do not know yet.

[gebhard73]

Thanks @dragotin for the hint, it sounds promising. Unfortunately I don't have a test server at hand so I think I have to wait a little bit before testing ... Looking forward for the new version of the client.

[ckamm]

@icewind1991 @gebhard73: @dragotin's pull request was now merged and one can now sync folders to multiple accounts. Please test the nightly builds! I am not quite sure if there are more issues here: Could I ask you to make new, separate tickets if there's something left that you need or doesn't work well?

[gebhard73]

Many thanks, this really sounds promising. Unfortunately I'm not able to test the nightly builds for various reasons right now. If this changes so that I can test I'll provide feedback as requested.

[guruz]

@gebhard73 FYI there is also the testpilotcloud builds which use a separate config directory so you don't have to mess with your current sync configuration.
http://download.owncloud.com/desktop/daily/

[phil-davis]

On Windows10. I first installed testpilotcloud-2.3.0.6553-nightly20161207-setup and lost all the status icons on folders/files, and no "Share with ownCloud" context menu entry. So I uninstalled testpilot and installed the real thing ownCloud-2.3.0.6552-nightly20161207-setup - still no icons or context menu entry. I rebooted and still no icons or context menu entry.
My existing real ordinary accounts are working fine.
I have connected a folder to 2 different test accounts. When I add/change a file it syncs up to both places on the server - good.
When I add a file to the server in Test1, the client pulls it down, and then syncs it back up to Test2 - good.

I was interested to see what would happen about the context menu entry "Share with ownCloud", because that will need to somehow have the options for which ownCloud account you want to share from.

Is the context menu supposed to be there still?

[phil-davis]

@ckamm or whoever is appropriate - are the status icons on folders/files known to be not working on Windows on the nightly builds? Or is this some "feature" of my machine?

[SamuAlfageme]

@phil-davis hm, just tried to reproduce this and the particular condition that seems to be triggering it is having multiple sync client processes (e.g. one for OwnCloud itself and one for the testpilot edition) just the first to run is getting the overlay icons and the context menu entry. When you kill (or quit) the first, the second gets all the attention:

I'm moving this to #5377, as this condition seems not to be directly related to the original reason of this one. Let's follow up there. Thanks for reporting 😉

[phil-davis]

My test scenario is:
1) Add new account to client that syncs user Test1 on server to local folder "test1and2".
2) Add another new account to client that syncs user Test2 on server to the same local folder "test1and2".
Files sync up/down to both places/servers - all good.
Overlay icons and context menu entries are there - good.
3) Right click a sub-folder in "test1and2" and "Share with ownCloud" - select to make a share link.
a) There is nothing to indicate which account@server the share will come from (Test1 or Test2).
4) Make use of the link - it is a link to a folder in Test1.
5) Remove account Test1 from the client.
b) Overlay icons disappear for the files/folders in "test1and2"
6) Right click a sub-folder in "test1and2" and "Share with ownCloud" - select to make a share link.
Note: step 6 works even though the overlay icons have disappeared.
7) Make use of the link - it is a link to a folder in Test2 - good, Test2 is the only account syncing this folder from the client.
8) Add (again) account to client that syncs user Test1 on server to local folder "test1and2".
c) Overlay icons reappear for folder "test1and2"
9) Right click a sub-folder in "test1and2" and "Share with ownCloud" - the share link shown is the link to files in Test1.

Summary:

• something reasonable does happen at each step.
• it would be nice if the overlay icons did not disappear at (b), but it is cosmetic.
• it would be good to display to the user what account@server the share will come from at (a), so they know exactly what they are sharing.
• even nicer would be if the user gets to choose which ownCloud they will share from - maybe the context menu should have 2 "Share with ownCloud from Test1@server", "Share with ownCloud from Test2@server", or when the context menu is clicked, the dialog could allow choice of the account@server to share from.

The good result is that nothing explodes, no files go missing, stuff copies around as I expected.

[SamuAlfageme]

This brings an evil idea to my mind, imagine the following:

• [email protected] shares a folder "SHARED" with account_B in that same server.
• Both account_A and account_B set their local folder sync connection to be local_folder
  
  
  
  • account_A maps the folder to SHARED
  
  
  • account_B do the same with SHARED/subfolder
  
  

This could lead to infinite recursive sync (synception), since any uploads to local_folder would replicate in the top folder and the subfolder, which then would trigger changes in the top folder again (as the child has changed) and so on...

---

btw, @phil-davis great test scenario!

[phil-davis]

Don't think so hard ;) I guess similar things can happen with Federated Shares between servers that make loops and/or have shares of sub-folders of shares that then loop back. I guess Federated Sharing on the server must have some loop-detection mechanism to prevent looped shares from being created, and also to prevent "death by vortex" if packets start spinning around trying to find the real home of a file.

If the server-side API has a way to query the real ultimate server that a folder resides on, then the client has some chance to check that and disallow multi-sync of the same local folder to multiple places that ultimately lead to the same real folder (or sub-folders of each other).

[guruz]

What's the state of this? Does it work with the latest nightlies?
Can we close this?

If there is small follow up issues we can remove the milestone from here if they don't block the release

[phil-davis]

I would say this works fine for ordinary use.
The followup issues are:
1) Overlay icons disappear when one of the accounts is removed - they should "switch over" to show the file/folder status with respect to the remaining account(s) that sync the folder tree. There might be a whole issue of the state to be shown when a file is being uploaded to multiple places (I guess it should show the "highest severity" state out of all the places the file is currently syncing to).
2) Context menu for sharing - add some way for the user to know and choose which server they are sharing from when they "Share with ownCloud".

[SamuAlfageme]

@guruz Oh yeah, the basic workflow does.

The whole concept needs some more work though (refer to @phil-davis's https://github.com/owncloud/client/issues/3764#issuecomment-266481913), that's why I kept this one open. We can move those concerns to a different issue to get closure here.

[gebhard73]

Thanks @phil-davis and @SamuAlfageme for testing, appreciate it. And again sorry that I'm not able to test at the moment...

[SamuAlfageme]

I was running some test to check migration strategy (if the old .csync_journal.db is renamed and its contents preserved) after updating the client in different platforms. So far, so good.

One thing that strikes me though is that the old temporary SQLite files (.shm and .wal) so they stay and are recreated for the new journal. I don't know if their contents are important; if not they should be erased.

[SamuAlfageme]

As pointed in https://github.com/owncloud/client/issues/3764#issuecomment-270632561 the Share with ownCloud is troublesome for shared folders. Couple of issues that should be addressed:

• [ ] Right now, only the first user that created the folder sync connection get to share files from the contextual menu. And you get SocketApi: Sending message: "SHARE:NOTSYNCED:/file.txt" until the new user's files that you are trying to share are uploaded to the first user account.
• [ ] An account picker should be implemented to choose which account should originate the share

[phil-davis]

And so the client needs to know which upstream servers have the file - e.g. if it has been just downloaded from account1 and is yet to finish uploading to account2 - so it can present only the valid accounts to share from in the account picker. (I guess the client should have all the meta-data locally in the sync db for each account to know this)

[guruz]

@SamuAlfageme Could you create new issues for this and close the issue here?