Server: WebDav file locking to prevent overwrites

[watermark]

It would be really nice to have true file locking in NC, such that opening a file prevents editing/opening by a second user until closed. I'm aware this has been discussed pretty heavily in the OC github, but I was hoping the opinion may be different in NC.

Our main use would be through webdav; however, it would probably be necessary to honor the lock in the web interface and possibly also the sync client.

This issue has been brought up (independently) time and time again by our customers, so it's a much sought after feature from our client base.

Steps to reproduce

1. User 1 opens FileA in WebDav
2. User 2 opens FileA in WebDav
3. User 1 edits FileA and saves (which updates through WebDav)
4. User 2 edits FileA and saves (which updates through WebDav)
   

   Expected behaviour

User2 shouldn't be able to open and/or save their changes because User1 already had FileA open

Actual behaviour

The changes that User2 made overwrites the changes that User1 made. This is done without any notice/message/warning to User2 or User1. Neither are even aware that changes were overwritten.

[putt1ck]

also interested in some solution around this, but client would be happy for it to be a plugin/manual checkout method like https://apps.owncloud.com/content/show.php/Workin2gether+(Lock+files+in+WebUI)?content=164016

[klm46]

any news?

this is really an important feature for business-using. there is actually npo chance to edit a pool ogf files by a pool of people at the moment.

can you please say somethink about plans for that? no milestone yet?

[Kinamo]

Quite frankly I think the idea got ditched because of the integration with Collabora Online... Although it seems a bit overkill to set up a Collabora Online environment just for this purpose... :)

[putt1ck]

Well, if installing Collabora Online means we get sync client file locks (and web client) we'd install it even if the client would never use it...

[klm46]

the problem is, that then the xls and co will beopen by libreoffice. this is no possibility for me. any news?

[grayum]

Recently migrated from ownCloud 7 (file-locking worked fine) to a new server with ownCloud 9.1.4. File-locking doesn't work any longer but I decided to upgraded to Nextcloud with the Nextcloud installer to the latest version (12). Very easy to do but the file-locking still broken.

This is a must have "feature" like @klm46 posted on 10 April for businesses and a big deal breaker for continued use of Nextcloud for us.

[putt1ck]

The Workin2gether app is now released for NC12: https://apps.nextcloud.com/apps/workin2gether - might be just what you need

[grayum]

@putt1ck does that work for webdav though?

[putt1ck]

@grayum not to my knowledge, but it also doesn't work for email attachments, USB sticks, Dropbox, WeTransfer, GDrive and all the other ways people move files around when we wish they wouldn't.

Locks via webdav (et al) also won't work if someone is editing a document offline - some things are best dealt with through policies and working practices. One client introducing NC has introduced policy of "sync only things you might work on near time or only want for reference" and then mandate users collaborating on files to visit the web interface to lock files with the aforementioned app before they start to edit them.

[grayum]

@putt1ck I think we're getting a bit off topic. In my case (and probably most of the participants here) the customer uses webdav exclusively (via a networkmap). So files are only available online.
Then if two people open the same file the first one gets no prompt and can r/w but the second gets a prompt that the file is only available for r/o. That's what ownCloud 7 did. That's the functionality my customers currently lack in NC 12.

The app you suggest involves users to use the webinterface of NC/OC and while this might work it's not very user friendly. And unrelated to the original post/request by @watermark. I do thank you for the suggestion though.

[klm46]

this should get a bit more priority

[marcinkaleta]

Hi guys,

we are also waiting for this correction.

[klm46]

hey guys,
couldn't you say anything regarding this issue?

[alexanderdd]

Shouldn't this be easy to fix? Just allow syncing of the .~lock.documentname.odt files, and we are done!

Or is there a way to manually allow for syncing of the .~lock files?

Btw, I understand the problem that users can work on their files offline, and in this case a lock would have no effect. In practice, people are online, so in most of the cases the lock will be effective. If they worked offline and later sync, the file should be uploaded with an corresponding name, e.g. filename-conflicted_copy_2017-05-03--12-34.odt

Thank you.

[christianpoortvliet]

Still a very-much-wanted feature. Any news on this?

[hieronymousch]

Same here, also customers who need this feature to avoid overwrites. MAnual locking via the webinterface is not a realistic solution IMHO. Bit of a shame that this isn't implemented, want to avoid having to install a samba access just for thus ;(

[idmacdonald]

The Workin2gether app does sound like a bit of a workaround, frankly. Most of our users use the NextCloud sync client or WebDAV to access their files. One organisation has been querying options file locking for a while, at this point we have to tell them that it's not possible.

Collabora Office is good, but it is a lot of code for the browser to execute and in our experience it doesn't work so well with really big spreadsheets or word processing documents. I see the lack of WebDAV file locking support as a current major limitation and missing feature.

My employer would probably be happy to collaborate in writing and testing code to implement this feature, though it would probably make sense for someone more familiar with the NextCloud code base to work on it.

[putt1ck]

How do you file lock something you're not connected to?

[idmacdonald]

Hi @putt1ck, if a person is using the NextCloud sync client and is offline, then of course they will just be opening up a file on their local computer and there will be no way to lock that file via the server. But quite a lot of file editing, in my experience, happens via the sync client with an internet connection.
But a lot of our NextCloud users don't even use the sync client, they connect directly via WebDAV clients to their NextCloud files. So they are always connecting to the server in order to open or edit files. WebDAV as a protocol has support for file locking, but WebDAV file locking is not implemented or supported by NextCloud, and so any attempts at file locking by WebDAV clients are either ignored or produce errors.

[putt1ck]

I can only see one scenario (people all accessing files via webdav) where automated file locking will add value - but in all other scenarios it doesn't add value and potentially lulls people into a false sense of security "this file isn't auto-locked so must be ok to edit".

Where automated file locking is a useful feature, in a legacy "shared drive" scenario, it is because that is how people access files for editing. It is already breaking down because of people working around the limitations of shared drives, by using dropbox, email, Gdrive, to grab a copy of a file to work on it at home, on the train etc., and then having version conflicts when they bring it back inside the LAN. A document management system amplifies this by making it easier to have a local copy; In some DMSes, they differentiate between a download and a "check out", where the latter results in the file being locked; but this tends to breakdown anyway, because the download process is easier (doesn't require providing a reason) and is a more immediately obvious way to get the file.

Right now the only broadly useful tech solution to collaboration I can see is something that can be set remotely to communicate to other users that a file is being edited. Automated file locking can only work for those opening files over webdav connections; every other access method requires a conscious "lock" trigger. Of course in teams with great communication a tech solution isn't needed...

[alexanderdd]

Hey everyone, sorry I have asked this before but did not receive a response, I'll try again:

Is there a way to manually allow for syncing of the .~lock files?

...cause that would be a work-around that would help my use-case a lot.

@putt1ck

Of course in teams with great communication a tech solution isn't needed...

I think what makes an application great is when even huge teams with mediocre communication can use it without messing their files up. I believe version conflicts are still easier to deal with (if conflicting versions are saved alongside each other) than the current situation, where person A opens a file, person B opens the same file, A saves, then B saves, but only B's version remains. All of A's work is lost, and neither A nor B get notified of this problem. (I hope I got this right?!)

[putt1ck]

If using sync, and person A opens a file, person B opens the same file, A saves, then B saves, B gets a warning and A's version doesn't get overwritten. If using web UI, it does get overwritten, but A's version would be in the version history; if using webdav access I think that would still be true.
OTOH if using traditional "shared drive" systems and B has taken a copy of the file home to work on, the overwrite happens and all evidence of it is lost.

If you want to sync lock files, then (depending on the variety) you'll need to do one or more of the following:
in the sync client, go to the General Tab, click Edit Ignored Files, enable "sync hidden files";
on the local machine, you'll need to edit the system's "sync-exclude.lst" and modify as needed for the applications in use (where the prefix of a "]" means "Allow Deletion".
It should be possible to distribute a suitably modified sync-exclude.lst, but this would either have to via central management (group policies, salt, etc.) or the end user have admin rights. No idea where the file would be in Windows, but in Linux it's in /etc/Nextcloud/).

[e-alfred]

@idmacdonald If you can implement it, then I think the best way to go forward is to do it by yourself. You could either make a PR to this repository or create an app for that. An app is more flexible because you can publish them rather freely on your own schedule whereas in core you would be bound to the release schedule of Nextcloud (and the uncertainty whether your PR is accepted).

[jospoortvliet]

Look, guys, if you have customers who want this - why not make it happen? Contact our sales team. Become a reseller. Sell the customer a support contract with consulting for this feature and it is done... Or built it yourself. @putt1ck made this work for his customer, just like we make things work for our customers. Why not do the same?

[BloodyIron]

Yeah, file locking is something that is needed for some things I have in the pipe too. What's going on here? Are we going to let this issue rot like it has the last year+, close it, or actually talk about what we can do about this? Because really, no response since...mid last year? really isn't cool.

Really need input as to what the NC devs think on this one!

[christianpoortvliet]

Bit of the chicken and egg story: if it is not in my customers aren't going to buy it. So i can't sell them a product with the possible but uncertain feature that might be implemented.

[putt1ck]

As previously noted, file locking of the type described doesn't fit well with a multi-access file collaboration system - "windows shared drive" mentality fits only with that sort of access, not people who sync the files and work offline, or open them from the web interface; and if you have a scenario where people only work as if on a LAN, why would they be interested in a system that allows access to files in multiple ways.
If you really, really want NC but on the basis of only providing shared drive access to the files, don't have any NC file storage. Have an SMB server which people map drives to, then mount that in NC as external storage.

[christianpoortvliet]

1. File locking on "files only residing in nextcloud"
2. File locking on "files residing in shared storage, such as SMB share Windows"

Have both scenarios (all clients use integration with SMB share, Windows file server". Anyway it is very obvious you need to be online to lock a file. Then you can go offline change the file and upload and unlock. However this functionality should reside within the sync client for both option 1 and 2.

Option 2 is getting complicated when people access the file directly over their shared drive (not making use of Nextcloud). Guess the file lock won't hold then.

To get a good understanding of what i mean: check the check-in and check-out function on Microsoft Sharepoint, i still maintain a complete (free) Sharepoint 2013 Foundation server for files where you can check out and check in.

Before you step on a plane you check out the file (can be done straight from Excel or Word), and when you get off the plane you check in again.

https://support.office.com/en-us/article/top-questions-about-check-out-check-in-and-versions-7e941339-e972-4c7a-a79a-80a1fcf84076

[putt1ck]

That's what the file lock apps (Workin2Gether and W2G2) do; allow a user to indicate that a file is being edited. Equivalent of "check out" and "check in" with just a single click (and the option of using comments to add info about the editing work)

[christianpoortvliet]

Within MAC/Windows Desktop app?

[putt1ck]

No, they are apps so through the web interface - well, until such time as the desktop client allows plugins, or there's enough interest in us providing a tweaked version of the desktop client that supports the locking (it's just a flag in a table, so...). You would already be able to see that a file has been locked, as it registers as activity for a file.

[BloodyIron]

Checking files out/in might be a good start for such things. I totally see the challenges with offline/online jumping for locks. Are there other ways, conceptually, this functional need could be addressed? Whether it's the traditional methods or not.

[putt1ck]

The locking apps offer an equivalent of check-in/check-out, but necessarily this is via the web interface as we don't know a way to extend the desktop client, nor a way to notify any given piece of desktop/mobile software - feel free to post an enhancement request on https://github.com/newroco/W2G2/issues :)

[BloodyIron]

I'm thinking the desktop sync client should have a way to lock/checkinout files, not just the web interface. This really is one of the most notable differences between NC and SeaFile, like a painpoint for certain clients.

Does anyone have any idea what it would take to have the desktop client also be able to do this?

[putt1ck]

I've opened a feature request on the client github https://github.com/nextcloud/client/issues/217

[BloodyIron]

Hopefully we can get this all sorted! :)

[jospoortvliet]

@BloodyIron

Does anyone have any idea what it would take to have the desktop client also be able to do this?

From what I know and the discussions I've had with engineers, this is a multi-(wo)man-many-month job, and more complicated than it seems at first sight, in part due to the semantics you need for this to work smoothly (you need to figure out hard questions about when to lock and unlock for example) and to make it work on all platforms and situations.

So I suspect it needs a team of our engineers from server and all clients to get together at one of our hackweeks or the Nextcloud Conference and whiteboard this, make a draft implementation (probably looking closely at the W2G2 app) and then build a final implementation. It would probably take 1-2 release cycles of the server and clients. You can imagine we'll only do this, as a company, if we're utterly bored or see strong customer demand. I see neither of these situations at the moment :smile_cat:

Of course, a group of volunteers could get together and do this - but I have rarely seen such big, cross-platform, complicated and involved projects being done from beginning to end by volunteers and if it has to be done in the evenings count on this taking a year or more for half a dozen people to do it so well that it can be merged in Nextcloud. Of course, a rough prototype can be done faster but that won't be good enough for a serious company to rely on.

[putt1ck]

@jospoortvliet the problem with the "customer demand" requirement is the people who need locking never become customers; they evaluate the available products based on a feature list including the locking requirement, and NC wouldn't make the shortlist.

Now, if NC were to make the sync client support plugins like FF et al, someone else could work out the hard questions about the locks (many/all of which have already been asked in relation to the app).

[BloodyIron]

Considering this very github issue was opened in 2016, and has been revisited repeatedly since then, I'm not sure how it's concluded that the feature is not a desired feature (by free/paid client base). Plenty of other features get baked into NC all the time that don't require "strong customer demand" (which actually is what people are trying to convey in this, and other github issues).

To say it succinctly, there is strong customer demand. Many people have asked for it, and even stated they went with SeaFile (or others) instead because of a lack of file locking. How is that not sufficient visibility of demand?

[biva]

The issue I opened (https://github.com/nextcloud/client/issues/83) is very similar, even if it's a slighlty different need: locking is not what I need in priority, I would already be happy if I "know" that somebody is editing a file. And I could imagine these steps: (Client = Desktop Mirall Client):

1. Wedding.xls is shared by UserA and UserB
2. UserA opens Wedding.xls
3. ~$Wedding.xls is created by Excel in the same UserA local folder
4. ClientA detects ~$Wedding.xls (nota: it doesn't sync it, as it's in the exclude list)
5. ClientA sends the information to the server that UserA is modifying Wedding.xls
6. UserB opens Wedding.xls
7. ~$Wedding.xls is created by Excel in the same folder
8. ClientB detects the creation of ~$Wedding.xls in UserB local folder
9. ClientB sends the information to the server that UserB is modifying Wedding.xls
10. The server knows that UserA is already using it: it sends this information to ClientB
11. ClientB displays a button* (similar to the Dropbox badge) in front of MS Word, informing UserB that Wedding.xls is currently being modified by another user
12. UserB decides what to do: he can call UserA (using NC Talk ;) ) to ask him to close it, or continue editing the document (that could lead to the creation of a conflict version)

Would it be a good solution for you also? Without locking feature, would it need 1 year working time to develop this?

[jospoortvliet]

@putt1ck the chicken-egg you describe exists, of course, and picking a feature you think the market needs is always a little bit of a gamble. We do that, for sure, as I mentioned in another place where we discussed this, we make strategic decisions based on needs from customers we are told about or needs we see coming in the future. And I personally think this is a thing we should look into more, even though I must admit I never met a single (potential) customer at any event who ever asked about it or brought it up - and they bring up lots of stuff.

Sales never told me about it either, so (almost?) nobody has shown interest to them. People show interest in other things, so that isn't because they don't see Nc as a viable solution just because it doesn't have this feature described somewhere... So I have nothing to argue for this feature when we meet in a week or two to discuss Nextcloud 14 priorities: I can say I have a hunch and 5 people on the forum who want a feature but that means it falls of the white board compared to every other priority we have.

Wrt plugins, if people would be developing patches or separate versions of the client for specific purposes, we could take that as evidence for a need for a plugin structure. I haven't seen that and a plugin structure doesn't take away the need to write code. We barely get any contributions to the client from outside so there doesn't seem to be anyone who wants to write C++ client code - plugin or not. Again, putting in a lot of work (depending on what the plugin should be able to do this could mean rewriting half the client, a multi-year effort) so maybe maybe one person writes a plugin seems not a great idea.

@biva in theory, that could be developed, and it would not take that long, because it is only useful in one very limited case: windows+Microsoft Office users. You might say "but that solves our use case" and you're not wrong. However, the problem is: if we do this, people will expect it to always work. Also if they open an image in Gimp on Linux. Unless we develop a solution that works for every situation we are unlikely to implement this in our mainstream product and advertise it as a feature: it creates confusion. People will expect more and be disappointed when it does less. You and I and the techies here probably understand that it was only implemented for MS Office because that was doable - but a average user won't understand why his Photoshop file now has a conflict and he/she didn't get warned.

It IS possible, on every platform for every app there are ways to track if a file is opened. But then you're back to the "1 year working on this"...

If a customer comes and pays us to develop the scenario you describe, we'd do it, of course - and it wouldn't take too much time. But we would probably NOT integrate it in the public version of Nextcloud, unless it works very well somehow or, of course, we see demand and decide to make a 'full' implementation.

[biva]

@jospoortvliet thank you sincerly for your clear and educational information: we, users, also need to understand your priorities and your way of taking decisions.

I also understand your fear about the confusion it could create. However, I think it's overestimated. First, everybody (almost) understands that you can edit a document within Nextcloud web interface (with Collabora), and not a picture. No confusion. But most importantly, I think that a very large majority of usecase do affect MS Office or Libreoffice documents. Don't you think?

I do agree that it should be available on all platforms, and on other softwares. I think MS Office and LibreOffice are a very good start, covering the main usecases. Both create a file which is easily recognizable (~$Wedding.xls for MS Office and .~lock.Wedding.ods# for LibreOffice).

Regarding other platforms, I don't have any Linux nor MacOS available, but I imagine that it works in a similar way: are there Linux and Mac users around to confirm this?

[BloodyIron]

@jospoortvliet I literally have a client that is going with SeaFile instead of NC because of the file locking. It would be very much appreciated if you could bring it up at the NC event, as I myself don't have the means to attend such events at this time.

[biva]

New insight from a friend of mine in Paris, system admin for several companies: they don't go Nextcloud only because of the lack of file locking (or similar feature).

[putt1ck]

@biva There is a file locking app. Just not one like local shared drives (because you can't sustain it in a shared collaborative work environment).

[biva]

@putt1ck Do you speak about https://github.com/newroco/W2G2 ? As far as I understand, it locks file in the Web UI, not on the desktop. That's not what I'm looking for.

[putt1ck]

Yes, I mean w2G2. Describe to me how locking on the desktop works in a system designed to support "off LAN" users and we'll add it to our list of "apps we want to build". If someone is considering NextCloud as an essentially LAN-focused setup, there's an easy workaround vis a vis file locking for the LAN users.

[biva]

Thank you for your interest @putt1ck
I'm not a dev, but my idea was this one: https://github.com/nextcloud/client/issues/83

[putt1ck]

I'm not a dev either...

The idea of locking feature that only works with one product doesn't fly as something NC core (or our team) would work on. As described and implemented by Dropbox it has potential to cause user confusion in any org that doesn't only use that desktop software (for example graphics, CAD, GIS, audio/video).

The feature as described by DropBox (pay for version) also seems no more user-friendly that pressing a button to say you're editing (and then it encompasses people opening just to view). What happens if someone is editing off-line? Or start editing when they are on line and then go offline before they close the document? Or with the MS Office feature, what happens if someone collaborating on the doc isn't using MS Office?

What if it was just a button you had to press, like on W2G2 in the web UI, but also available on the sync client? That I could see working.

[biva]

• "Works with one product": I agree, but I think it's a good first step covering most of the usecase (see https://github.com/nextcloud/server/issues/1308#issuecomment-377191589)
• I am using the Dropbox badge professionaly: it is automatic, it does the job, it is user-friendly in our experience, and we like it.
• Offline: I agree, but there's no easy solution for this: conflict management afterwards will happen anyway in this case
• Pressing a button: I understand the idea, but I don't believe in it. Users will forget to do it. It should be automatic.

[BloodyIron]

@putt1ck there are technical nuances to figure out for sure, but there are more and more clients that are passing up NC while we sit here and just say it won't be done. When is enough... enough?

Not only that, how about we flip this on it's head. Don't we want NC to be the BEST file sharing tool we can make it? It's clear people want this feature, and not having it prevents it from being the best.

[biva]

For the record, here is the Seafile feature: "File locking: Seafile supports file locking to prevent concurrent editing of files and generating of conflicts files. Users can lock files in web UI or desktop clients. Office files are automatically locked when they’re opened."

We're not that far away from having this in Nextcloud:

• lock files in web UI: W2G2
• lock files in desktop client: idea of @putt1ck to extend W2G2 feature
• automatic lock of Office files: Badge or similar feature

[jocarren]

@putt1ck you mentioned a workaround, can you please elaborate on that or share a link about it?

btw, I've been struggling with this a lot, this is what I've tried so far:

• Mount user folder as webDAV share in Windows: it still allow 2 users to edit the same file (no locking).
• Disable sync ignore for ~$* files: client sync said files, but MS Office ignores the lock when opening in any device other than where the original ~$ file was created.
• Mount NC user folder via webDAV in a different unix FOLDER in server, then share FOLDER via samba and mount on client device. Locking works but unix permissions make it such a mess that there is no point in using NC for such "application".

edit: I also acceded NC via webDAV using other sync clients such as Goodsync and Cyberduck (and many others), but couldn't find any that supports locks.

[biva]

@jocarren afaik, the workaround is this one: https://github.com/nextcloud/client/issues/83#issuecomment-377030532
And it works only for Libreoffice

[BloodyIron]

@jocarren my vote is that we get file locking through the desktop sync client (nextCloud preferably, not the ownCloud one). I don't use WebDAV myself, but since it seems you do, I can understand why you care for file locking for that.

I think we should get file locking regardless of the protocol used, but perhaps to start, have it enforced through the desktop sync client.

[jospoortvliet]

@jospoortvliet I literally have a client that is going with SeaFile instead of NC because of the file locking.
New insight from a friend of mine in Paris, system admin for several companies: they don't go Nextcloud only because of the lack of file locking (or similar feature).

Look, it isn't so complicated:

• home users don't need this
• we're not so excited about the needs of companies who don't want to pay us. Do you guys work for free for your boss or customer? Right.

@putt1ck I appreciate the work you do to keep W2G2 working! And all of you are, of course, welcome to contribute these features.

If any of these companies who is interested is a serious enterprise (eg 1K users or more), tell them to ask us, or, heck, become a partner, resell them a support contract. If we get a few asking for this, we can develop it.

I'll close this now for discussion, we're not adding much new info.

[putt1ck]

@jocarren the workaround in question assumes that the use case for Nextcloud is just an alternative to Dropbox or WeTransfer etc. i.e. a way for people to access files from home etc..
In this use case, continue to use SMB for old-school shared drives, and then make the SMB shares available as "external storage" using "Log-in credentials" or "User entered" for authentication. This requires SMB client to be installed on Linux NC hosts but is otherwise easy enough to maintain.

[MorrisJobke]

See also discussion in #9969

[aignerat]

Hey,

I think an important first step would be to inform the user, that another user is editing the same file at the moment.

[skjnldsv]

@rullzer @daita this is fixed with 18, right?

[rullzer]

Well there is the locking app. So that covers the basics yes

[putt1ck]

There's been a locking app for a while e.g. W2G2. Not sure that nor the new one covers the request to have webdav locks.

[skjnldsv]

@putt1ck the new lock file app include proper webdav locking properties.
Webdav mount on debian reject stating the file is locked :)