Syncthing: Local sync between directories

[jebeld17]

Local Directory Sync, in which the feature:

• Synchronizes two or more (2+) local directories
• Allows "hybrid synchronization," such as synchronizing the above two or more local directories with a single network sync folder
• Abides by the same rules, logic, and parameters currently used by Syncthing for network Folder syncs.

[calmh]

I'm not going to immediately speedclose this, but I think it's unlikely to happen. After thinking about it for five minutes I think it's more difficult than you'd expect. We're quite focused on syncing the same folder between different devices and there's a lot of internal state that depends on that assumption. Tying together two (or more!?) different folders on the same device doesn't make sense in that model.

[AudriusButkevicius]

We've closed a ton of these right off the bat, I don't see why this one should survive. Syncthing is for syncing remote devices. If you want to sync local devices, run multiple syncthings, or use the right tool (as syncthing is not one of them).

I want syncthing to wash my dishes, but who cares?

[calmh]

I think it's mostly been forum chatter. If there were previous issues we might still keep one around to collect the +1s and whatever.

Dishwashing is out of scope, however.

[ibizaman]

@AudriusButkevicius I'd love anything that can wash my dishes.

But seriously, @jebeld17 you might want to something like https://en.wikipedia.org/wiki/UnionFS

[alaw005]

I've got a use case where I have a 50GB data folder on my local SSD drive that I do not want to sync but I do want to sync the Documentation sub-folder with my main documentation folder. I want to sync because I also need to keep within the 50GB folder so that the documentation also sits alongside the data is applies to.

I don't want another tool just for this as I am already using syncthing and it seems a perfect option.

There is a workaround I am about to implement where I sync the Documentation folder withing the 50GB folder with another computer and sync that other computer folder back to my main Documentation folder.

[themoodymann]

Why not? I'd use Syncthing to have a backup on an external drive for some of my more important folders. Surely there are other tools for this, but I'd rather have Syncthing do this (plug external drive in from time to time, and then plug it out again after the job is done).

[AudriusButkevicius]

You have to understand that syncthing was written as a network application from the beginning, so everything evolves around connections, devices and so on, which makes no sense in a context of local directories, and any implementation of local sync is either a hack, or a monumental undertaking.

I can also make blanket wish statements that I'd rather have my text editor act as a browser, but some things just don't make sense.

[themoodymann]

Your metaphors may but funny, but they are not very convincing. TCP can handle local connections just as well as remote connections, whereas "washing dishes" is something else.

Allowing this feature is almost surely not more than changing of a few lines of code, i.e., just allowing to add the local ID as a new remote device (with localhost) instead of actively checking and preventing it. No need to document it even. I understand that this is a hack, and possibly somewhat against the philosophy of Syncthing. But it would be highly valuable for people like me. After all syncing for backup is a prominent special case of syncing.

[calmh]

Allowing this feature is almost surely not more than changing of a few lines of code

Oh, it very much is. Feel free to prove me wrong. :)

Even if it were that simple, running two instances of the folder and copying all data with encryption, decryption, compression etc is not something we'd be proud of when explaining how we do local sync. That way is already possible by running two Syncthing instances.

[canton7]

People always underestimate the cost of writing or changing software, despite what the people who actually wrote the software say... I've never worked out why.

[calmh]

https://github.com/syncthing/syncthing/wiki/NoPlusOne

[jebeld17]

I have a middle-ground solution: User profiles.
It may seem a bit far-fetched and distant from this idea, but having some sort of user profile management intertwined into Syncthing can allow simultaneous instances and deeper user-based customization and actions - such as syncing to poly-locations.

Just an idea.

[calmh]

Syncthing is lightweight enough that it's no great hardship to start a separate instance per user who needs to run it. Having built in user profiles would also necessitate handling ownership and permissions and stuff in a way that we currently don't (and don't want to).

[jebeld17]

Just an idea:)

[creeloper27]

You have to understand that syncthing was written as a network application from the beginning, so everything evolves around connections, devices and so on, which makes no sense in a context of local directories, and any implementation of local sync is either a hack, or a monumental undertaking.

I can also make blanket wish statements that I'd rather have my text editor act as a browser, but some things just don't make sense.

Actually Atom has a plugin that lets you turn your normal text editor/ide into a fully functional browser 😄

[AxisAssasin]

Just run 2 instances in a docker container. Job done

[saas786]

I am not sure about other users reasons to sync local directories.

But I have been doing it with various tools over the years, and been looking for something like rsync for it (specially on windows :() and stumbled upon your tool.

And was hopeful that it might solve my problem. But reading this thread disappoints me.

In my opinion this app by design should not assume that I only want to sync between two devices, its primary purpose is syncing the files / directories, which it should focus on.

And as an addon, there should be features allowing users to sync between devices, between local and remote servers (http, ftp, dropbox, amazon s3, whatever), or in this case on same device.

My primary use case is,

I work on source code of my project, like SCSS, JS (es6/es7 etc) and compile it to css/js using tools such as grunt, webpack or similar, and sometimes I test same code on multiple websites (on my local server), so I don't want to manually copy the files on two separate folders on my local machine. Most importantly I want this process to be automated and accurate.

So hopefully this is enough reason for you to reconsider your approach, and allow for local sync on same machine. As I am pretty sure there are hundreds (if not thousands) of users looking for something like this.

[AudriusButkevicius]

Please, just stop. The design was made probably 5-6 years, so telling what the design should have been is a bit pointless 6 years into the implementation.

If you support the feature use emoticons, telling us that you are a web developer, use grunt, test your code and that your cats name is Barry adds no value to this thread.

If you perceive the usecase to be different to what the original ticket talks, you should explain how exactly is different in the context of syncthing and not the context of your tools and the things you do.

There is nothing to reconsider here. We agree it's a feature people want, yet it does not fit into the current design, it's a lot of work to address that and someone who actually cares about this needs to step up and implement it.

If we didn't think it's a useful feature, this ticket would be closed.

[saas786]

I am not sure why the contributors of this app seems so uptight.

I don't think anybody is here to criticize you or force your hand to make a decision which ultimately is a lot of effort on your part, specially when your apps initial design was different then our needs, and it does well what its built for.

I just tried to emphasize the use case, where this tool can be v. helpful. Apologies if it somehow offended you or made you uncomfortable. I never meant to do that.

I never used this tool, and I think might not use it in future. So not sure if I can implement this feature myself. If I have used this tool for sometime and was fluent in GO, I would definitely have taken this initiative.

Anyway, if this is too much trouble for you guys, I think you should close this issue. As it seems like we can't convince you to reconsider your app base model.

[AudriusButkevicius]

Why should the issue be closed? We agree it's a valuable feature but nobody has the amount of time required (or care enought) to implement it.

[adiabatic]

I take it the people who want local sync have tried Unison?

[4world]

If my understanding is correct, the developers are writing this code for no compensation other than the trickle the donations may provide. And the public who gets this product free of cost (like me) should give their feature requests with a hint of gratitude and not as demands. Quite contrarily, there are several posts that are rude sounding. Changing even a couple of innocuous lines of code can sometimes be catastrophic, and that is why testing takes so much longer.

For those who claim a feature is easy to implement, why don't you jump in and do it so everyone benefits?

Another idea that may need some way to implement:
If a feature is desirable by so many people, how about setup a contribution pool specifically for it, which has a target amount depending on the difficulty of the task. When the target is met, the developers would implement it. Of course, more intricacies would need to be worked out, e.g. what happens if the pool never reaches the threshold or if it goes over, who manages the funds, when the developers get paid, bug fixes, etc...

It's a random idea and I know it's easier said than done

My use case is quite simple:
I want a one-way 'mirroring' from my SSD to an inexpensive HDD on the same PC, to cover against SSD failure. Don't want to implement RAID on the SSD.

A respectful thanks to all the developers. I am sure to at least send in my tiny donation because I want to support this initiative.

[4world]

Please feel free to delete this and my previous post as I understand less meaningful posts (and off-topic ones like this) cause unnecessary clutter, but I had to convey this:

I went to the donate site, and OMG, it made the idea of a contribution pool in my previous post look stupid because you guys are already soooo beautiful that you are not even keeping the donation money, and using even that for public benefit!! Double social service. My salutations to your high ethics, Antony.

I didn't donate because it that said people can donate to another charity in SyncThing's name, and I am at present working on a startup (using charitable funds from parents and self), which if successful, will go to charity, with only living expenses for me. Of course, I will put SyncThing and Antony Male on my list of contributors.

[canton7]

Bear in mind that Syncthing and SyncTrayzor have their own donation links. Syncthing's goes towards the infrastructure, mine goes to charity. Perhaps that should be more clear...

[tristanbrown]

I have the exact same use-case as @4world. On a Windows machine, I have an SSD that I want to sync to an internal HDD RAID 1 array for backups. I would like to use syncthing for this for two major reasons:

1. Consistency. Every different tool has its own syntax, structure, and quirks. I want certainly directories synced seamlessly both locally and remotely. I don't want versioning stored differently for different tools. I don't want different .*ignore files. I don't want to accidentally miss some gap between the two different systems and lose some files because of it.
2. Centralization. It would be nice to control all of my file synchronizations from a single interface. Same UI/syntax. Same console.

@AudriusButkevicius This isn't asking the vacuum to wash my dishes. This is asking my sink head to rotate so I can wash dishes in either bin of my sink, rather than having to install a completely separate plumbing system to do the exact same thing in a slightly different orientation.

I took a look at the Unison docs, and it's not at all clear that it would do what I want it to (backing up every day, with a user-defined amount of versioning, no compression or archive formats). I also looked at FreeFileSync, but it is problematic in that it versions by changing filenames. We need local sync functionality in Syncthing.

Syncthing is lightweight enough that it's no great hardship to start a separate instance per user who needs to run it.

How, exactly? I just tried (Windows 10), and it detected the other running syncthing instance and automatically exited.

[ekt-]

I have two use cases

1. On a (Synolgy) NAS I'd like to mirror folders between different volumes of the same NAS. You'd think this is a common thing on a NAS but quite surprisingly Synology does not support it. Sure I could use an rsync script, but I already have syncthing on this nas for other reasons, would be nice to use it

2. Sync a sub-selection of files to another local folder using ignore files. This is a big thing. There are literally petitions and years-long threads on DropBox, GDrive, OneDrive forums asking to implement something a la .gitignore. But no one does it (because they want to sell you more cloud, cynics might say).
   Syncthing is one of the few tools that implement it. I could have my 'c:\working' folder mirrored to my OneDrive local folder, and using an .stignore file I would be able to avoid sending temp files/temp folder/caches/you name it to the cloud.

That said, I really really understand devs on this. It hugely smells like feature-creep.
If it was me, I wouldn't do it. But if ever decide to implement it, I'll surely use it. And I won't care if it is greatly inefficient

[dmaj]

@tristanbrown:
I took a quick look at the code. The instance data is stored in the user's local appdata directory (LOCALAPPDATA). So the program can easily be started with a script that configures the environment variable "LOCALAPPDATA" to an individual folder. This makes it easy to start multiple instances. I have tried it briefly. At first sight it worked well.

edit:
Now I have also read the documentation. "-home" should also be an option.

edit2:
But for local copies I use "DirSync Pro". The tool is Open Source, even if it has the suffix "Pro". For a local copy (e.g. on USB stick) I don't know anything better. Above all it is incredibly fast, although it is written in Java.
https://dirsyncpro.org/index.html

[jeffreysmooth]

I have multiple External HDD attached to single system.I want to sync between these disk to take a backup. It would be really good if syncthing start supporting local sync.

Thanks.

[orenk1]

I'd like to sync between FTP and local storage.
I have mounted the FTP as a local storage. I am looking for a way to sync between local folders (one is mounted FTP, the other is HDD)
Since I already use Syncthing, it'll be nice to use it as a local sync tool too.