Ungoogled-chromium: Plans to introduce Chrome Sync (settings, extensions, bookmarks) functionality?

[dryshirt]

Currently, one of the features that I use Chrome for is the cross-platform sync of browser-related items, such as extensions, settings, passwords, etc. In addition with user profiles, this feature is what makes Chrome indispensable: being able to separately manage profiles (and the related extensions, browser settings, etc.) without hassle makes my life a lot easier. Unfortunately, since the goal of ungoogled-chromium is to remove all instances of Google in the browser, this means that synchronization no longer functions.

I would have opened a feature request, but I'm not exactly sure what kind of solution to offer; in addition, owing to the lack of relevant issues/feature requests, it seems more like a personal use-case problem. In my case, I typically use three computers - one laptop, one desktop, and (occasionally) public machines (library, school lab, etc) - having to re-configure chrome after an install or on adding a new user profile is a hassle, and maintaining settings, extensions, and bookmarks (foregoing passwords and history) is very time-consuming.

With that in mind, I'd like to open a discussion on the implications of implementing this feature; specifically, for settings, extensions and history. (bookmarks and passwords are possible via third party services, and open tabs seems impossible.) Is it possible?

[Tank-Missile]

This is where altnerative services come in. For bookmarks, you can use Floccus with a nextcloud instance or provider, or xBrowserSync. For passwords, you can sync a KeePassXC database using the nextcloud client along with the browser extension. As for syncing extensions, history, and settings... Yeah I got nothing there at the moment.

[Eloston]

It is possible to do this, but it won't be easy. The syncing code in Chromium is deeply tied with Google-specific services and specifications. We would need to define how we would want this syncing to work (as you implied at the end), and then investigate what we would need to do to implement it in the Chromium code. It will a large undertaking, to say the least.

If you don't use all three systems at the same time, maybe you could consider an external syncing system that can synchronize the entire profile directory?

[restfuladi]

I don't mind using Resilio or Syncthing to sync the entire profile. Which file/folder should I sync? Using Mac and Windows 10.

[Eloston]

I'm not entirely sure if the profile directories will be compatible under such platforms, but you'll need to synchronise the user data directory. The path to the currently-used profile can be found under chrome://version under "Profile Path". The user data directory is the parent directory of the Profile Path.

[dryshirt]

I tried out Resilio Sync for a couple days, and although it works most of the times, there are a couple key points which are definitely worsening the issue instead of fixing it:

1. Update intervals: Whenever there is an update to the directory, Resilio Sync attempts to send the changes to all other peers. Paired with an unstable or slow network (like my school's, for example) - this quickly becomes tiresome. There is no way to manually adjust sync intervals in order to reduce bandwidth usage (not that I would anyways - with the frequency of changes that are made, syncing on any reasonable interval of time (>1min) isn't very useful due to the large amount of changes made)
2. File sync failure/corruption: If Chromium is opened and incoming changes are made, then after restarting the app and visiting the settings page you will be told settings have been corrupted, and that they have been reset to default. This is by far the largest issue: there is no built-in method to prevent the corruption from happening, and it can happen very easily. After it happens, the user is forced to reconfigure their settings - there is no failsafe to easily revert changes.
3. File versioning: Another problem occurs after some usage: if the sync service doesn't start itself reliably, then point 2 happens repeatedly. This doesn't really have anything to do with Chromium, but it's worth thinking about when recommending third-party services in order to synchronize files across two or more systems.

Apart from that, I suppose the solution works as a temporary bandaid. However, after navigating the initial setup (and following mess), and the problem-filled days after that, I won't be likely to try it again.

[Eloston]

Yes, blindly applying a generic file-synchronization service to the profile directory is a bad idea. File synchronization services make assumptions about how the files are used that are not valid for a Chromium profile (as you experienced first-hand). For example, generic file sync assumptions about consistency and how to approach consistency do not work for Chromium profile files; the atomic changes (e.g. adding a new bookmark, adding a new website to the history) made to the Chromium profile may require changes across multiple files.

If there is some way to translate the atomic changes to a format that conform to the assumptions of generic file synchronization services (e.g. DecSync, which assumes Syncthing), then it should be possible to use a generic file sync service to sync changes. However, this still isn't a trivial task...

[xhiddle]

we would need to do to implement it in the Chromium

export whatever to a fixed filesystem location outside of chromium profile directorie trees. Use third party tool to sync given location to whichever cloudy services. User provides own sync tool, and cloud credentials via that tool.

way less huge undertaking? Option to auto export user selections on startup as shutdown seems sketchy.

consider an external syncing system that can synchronize the entire profile directory?

And that's what I get for not completely reading prior to replying.

nextcloud

This reminds me to ask vultr why the listed $2.50/mo USD price is unobtainable.

Maybe this tool has the desired promise:

https://github.com/ncw/rclone

especially with

• Optional encryption (Crypt)
• Copy mode to just copy new/changed files
• Sync (one way) mode to make a directory identical
• Check mode to check for file hash equality

and/or

https://restic.readthedocs.io/en/latest/010_introduction.html

: via https://help.nextcloud.com/t/rsync-to-cloud-storage-for-backups/43574

[t00]

How about reverse-engineering google services api and keeping the changes in ungoogled chromium to a bare minimum, possibly only introducing a setting which points to a custom api server.

This would require a separate project implementing the api and user can host their personal api themselves or possibly have a cheap encrypted hosting service available for anybody. This would definitely improve ungoogled chromium adoption while preserving all advantages of the stock chrome.

There might be many urls needed but hopefully these share a common domain. By default as currently this domain would point to a non-existent site.

As a bare minimum sync settings can be hidden as currently until user manually changes the domain in chrome://flags/, until the new api implementation will be production ready.

I understand this will be another chromium feature to chase to keep up with api changes but it should not add any noticeable additional ongoing workload for ungoogled-chromium developers.

[Eloston]

@t00 Sounds like a fun idea. For ungoogled-chromium, I also think the changes can be manageable. The only downside with this approach is the time and effort required to reverse engineer and implement the server:

• I speculate that reverse-engineering based on the code alone could be difficult, especially for determining any stateful logic on the server-side.
• Also, we would need to create a solution for authentication (Maybe some secret token generated from the server that we add to the browser? Maybe reverse engineer sign-in logic to the custom server?)
• Creating a reliable and secure web service takes time to get right.

I won't have time to work on something like this, but I think it would be a nice feature to have.

[f4most]

Could this new option perform local encryption prior to upload?

We can haz privacy?

[midenok]

Sorry, but I don't understand what's the problem. Chromium syncs with Google, therefore the API is open source. Why this project can't sync with Google like Chromium does?

[t00]

@midenok Sure the api client code is open source but the idea behind ungoogled chromium is not to use google services so instead a new open source api service needs to be implemented, identical in behaviour to the original google api, to allow sync without ties to google.

[midenok]

That looks not very wise idea to me. It would be better to take good things from Google and don't take bad things.

Having Google account probably is a no-go for this browser for users like me. Though I'm pretty happy with Slimjet (which syncs just fine).

[jstkdng]

I'd say google having any kind of data about ourselves is a bad thing.
And we do take the good things google does, just look at this project, is chromium but without the stuff that gives google our data (bad thing) and just a nice browser with good performance.

And the solution @t00 gave would be a good one. Sort of how firefox does:
https://mozilla-services.readthedocs.io/en/latest/howtos/run-sync-1.5.html
But do it how nextcloud does, allowing users to setup their own server or being a provider for other people who can't setup a server, of course, encrypting everything server side.

[TheJags]

I know this is an old thread but I just came across this, while searching for an option to sync (bookmarks, passwords, custom search engines, history) between Ungoogled Chromium on Ubuntu Focal and Ungoogled Chromium Extensions on Android 10 (LineageOS 17.1)

Has anyone found a solution by Oct 2020? Thanks alot.

[mind-overflow]

Hey @TheJags, I'm actually working on this.
I've been working on a self-hosted server that allows you to sync browsing data between different browsers (even both Firefox & Chrome) since August, as I would've really liked something like that which, however, didn't exist.
Development has, however, stalled for the moment, as I'm getting busier everyday, having to study and work.
This does not mean that I'm abandoning the project, just that it might take a little longer to complete. I really can give no ETA, since I'll have to dig deep into Chrome & Firefox's databases structure and extensions API to make this all work good.
There is a GitHub organization already set up for this, however, all repos are currently private as they are still in experimental/alpha state. Just give it a quick check sometime, I'll try to remember to update the status here with another comment!

[samcday]

Perhaps an interesting approach might be to integrate Mozilla's Firefox Sync into Chromium?

As @jstkdng pointed out, Firefox sync server is completely open source, and you can run your own sync endpoint.

Furthermore, the guts of the sync client behaviour has been split out into a set of Rust components called Firefox Application Services. This appears to handle all the heavy lifting of the low-level sync engine. The higher level manager behaviour is then implemented differently in Fenix (Firefox Android), Firefox iOS, and Desktop.

So, theoretically, one could try hacking those components into Chromium. No idea how feasible such a feat might be. I've started looking at the Firefox code to understand their Sync stuff better, but I've not looked at the Chromium side at all.

[mind-overflow]

@samcday what's the difference between this and what I'm working on? yes, obviously Firefox Sync is already supported in Firefox. but we are talking about Chromium, so either way the thing needs to be implemented from scratch. also, implementing Firefox Sync would mean that you need to do it from Chromium's source code, which is worse, since you obviously can't move to mainstream Chrome or any other Chromium browser for that matter... while an extension would be more versatile, and more os-independent. I'm pretty sure Ungoogled Chromium maintainers are not interested in this, since at that point anyway you'd be developing a Chromium-based browser such as Brave or Opera, with your own custom homemade features, and not simply providing an "ungoogled" version of Chromium anymore.

[samcday]

@mind-overflow

what's the difference between this and what I'm working on?

No idea, haven't seen what you're working on, there's no repos in that org you linked in your previous comment! :)

but we are talking about Chromium, so either way the thing needs to be implemented from scratch.

My point is that Firefox Sync already has what appear to be fairly portable clients written in Rust to handle all of the complexity of low level syncing. Put differently, I think the Mozilla folks have been thinking about and dealing with a large deployment of a browser data syncing service for a while, so the possibility of reusing most of that client + server logic seems appealing, at least to me.

However I acknowledge the higher level manager / UI code will be bespoke and may not be able to reuse much of what already exists in either the Firefox or Chromium code.

while an extension would be more versatile, and more os-independent.

Sure, this could be one way of doing it. I'm not sure an extension gets deep enough access into the guts of Chromium to achieve the kind of sync that native Chrome / Firefox Sync offer though. You might end up needing to write bridge glue in Chromium codebase to make such an extension possible? :shrug:

I'm pretty sure Ungoogled Chromium maintainers are not interested in this, since at that point anyway you'd be developing a Chromium-based browser such as Brave or Opera, with your own custom homemade features, and not simply providing an "ungoogled" version of Chromium anymore.

Perhaps. I'm certainly curious what opinions said maintainers actually have on the matter!

[mind-overflow]

No idea, haven't seen what you're working on, there's no repos in that org you linked in your previous comment! :)

>

while an extension would be more versatile, and more os-independent.

> Sure, this could be one way of doing it. I'm not sure an extension gets deep enough access into the guts of Chromium to achieve the kind of sync that native Chrome / Firefox Sync offer though. You might end up needing to write bridge glue in Chromium codebase to make such an extension possible? 🤷

You make me sound like I don't know what I'm doing. I wouldn't have started nor even tried undertaking such a huge project if I already knew this wasn't possible, and obviously I checked and digged deeply before even considering the idea.
In fact, both Chromium and Firefox share the same API for history handling: Chromium | Firefox.

What I'm trying to do is have both browsers sync to a server, by using an extension. The server provides a REST API including login/registration; the extension sends GET or POST requests signed with a JWT. Finally, all history is saved in a remote SQL database (SQLite for the moment, given that I wanted simplicity over complexity of setting up MySQL or Postgres).

I don't really care if Firefox Sync can be modified in such a way that it'd be compatible with this, as I wanted this project to be a self-challenge, and also, I wanted to put in practice everything I know about database management, REST APIs and such.
Now, I haven't made the source public yet because, as I already stated, I don't feel comfortable with it at the moment, as it's still very incomplete.

But I really struggle to understand your attitude towards me. I'm not here to hide anything, I'm not here to sell any product. You will obviously be able to both see the source code and download + self-host the thing all by yourself. Heck, even compile it if you want to. I'm not trying to prove I'm better than anyone, I'm just a dev hitting up and saying that there's actually someone working on this - which by the way is far more on-topic and very similar to what both the OP and other participants were talking about, suggesting things like xBrowserSync or Floccus, that are missing the history syncing feature. Even if the underlying structure is gonna be totally different, the experience for the end user will be comparable to using any of the said extensions: install it, connect to your server, automatically sync.

Again, I'm not trying to convince you or anyone about anything. I just wanted to share something that might interest the ones who partecipated in this issue, as I started developing this exactly for the same reason.

[t00]

@mind-overflow your solution from what you described is implemented using extensions - there is not much relation to this issue of making Chromium inbuilt sync possible.

@samcday suggests using a working and tested library written by Mozilla to sync with Mozilla sync server, also open sourced. All this means that only a thin bridge layer needs to be implemented and maintained for Ungoogled chromium, which would need to be hooked up in place of existing chromium sync code and translate structures expected by Chromium to these implemented in the Firefox Application Services.

Such undertaking requires "only" considerable knowledge of Chromium codebase, as the Mozilla part is well documented and client-server compatible. It also means more patches for Chromium code are needed - Rust interop + build tools and new facade code to translate structures, all of it more fragile to Chromium changes (many times a day).

It is hard to balance to be honest which solution is better - reverse engineering Chromium sync logic or swapping it for Mozilla - reverse engineering has a value of making all of Chromium codebase usable without additional dependencies on the client side while the Mozilla path will make a robust and tested sync most likely quicker to finish but harder to maintain.

I am also interested in maintainers opinion on that :smirk:

[jbalme]

https://github.com/brave/brave-browser has sync, is based on Chromium, and is open source, seeing how they bolt on their sync engine might also be a good starting point.

Relevant PR: https://github.com/brave/brave-core/pull/5294/commits

[samcday]

@jbalme wow, that's very interesting stuff! It appears that brave/go-sync is the code for the server endpoint that Brave v2 Sync uses.

Some things I found interesting from a cursory glance:

• The PR @jbalme linked seems quite straightforward. Most of the diff weight appears to be deleting all the custom "v1" Sync stuff Brave previously shipped, in favor of using the upstream Chromium components/sync code.
• The server looks to be much less documented than the Mozilla Sync server.
• The server needs a DynamoDB backend. Once upon a time I would have frowned at that, but I noticed recently that Scylla shipped a DynamoDB compatibility layer called Alternator which might work and eliminate the hard AWS requirement.

[jbalme]

I didn't read too far into the PR, mostly just the new UI. Nice to know they use upstream sync, should make it more maintainable.

I just had a look at brave/go-sync and it appears that replacing DynamoDB with something else wouldn't be too difficult as they abstract datastore operations into a simple interface and the language in the README ("Currently we use dynamoDB as the datastore") implies they want to leave the possibility of migrating to/adding other backends open.

[dm17]

Is there a favorite so far between floccus, xbrowsersync, or another?