Fenix: Add the ability to use self hosted syncserver

[TcT2k]

Why/User Benefit/User Problem

As in lockwise (see https://github.com/mozilla-lockwise/lockwise-android/issues/542) there doesn't seem to be any way to specify your own sync server like you could do via about:config and identity.sync.tokenserver.uri.

This should at least be accessible some how. IMO it should even be prominently visible in a browser that prides itself on it's privacy features.

[grigoryk]

We have most of the necessary A-C and A-S APIs to achieve this. FxaAccountManager takes a ServerConfig already, so we'll need to ensure that it can be swapped out "mid-flight". Alternatively, an app-restart may be a good option as well, which would mean that no A-C changes are necessary.

In Fennec we have a "debug mode" that can be turned on in Account Settings, which enables specifying a custom accounts server URL. A similar approach could work in Fenix as well.

[grigoryk]

So, A-C actually needs a bit of work here to not force people to also self-host FxA servers, but that work should be trivial: https://github.com/mozilla-mobile/android-components/issues/3729

[grigoryk]

We need to figure out where to place this configuration. It should live somewhere in settings, but perhaps outside of our current accounts settings, as those are accessible only after logging in.

Configuration in question is a single editable text field, without a default value.

[lime124]

this looks like a bigger feature request so removing needs ux label. once the team prioritizes this for a release we can plan on having a designer take a look.

[rfk]

One potential issue here is the new webchannel support infrastructure, which IIUC hard-codes the browser to only accept webchannel messages from one specific domain specified at build time; /cc @vladikoff

[nandryshak]

Any update on this? I tried editing identity.sync.tokenserver.uri in about:config today (Nightly 200205 06:00 Build #20360607) and it seemed like the value I entered was ignored. It synced my old stuff from the Mozilla servers.

[tublitzed]

@nandryshak - it's still in progress; see this related pre-requisite. Once #3729 is merged, the plan is to come back here and get this implemented. I don't have an exact ETA, but this has been prioritized; I'd say roughly we're looking at sometime in the next few months for this to merge.

[rfk]

One potential issue here is the new webchannel support infrastructure, which IIUC hard-codes the
browser to only accept webchannel messages from one specific domain specified at build time

I happened to be poking at the code for this for other reasons, so wanted to add a bit more context here.

The FxA integration in Fenix includes a custom webextension that allows message-passing between the web content on accounts.firefox.com and the native code in the application. The code for this webextension lives in android-components here.

For security, it's very important that the browser only accept messages from the legitimate accounts website, not from arbitrary other sites. This is currently enforced by a hard-coded check that only injects the webextension's content script when the origin is precisely https://accounts.firefox.com.

In order to allow the use of non-default FxA servers, we'll need to refactor this to allow the check to be dynamic based on the configured accounts server, rather than using a hard-coded constant.

[rfk]

In order to allow the use of non-default FxA servers, we'll need to refactor this to allow the check to
be dynamic based on the configured accounts server, rather than using a hard-coded constant.

I just realized that another, shorter-term option may be to fall back to using the default OAuth flow (rather than context=oauth_webchannel) for users who have configured a custom FxA server. This would disable some features of the sign-in flow (such as choose-what-to-sync) but would allow users to get set up and syncing with their own self-hosted servers.

[rfk]

@grigoryk we've talked recently about what a first version of this might look like, do you think we might be able to approach it as part of the "integration testing" effort in March? It seems at least partially on-theme given that QA folks need to use this functionality for testing.

[Dunexus]

Why is this in a hidden menu "not intended for end-users but for QA only" ?

This feature has been requested by end-users in the first place so if it is working now, some end-users would be happy to use it.

[flyser]

You can not seriously close this feature request as "done" with this workaround.

[rfk]

It looks like this was closed automatically due to the "Fixes" comment in #8916, but I agree that's not accurate and this issue should not yet be considered fixed. #8916 only brings support for domains used by internal QA, not for arbitrary self-hosted domains:

The FxA integration in Fenix includes a custom webextension that allows message-passing
between the web content on accounts.firefox.com and the native code in the application.
For security, it's very important that the browser only accept messages from the legitimate
accounts website, not from arbitrary other sites.
[...]
In order to allow the use of non-default FxA servers, we'll need to refactor this to allow the check
to be dynamic based on the configured accounts server, rather than using a hard-coded constant.

AFAICT #8916 and the associated android-components change in https://github.com/mozilla-mobile/android-components/issues/6012 did not fully address this concern. It's now possible to to use servers from a short list of allowed domains rather than only from the production FxA domain, but the FxaWebChannelFeature does not yet work with arbitrary self-hosted domains.

It has, however, brought us a lot closer to full self-hosting support.

@ekager or @grigoryk could we please re-open this?

[eoger]

Apologies, I shouldn't have linked this issue with the pull-request earlier. So what has landed?

You are now able to use your own Sync server authenticated with Mozilla's Firefox Accounts servers (https://accounts.firefox.com).
Most people have this setup as it's easier than hosting the whole FxA+Sync stack.

Allowing users to also point Fenix to their own Firefox Accounts server is covered by https://github.com/mozilla-mobile/android-components/issues/6225. I might get to it in Q2 as I have other priorities, however I wouldn't mind mentoring somebody to work on it.

[rfk]

You are now able to use your own Sync server authenticated with Mozilla's Firefox Accounts servers

Ah yes, right. I guess technically you can now do what the initial issue report requests (which was only about self-hosting the sync server, not the account server). But I expect at least some users use "self-hosted syncserver" as a shorthand for both FxA and Sync so it's worth keeping this open until we have support for both.

[Feandil]

Could the fix be backported to the v75.0.0-beta familly?
It doesn't seem to be present e.g. in https://github.com/mozilla-mobile/fenix/blob/v75.0.0-beta.5/app/src/main/java/org/mozilla/fenix/settings/about/AboutFragment.kt

[nandryshak]

This seems to work for me now on nightly! (Sync + auth using Mozilla servers). Should I report any bugs in this issue?

[rfk]

Should I report any bugs in this issue?

Please file as separate issues so they're easier to keep track of and respond to.

[jerheij]

This seems to work for me now on nightly! (Sync + auth using Mozilla servers). Should I report any bugs in this issue?

How did you manage to get this to work on the Nightly build?
Ignore this; I digged a bit deeper and found the way to get it to work in the nightly as well. I just hope they soon promote it to the beta and stable branches because it is a feature I use quite a bit!

[jerheij]

Any update on this issue? It seems to work through the developer options in the nightly build but not in the beta branch. Or is there a separate git issue for using a custom syncserver?

[cadeyrn]

It seems to work through the developer options in the nightly build but not in the beta branch.

It works in the stable release of Firefox Preview since version 4.3. Firefox 75 on the Beta channel is still based on Firefox Preview 4.2 so it's not yet included. This feature will arrive on the Beta channel in the next few days (I guess) - once the update to version 76 happens.

[jerheij]

It works in the stable release of Firefox Preview since version 4.3. Firefox 75 on the Beta channel is still based on Firefox Preview 4.2 so it's not yet included. This feature will arrive on the Beta channel
in the next few days (I guess) - once the update to version 76 happens.

Oh okay, I guess the different versions (Preview / Regular / Focus got me a bit confused. That answers my question about the feature!

Unrelated question; why is there a different preview Firefox Preview app or are "regular" and Preview in the process of being merged?

[p-schneider]

With Firefox Beta 77.0.0-beta.1 in 100% rollout we can now test Self Hosted Sync Servers in "Firefox for Android Beta" (org.mozilla.firefox_beta).

(AFAIK the debug mode still needs to be activated by tapping the logo in "About Firefox" multiple times in order to access the settings for custom account server and sync server.)

[d98ama]

As soon as I turn on sync in Firefox Beta, it stops working for my other Firefox installations (on my phone, tablet and Linux PC).
Is this expected?

If not, maybe I can supply some more information. Just let me know what you need in that case.

[rfk]

As soon as I turn on sync in Firefox Beta, it stops working for my other Firefox installations
(on my phone, tablet and Linux PC).

@d98ama that's definitely not expected, could you please file as a standalone bug in bugzilla so that we can investigate? (by moving it to bugzilla we can mark the bug confidential if needed, in case you want to provide any debugging info that you wouldn't want to share in a public github issue).

[jeena]

~Wait so updating Firefox on Android installs this new version with a broken sync because of https://github.com/mozilla-mobile/android-components/blob/f362c94e32c715f334b3880f7bb696a94cdbca6a/components/feature/accounts/src/main/assets/extensions/fxawebchannel/manifest.json#L7? Is there a way to downgrade? Update: that seemed to be only about the accounts server. But sync still doesn't work for me.~

Update 2: Appearently I had a old version of the syncserver after upgrading it sync on android works for me.

~I tried the beta too but both have same behavior, I log in and then it gets my username then that part jumps a couple of times like it would try to load something repeatedly and then it shows this error:~

Error image

~Clicking on it and logging in again does end up back there again. My server log shows a http error 401.~

[Starfiresg1]

Since upgrading to the new Fenix-based Android (79.0.5) I've unsuccesfully tried to get this to work.
After setting the custom sync server URL and signing in to the Firefox-Account it never actually syncs succesfully - "last synced" shows either Jan 1st 1970 or simply never - clicking on it to trigger a manual sync seems to do nothing.
Furthermore my DNS logs show no connection attempt to my own sync server - the only relevant entries I find are "firefoxusercontent.com" and "api.accounts.firefox.com" - so the sync never seems to be redirected to my own server.
No error is shown the entire time.
If I access the devices list in my Firefox Account the Android device is shown as last seen "several seconds ago" after an attempted sync.
After many unsuccessful attempts I tried the current Nightly for Android which exactly behaves the same.

Screenshots of my settings

Logcat after issuing a sync doesn't seem to give a reason either:
08-25 22:52:50.503 23588 26148 I FirefoxAccount: Executing: get access token
08-25 22:52:50.504 23588 26148 D fxaclient_ffi: fxa_get_access_token
08-25 22:52:50.505 23588 26148 D fxaclient_ffi: fxa_to_json
08-25 22:52:50.505 23588 26148 D WrappingPersistenceCallback: Logging state to mozilla.components.service.fxa.manager.FxaAccountManager$FxaStatePersistenceCallback@175662a
08-25 22:52:50.512 23588 26148 D FxaStatePersistenceCallback: Persisting account state into mozilla.components.service.fxa.SharedPrefAccountStorage@854cbaa
08-25 22:52:50.519 23588 26148 I FirefoxAccount: Successfully executed: get access token
08-25 22:52:50.520 23588 23736 D fxaclient_ffi: fxa_get_token_server_endpoint_url
08-25 22:52:50.521 23588 23736 D BgSyncManager: Requesting immediate sync, reason: mozilla.components.service.fxa.sync.SyncReason$User@d772a9, debounce: false
08-25 22:52:50.521 23588 23736 D WMSyncDispatcher: Immediate sync requested, reason = mozilla.components.service.fxa.sync.SyncReason$User@d772a9, debounce = false
08-25 22:52:50.524 23588 26148 I FxaDeviceConstellation: Refreshing device list...
08-25 22:52:50.525 23588 26148 I FxaDeviceConstellation: Executing: fetching all devices
08-25 22:52:50.525 23588 26148 D fxaclient_ffi: fxa_get_devices
08-25 22:52:51.239 23588 26148 I FxaDeviceConstellation: Successfully executed: fetching all devices
08-25 22:52:51.239 23588 26148 I FxaDeviceConstellation: Refreshed device list; saw 17 device(s).
08-25 22:52:51.240 23588 26148 I FxaDeviceConstellation: Executing: polling for device commands
08-25 22:52:51.242 23588 26148 D fxaclient_ffi: fxa_poll_device_commands
08-25 22:52:51.510 23588 26148 D fxaclient_ffi: fxa_to_json
08-25 22:52:51.512 23588 26148 D WrappingPersistenceCallback: Logging state to mozilla.components.service.fxa.manager.FxaAccountManager$FxaStatePersistenceCallback@175662a
08-25 22:52:51.522 23588 26148 D FxaStatePersistenceCallback: Persisting account state into mozilla.components.service.fxa.SharedPrefAccountStorage@70bf876
08-25 22:52:51.538 23588 26148 I FxaDeviceConstellation: Successfully executed: polling for device commands
08-25 22:52:51.538 23588 23588 I AccountEventsIntegration: Received events, notifying listeners

I tried this on two different devices (Android 7.1 and 9.0). All desktop and old android fennec clients sync without problems.

[liuche]

I'm going to close this bug because the feature to use a self-hosted Sync server has been added.

Please continue in https://github.com/mozilla-mobile/android-components/issues/6225 or https://github.com/mozilla-services/services-engineering/issues/50

[Silvenga]

@Starfiresg1 did you figure why your sync error is occurring? I'm seeing the same.

[Starfiresg1]

@Silvenga Sadly no, I took several logcats during connections attempts without finding anything indicating an error. Combined with the fact that no connection attempt is even made to my self hosted sync server (not even the domain is being resolved in the dns server) makes me think that while the setting is there it doesn't seem to currently work.

[Silvenga]

Ugh, that's frustrating.

I wished this last update wasn't forced... I can deal with having my extensions being removed, but not having a self-hosted sync server defeats the idea of using Firefox.

[d98ama]

I also had problems when first upgrading to the latest version on Android.

I read somewhere that it could be because of an old version of the sync server. So I updated that, signed in on all my devices again, and now it seems to work.
At least history and passwords are synced. I haven't managed to sync any open tabs yet.

I can see activity in the logs on my server, so I'm pretty sure that it is used.

[metal450]

@d98ama, which version of the syncserver did you have to update to? The most recent tagged release is 1.8.0 (https://github.com/mozilla-services/syncserver/tags), which is what I'm using. Upon updating Firefox Mobile, syncing broke for me as well.

Also - syncing tabs is essential. @d98ama mentioned this not working. Can anyone else confirms whether this works or not (and if they're using syncserver 1.8.0)?

[d98ama]

I simply did a git pull on my sync server. I'm currently on commit e1aab54cbcb6e570979835789f7b4624eebdc875 from July 3rd (master branch).

Just tried tabs again, and it kind of works now.
On my laptop I can see tabs from my phone. But on my phone I can only see tabs from my tablet and my office workstation, not my laptop.
This may be unrelated though, I'm not sure it worked even on the previous version.

[metal450]

Ah, gotcha. I'm using a Dockerhub image that only pulls tagged releases, and they haven't tagged one since 2018 - so I guess that would explain it :/ Thanks.

[Silvenga]

I was using the v1.8.0 tag via my image - https://github.com/silvenga-docker/ff-sync

I'll rebuild with e1aab54cbcb6e570979835789f7b4624eebdc875 and see if that helps.

[Starfiresg1]

After endlessely removing and adding the new Firefox Mobile to the Firefox Account without any success I was finally able to make it work by using the brutal way of completely deleting the Firefox Account and recreating it. After this I started reconnecting both my desktop and Android device - this time it starting working instantly. I didn't change anything else (I was already using the current pull of the sync server).
The following lead me go this route:
In the process of trying to solve this I set up a new URL for my sync server. After I reconnected my desktop with the new sync server URL and noticed that even through I changed the URL on the desktop the sync logs (about:sync-log) used a mix of old and new URL which I found really strange. Since I didn't know where it was pulling the old URL from I decided to delete everything by deleting the account.
Which in turn also fixed my Android sync issue.

[Silvenga]

Using the latest version of the sync server had no impact for me.

@Starfiresg1 so you deleted the mozilla hosted account from accounts.firefox.com? What sync url did you use?

[laggarcia]

I tried last sync server and it didn't work. I tried then to delete and recreate my mozilla account, but still not working. I am using my own sync server.

[Starfiresg1]

@Silvenga
Yes I deleted the Firefox account.
My sync url points to my own hosted syncserver: https://domain:port/token/1.0/sync/1.5 (I'm using https through a reverse proxy)
Account server is empty since I'm not hosting my own.

[doggogoesbork]

same setup as above. works perfectly on all my desktop devices but not firefox mobile. replicated with the lastest firefox client, and firefox beta, and nightly.

another thing to note, when trying to sync manually with these settings, the last sync date becomes "December 31, 1969"

[thomaslitk]

The same problem: at FF daylight v82 on my Samsung Galaxy S20+ (Android 10) there seems to be no connection to my own sync server (https with let's encrypt), because sync failes as described above (last sync: ... 1970) with existing account. Furthermore, by trying with a new account, it is not added to the mariadb.

it works at FF nightly, but there are strong display errors in FF at Samsung Galaxy S20+. I don't know why.

It works with FF v68 (updates disabled), but on second day after (re)installation, no cookies are accepted in FF. However, in the settings, there are cookies enabled.