Talk-android: Publish to F-droid

[lachmanfrantisek]

Hello,

thank all of you for such a nice app. Would it be possible to publish talk also to the f-droid repository like other nextcloud apps for android?

Thanks in advance!

[mario]

Hopefully in the future, yes - but the issue is you won't be able to receive push notifications unless you have MicroG, and I hear it's relatively hard to install.

[lachmanfrantisek]

Thank you for a quick answer.

I am not sure about the MicroG -- it can be probably installed directly via apk? (Or use play only for this.)

Would be nice to install at least talk via f-droid.

[mario]

MicroG is a set of services that are open source replacements for the Google Play services on "regular" devices.

https://microg.org/

You can download an APK of the Talk app directly until I figure out a proper way to support F-droid.
https://github.com/nextcloud/talk-android/releases

[lachmanfrantisek]

OK, sorry that I messed the MicroG and GlooglePlayServices...

I have a google-play, but would like to use it as little as possible...;-)

Thank you!

[mario]

Understood. Feel free to install the APK directly for now and let me know how you like the app!

[lachmanfrantisek]

Still waiting for the server update and preparing on the phone side..;-)

[ghost]

To publish to F-Droid, there would need to be a seperate FOSS flavor without GCM... And I don't think it's acceptable to be dependent on Google services for a project such as Nextcloud in the long run. Anyway, really cool app, I would love it to be completely independent from proprietary alternatives like Telegram :)

[mario]

@Matttter I've done the separation for the Nextcloud Android app, so I know. There's no way to do push notification without Google services, at least not on the server side. On the client side I hear it's possible to use MicroG but I've never tried it.

Maybe I will in the future.

[hex-m]

We had the discussions of GCM-dependency for years until it was fixed in Signal. It's true that you don't have push notifications without gapps/microg but that just means the device has to contact the server regularly and thus use more battery - right?

[mario]

This is not a philosophical, but a technical issue. This is a talk application - once I get a call, I want to get a notification right away, not in 5 or 10 minutes. Now, technically:

• the only way to make sure I can check the server in an interval is alarm manager
• how often would you query the server? 0.5 seconds? 1 second? 5 seconds? 10?

• to make sure such frequent polling works, you would:

  • have to disable battery saving
    
    
    
    • and still, some phones/vendors would stop the process after you do this for a certain number of times
    
    
    • the battery would be depleted in half an hour to an hour max
    
    

• Signal implements a service which is more than likely to die on quite a few phones, even when you ignore battery optimizations and other things. Sure, you can restart automatically, unless the watchdog services dies too :)

So if you ask me is this possible: definitely. Is it possible in a way where you wouldn't yell at me after a while for killing your phone? Absolutely not.

That being said, there will be an implementation for F-Droid eventually. I just don't know when I'll get to it, but am more than available to help you contribute if you decide to do so! :)

[kertase]

hasn’t push been around since way longer than google play? So it should be possible to use for example an http push protocol instead of google play?

[tobiasKaminsky]

Until you find the time @mario to get push implemented on f-droid in a proper way, why not do it the same as in the nextcloud app:
If detected that push is not supported/working show a snackbar saying that the user will not receive a notification on an incoming call.
But the user can still use the app to call people, to join a conference call.

I think that people that are using a complete google free device are willing to accept some drawbacks.
(At least I am doing ;-))

[ghost]

Yes, if it's not possible or practical to implement a notification service, then just having the app available on F-Droid, even without the notifications, would be great. Like @tobiasKaminsky proposed, a snackbar or a dialog on first launch or something of the sort that tells users what's up would then make sure users who do really need the notifications could resort to the regular app

[ghost]

@mario

There's no way to do push notification without Google services

why is that? what is so exceptional about GCM that it can't be replicated by another party? is there any documentation where i can read up on this? how does it work, shouldn't it just be a server sending notifications to a client and that's it? how does Google prevent the enormous battery drain and what prevents anyone else from doing the same?

[mario]

It's a long story. Like I said - you can use MicroG to receive notifications sent by Google WITHOUT having Google services installed, but it's a bit involved. Alternate ways include things what signals does, or what we'll eventually do - service, IGNORE BATTERY OPTIMIZATIONS, and do http get every now and then (quite often, 10 seconds?).

I'd say GCM is magical in a way it can interact with the system due to having the privilege of being a Google product, so system will never kill, as opposed to even foregrounded services (or even apps that ignore battery optiomizations) which are less likely to get killed, but still do get killed.

You can download an APKs from here if you don't have Play Store:
https://github.com/nextcloud/talk-android/releases

Sure, they have Google Play compile code baked in, but I think it shouldn't crash - the only thing is you won't be receiving notifications.

Like I said before, there will be an F-Droid version, I just don't want to make promises when since I want to get it right, as much is possible in this situation.

[kertase]

Instead of get could you not use use long pooling or maybe webpush api could be implemented in nextcloud and could be used instead ?

[mario]

Since Nextcloud has to be accessible on lots of installations, websockets or such is not an option (and you'd still need a service!). And long polling still requires a service that would do it.

[ghost]

@mario

so the only actual advantages that GCM has over other notification services is that it's very optimized to not drain much battery, and that its service doesn't have to restart because the system will never kill it?

then maybe the best way to do it without requiring multiple different app versions would be:

if GCM/microG is installed on the system then the app uses it

if GCM/microG is not installed but the system is rooted the app asks the user for permission to run the service as root so it won't be killed

if GCM/microG is not installed and the system is not rooted, the app tells the user that the notification experience may not be perfect

to make sure that not every nextcloud suite app needs root maybe the main app could implement this and handle the notification service of the talk app and any other app that needs it, if GCM/microG isn't present.

also the user should be able to set a custom refresh rate for the notifications, so everyone can decide for themselves how much speed vs battery they want.

btw since google plans to switch from GCM to FCM, will any of this change or will it basically stay the same?

[mario]

GCM == FCM mostly. It's the same thing.

[testbird]

An elegant way for nextcloud to avoid depending on google push servers (or whatever current cover name) could be to leverage on the IMAP push. On phones without GCM it is proven to work (https://delta.chat) and it is using even less battery then having GCM installed.

All the NC server would have to do is to send a special email (so they can also be easily filtered), and the NC client could watch the IMAP server for such messages.

(For XMPP there might also exist a working implementation, but almost all users may already have an email account.)

[testbird]

Actually, there is a delta.chat issue open to support initiating video chat.
So maybe get in contact to work out an "email signaling format" that could let delta chat or other MUAs open a NC video chat.
Then you don't even have to implement and run another IMAP watcher on the phone.
https://github.com/deltachat/deltachat-core/issues/101

[testbird]

@mario is there maybe already some manual way to invoke the nextcloud app from another app? For example, can it answer a call if the caller sends a link in an email message, and the user clicks on the link.

[testbird]

Some "email push" message standard brain storming (possibly based on the autocrypt, and email chat standard) at https://github.com/deltachat/deltachat-core/issues/101#issuecomment-364890221

[testbird]

The paging / notification works very fast with deltachat. So if initiating a call with Nextcloud talk could send an email message with a link, then answering/receiving a call may be made as easy as clicking the link in the message when it arrives in the mobile email messenger (deltachat, conforming to autocrypt and other stanards https://delta.chat/en/standards), and have the messenger app open the link (call) with the nextcloud talk app.

[schemar]

I am assuming you do not want to look at push implementations where you need to set up a different server as everything should be part of the Nextcloud server, right?
Maybe it is time for Nextcloud to create an open PHP project for push notifications ¯_(ツ)_/¯

[mario]

Well ... afaik, you cannot do a websockets as part of PHP, so it would need to be a separate server, sure. And this would obviously increase the barrier to entry as not everyone will be able to host it. It would also need an app to nextcloud to make it work, or a patch to an existing one. So not that easy :)

[testbird]

That's why I proposed email. By now it has wide-spread push functionality, and virtually everybody has email, can get or set up email (even through self-hosting projects).

Establishing an Email-push-notification standard that defines push messages in a way that allows to auto-sort/filter them into a separate folder by sieve server filters and/or the IMAP push subscribed client (i.e. an Email-notify standard similar to the Email-chat standard https://delta.chat/en/spec) would realize an alternative that does not require to set up additional new servers. Push capable client libs that allow easy encryption through back-and-forth handshake are available (e.g. https://github.com/deltachat/deltachat-core), and seem easily expandable to support notifications in addition to chat messages.

[testbird]

Note that an email push notification standard could reduce the transfered data quite much. Only for the initial autocrypt initialization handshake it would have to use full email messages with a body and public keys attached. The actual push messages might not have to contain much more than a signature as "payload" (possibly in the subject). IMAP push then informs the client that a new message is waiting, and the client may optionally load only the subject for verification?

[mario]

The Telegram-FOSS issue is correct. You cannot guarantee a persistent service, and IMAP push cannot wake the device.

[mario]

There's MicroG like I mentioned before, and that's the best solution, even if it's still subject to background limitations etc etc etc.

[testbird]

Does push necessarily need to wake the device? Maybe ringing and letting the user wake the device is enough?

[mario]

You need to wake the device to ring :)

On Wed, 14 Feb 2018 at 09:27, testbird [e-mail] wrote:

Does push necessarily need to wake the device? Maybe ringing and letting
the user wake the device is enough?

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/nextcloud/talk-android/issues/58#issuecomment-365530340,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAWsn4s1XyPCQpC8cPFdTFfZm470zApks5tUpjjgaJpZM4RZfK7
.

[testbird]

Ok, then deltachat-core lib is already able to wake the device. It holds open a permanent notification that it is waiting for messages. To avoid that every app need its own watcher and hold an own notification open, deltachat-core client lib could maybe be packaged separately and provide a shared service that forwards messages to other apps.

[schemar]

One last try, even though I understand that push is practically not feasible :disappointed: Ratchet is a PHP WebSocket server implementation: https://github.com/ratchetphp/Ratchet

[X-Ryl669]

The issue with MicroG is that it's replacing/emulating GCM. In order to do so, it's replacing Google's private key in the ROM by its own keys for the GCM's service, so you must be root to install it. There is no MicroG without being root and you can't have both GApps and MicroG, so said differently, it's not a solution for the majority of users.

There is no solution to this issue, Google locked us all to its system for push notification.

The only solution that might work, it to build a common service/proxy that's using GCM to trigger fake notifications with some ciphered content that's telling which intent/application to wake up. That way Google will not know what is the notification about, yet your software can still be opensource and not dependent on Google's closed source (and accepted in F-Droid).

However, the "common service/proxy" that's relying on GCM can not be integrated in F-Droid (since it depends on GCM) but if it's open-source, and available on the play store, it might still be acceptable.

So basically, in this utopia:
1) Users having GApps will have to download the common service/proxy from the Play store and download the other software from F-Droid
2) Users not having GApps but MicroG can download the common service/proxy from the common service/proxy website (or maybe F-Droid ?)
3) Users without GApps nor MicroG don't have GCM anyway so they can not get push notification on recent Android versions/phones (unless they deal with battery optimizations, but this proved to fail anyway).

[thermatk]

A better idea would to move to foreground with a persistent notification(as notifications can be hidden per channel in Oreo, it's not annoying). Yet it would be extremely ineffective to do this in every app. Implementing one common gate for push notifications that users can self-host and installing one foss helper app that would talk to it would be the best. And I don't know a project better suitable to be at the center of such new system than Nextcloud, the self-hosted everything else 👍

That would also be the solution for our https://github.com/Telegram-FOSS-Team/Telegram-FOSS/issues/172
Continuing to dream, the client library API can be made as close as possible to GCM, while sadly I think on the server it would take more effort to incorporate it. The more reasons having a good common standard, instead of a myriad of solutions for every app.

[testbird]

Thanks for clarifying @X-Ryl669. This emphasizes the need for a free, federated and commonly maintained push messaging alternative. Not only for android devices, but also for UBports, Librem 5, plasma-mobile, custom roms etc.

Users without GApps nor MicroG don't have GCM anyway so they can not get push notification on recent Android versions/phones (unless they deal with battery optimizations, but this proved to fail anyway).

Experience showed notifications work very well with https://github.com/deltachat/deltachat-core which builds on IMAP push (federated email system).

[testbird]

And I don't know a project better suitable to be at the center of such new system than Nextcloud, the self-hosted everything else

Then one would only be able to reach Nextcloud users, instead of all email accounts. (=> includes IMAP-push email servers)

[X-Ryl669]

deltachat use a foreground service so you get a permanent notification (you currently can't mask because any other notification would be masked too). So no, deltachat solution is not good either. I don't want to clutter my notification area with many filtering rules, it's not intuitive and, IMHO, it's a hack that Google will fix in a later version anyway.

[X-Ryl669]

@thermatk Can you be more specific about the "hidden by channel" in Oreo ? How do you do that ?

[thermatk]

@X-Ryl669 Oreo introduced Notification channels, so even if the app implements it like having two channels "Persistent push channel" and "Everything else", one would be able to hide and mute just the persistent notification. Didn't see an app needing or doing it yet, but should work.
Yet channels are not important if we make one tiny unified push receiver with one open socket connection and one notification.

[testbird]

channels are not important if we make one tiny unified push receiver with one open socket connection and one notification.

Well, the android notification channels won't be important to hide the permanant notifications of multiple apps, but the "unified push receiver" would still need to be able to have something like channels to know to which app a message needs to be forwarded. (https://github.com/deltachat/deltachat-core currently implements "chats" and "group chats" (based on the From: and autocrypt).

[thermatk]

This has nothing to do with notification channels, just with the implementation of client app, respective lib to be added to apps and their inter-process communication

[testbird]

Right, we may better refer to that as something like inter-process receiver apps. (Current https://github.com/deltachat/deltachat-core frontends could become inter-process receiver apps as well, instead of packaging the lib.)

[Bubu]

Wow, this issue exploded...

Here is my 2c, why microG shouldn't be the solution:

• the needed libraries client-side are still proprietary, so no inclusion into f-droid for everything that requires it.
• We are still relying on the centralized google infrastructure which is bad for privacy concerns. Even if sending just empty pushes through that it still allows google to analyze all the communication metadata. (who communicates with whom and when.)

Otherwise I geneally agree with @thermatk here

[X-Ryl669]

@thermatk It has. Because the only way to avoid GCM is to have a process that's not killed by the system and the only way to have a persistent process is to have it run a foreground service. Because foreground service requires a persistent notification, it's a pain for users (why did google invented this?). Because it's "process" wide, unless you use notification channels, you can't really use this technic to bypass GCM (since hiding the foreground notification means no notification displayed whatsoever to the user). With notification channel and selective hiding of the "persistent foreground service" notification, it might be acceptable from a user point of view.

[testbird]

With notification channel and selective hiding of the "persistent foreground service" notification, it might be acceptable from a user point of view.

Thats how https://github.com/deltachat/deltachat-core currently implements it (as a lib, not yet a "unified push receiver" service), and it's even acceptable on Android =< 7 where I see one persistent notification "connected to [email protected], waiting for messages ..." (and way better than not getting any push messages).

[X-Ryl669]

True.
But, before Oreo, you can only hide the whole application's notification, one can not hide the "connected to ..., waiting for messages..." notification, else she would loose all notifications. If you have 20 applications like this, then it'll clutter your notification panel.

If, starting with Oreo, you can hide only the "foreground notification" but keep the other (that are sparse by nature) then it's ok from an ergonomic point of view.

[testbird]

before Oreo, you can only hide the whole application's notification

Wouldn't that actually be ok for a "unified FLOSS push service"? I mean if https://github.com/deltachat/deltachat-core would be converted from a stand-alone lib into a shared service app, that distributes messages to other apps, it's permanent notification to always stay in the foreground could be hidden by default. The inter-process receiver apps would then not need neither to each implement and run own push servers and clients, nor need to have a separate hidden notification channel to hide a permanent notification and always staying in the foreground. They would just need to use a "deltanotify-clientlib" to subscribe to notifications from the "deltanotify-core".

[X-Ryl669]

The issue is that not all FLOSS app use IMAP underneath (that's a huge dependency). So, only the "foreground service" in deltachat is interoperable. Instead, it could be interesting to have a common "Notification app" that being triggered by whatever FLOSS application running foreground services (and thus, hidden by user) to display something nicely and, on click, display the FLOSS application that triggered it.

Said differently, since all FLOSS app will have their own "push" protocol, it's hard to merge/unify them.
However, in all cases, when something is received on the pushed channel, it's always the same behavior: The user gets notified (somehow).
So, I believe the common part should be here, in the notification.

Or (the other solution):
Each FLOSS implement a kind of server plugin running on a GCM-like server, and we have a common FLOSS notification application that monitors what's talking to it and wakes up the final application (talk-android, deltachat, k9, whatever).

This is good on paper, but it's a lot of work (and it's both server work and client work and system administration work). Not sure all app developer will want to delocalize their "push" protocol server side and manage it.

[testbird]

Ok, I see two aspects to this then.

(One) Having a central notification source (daemon) on the device. This makes very much sense, because otherwise multiple apps would have to implement a part that stays permanently in the foreground, draining the battery. But GCM is increasingly locking apps into their central and closed system, so we need an alternative.

(Two) Relying on a common push protocol ("on the wire"). This is not a necessity, a notification daemon could support multiple protocols, but having some common protocol options could make sense to allow interoperability. Some important features for a FOSS solution would be federation, availability, and wide-spread usage (existing users, and bullet proof implementations). Mature contenders are XMPP, IMAP,...? Projects may choose to set up their own central servers, or let the users choose or setup their own servers, but they would not have to. Many users already have at least one IMAP email server that they trust, and they are free to change servers/providers anytime.

I can't say why google chose to force GCM on android instead of relying on a standard messaging protocol like XMPP, or IMAP push. But email certainly has the most users and fortunately IMAP-push is an adequate solution for mobile clients. If they would have made the right decision, in the interest of the users, we wouldn't be in this situation now.

It actually boils down to whether one would like that Nextcloud-talk, Telegram, Deltachat, Signal, Kontalk, ChatSecure, and all the other nice Messaging and Email-Applications would get a way to communicate with each other... due to force, or to choosing a free and open source protocol?

[testbird]

Well, to get forward, people will need to reach out now, consult, and step forward with areas of knowledge, ideas, support, availability, funding etc.:

Implementing FLOSS push message service (daemon app):
On android, it should create a permanent (hidden) notification to stay in foreground, and it provides

• [ ] inter-process communication with client apps
• [ ] network communication with support for:
  
  
  
  • [ ] autocrypt enabled IMAP-push email (using deltachat-core lib, with a filter for push spec compliant messages)
  
  
  • [ ] XMPP ?
  
  
  • [ ] ...other push server protocols?
  
  

Implementing android app support:

• [ ] client app library for inter-process communication with the push message deamon

Implementing server messaging support:

• [x] (php)sendmail with autocrypt helper (muacrypt or pyac https://autocrypt.org/dev-status.html)
• [x] own IMAP push server (https://en.wikipedia.org/wiki/Comparison_of_mail_servers)
• [ ] ...other push server protocols?

(@hpk42 @r10s @Ampli-fier)

[thermatk]

@testbird what am I missing and how is it even related to IMAP finally? To implement push emails one needs an open connection, for which there is an extensions for IMAP, but what's that have to do with the daemon? Is there an open source implementation?

In my opinion, priority would be API compatibility of the client library with GCM, changing just the package name in the best case and keeping other things equal. What has to change in the API is only the additional required setting with custom server domain/ip.

[X-Ryl669]

@thermatk What you describe is called MicroG. This has downside because:
1) GCM's application is a system application, you can't replace it without being root
2) The application is using cryptography to prevent being replaced -kindof- so you must be root to add your own keys
3) Not all application developers want to manage one or more servers for their users, yet GCM requires a server to work. This means that you have to delegate some of your privacy to a server (typically, the server needs to decrypt the message to figure who to notify) since it's the 24/7 running server that's sending message to GCM to wake your device.

I've proposed a solution for this issue in K9 without decryption, but it's specific to IMAP's IDLE implementation, and can not work everywhere.

[Bubu]

What about moving this discussion elsewhere? It doesn't seem particularly related to this issue.

I'd propose creating a new gitlab.com project (because we want to use free software, which github is not) giving everyone interested a wiki-like space for coordinating the efforts.

Additionally something for live discussion s would be nice. Not sure what we could all agree on. I can create an irc channel in lack of another common denominator. Other suggestions?

Would people be on-board to move this discussion somewhere else?

On February 15, 2018 4:30:20 PM GMT+01:00, X-Ryl669 [e-mail] wrote:

@thermatk What you describe is called MicroG. This has downside
because:
1) GCM's application is a system application, you can't replace it
without being root
2) The application is using cryptography to prevent being replaced
-kindof- so you must be root to add your own keys
3) Not all application developers want to manage one or more servers
for their users, yet GCM requires a server to work. This means that
you have to delegate some of your privacy to a server (typically, the
server needs to decrypt the message to figure who to notify) since
it's the 24/7 running server that's sending message to GCM to wake your
device.

I've proposed a solution for this issue in K9 without decryption, but
it's specific to IMAP's IDLE implementation, and can not work
everywhere.

--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/nextcloud/talk-android/issues/58#issuecomment-365962287

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[testbird]

IMAP, but what's that have to do with the daemon? Is there an open source implementation?

IMAP is an open and wide-spread "on-the-wire" protocol for message delivery.
I think 95% yes there is an implementation for what's needed for IMAP, the https://github.com/deltachat/deltachat-core lib is implementing the handling of general, autocrypted, and spam exempt, Email-chat spec compliant messages by listening on a connection to an arbitrary, regular email account (with IMAP push), without disturbing the regular email communication of the account with high frequency chat messaging. I think it would be desirable to not just rely on authenticated email chat messages for the notifications, but to have a slightly adapted spec that _further separates the notifications from the general chat messages_, and reduces the data overhead after the initial hand-shake. As the deltachat experience shows, there are quite a lot of things to solve to get messaging right, so the idea is to build on that.

The most prominent advantages that I see of using IMAP email is that the infrastructure is already there, and as a federated system, it does not enforce any particular organizational structure, while already being very accessible from virtually every programming language and platform. No need for app developers to run servers, no need for central servers, users may have their own servers.

[testbird]

moving this discussion elsewhere?

Time to spread the word, definitely, meet people who want this and are able to commit to this. Then start a gitlab project, or somebody is determined to take the lead and start a project.

https://stackoverflow.com/questions/20329403/android-push-notification-without-using-gcm

[iteco]

@testbird Isn't solution with IMAP leaving communication details between servers for a plain view (except encrypted message content)? If so, it looks that IMAP based solution is great for email apps (like K9) as a gradual step to improve things. But for the end-to-end encrypted new generation message apps (Telegram, Signal, NextCloud talk) that seems like a step backwards.

However, common FOSS notification server would be a great thing!

[testbird]

@iteco I really don't have an in depth answer concerning the implications for the newest generation apps.

Concerning IMAP, it should of course be secured IMAPS (non gcm servers). Think that it depends whether you communicate over your own server, and that one may distinguish between email as a universal way to communicate and just doing push notifications (for some unrelated updates etc.). This seems already quite useful for me for the general case.

To further this to the top level, one would have to use own servers, maybe there is an advantage to also implement another protocol (the checkmarks with ? above), but I don't know enough to tell.

[testbird]

@iteco Maybe you thought about something like the FOSS notification service on the mobile device, or your IMAPS server, to use something like a tor protocol. Though, just as the alternatives listed at
https://stackoverflow.com/questions/20329403/android-push-notification-without-using-gcm
this will require mobile and web app writers to support a new protocol, instead of just sending a (https://autocrypt.org compliant) email message.

[mario]

You should be discussing how to create a service that will not die, not the protocol :) Since the first is obviously harder and on some phones an impossible problem to solve.

[testbird]

Yes, what much can we do about such presuming phones, other than working on alternatives? Nothing.

@mario Your discussion is actually brilliant and right on point, you close the case with "impossible problem to solve", period.

[mario]

We need new phones that will get 50% of the market - done :)

[mario]

@Bubu there seems to be an alternative to client side libs?

https://github.com/microg/android_packages_apps_GmsCore

Edit: looks like I'm wrong and it doesn't work properly with Firebase :(

[mario]

@Bubu do you think you could mail me at [email protected]? I'd like some input on F-Droid support :)

[Bubu]

As nobody objected to gitlab I created a group and a wiki project there: https://gitlab.com/foss-push/planning/

You should be able to request access to the corresponding group in gitlab.

I also created #foss-push on freenode... all welcome to at least get some clarity about goals, requirements, options for this written down.

Let's see if this goes anywhere.

@mario Sorry for hijacking your issue tracker ;-)

[mario]

With some help from @Bubu, I've done lots of work on F-Droid support (without push of course), so we might still see this on F-Droid for 1.1 too :)

[testbird]

Thanks, Mario, looking forward to the F-Droid premiere!

(PS: @thermatk better explanations concerning push protocol and IMAP seem to be covered at https://gitlab.com/foss-push/planning/ now.)

[Bubu]

I've a preliminary metadata file here: https://gitlab.com/Bubu/fdroiddata/commit/3e5144287ebc004c9edf92100552f1b7ad31afe2

What's missing is a release containing the necessary changes for this to work. And @mario could you add descrition, summary and screenshots to your repo? See here https://f-droid.org/en/docs/All_About_Descriptions_Graphics_and_Screenshots/

[mario]

Hm, I thought I've answered this on IRC, but just in case .... ^_^

• screenshots, description and summary will be added
• beta2 is scheduled in a week or so
• as for tags, all unstable releases are either marked as unstable (via github flag (can you track this?)) or I'll have a "beta" in the tag name
• are capitalized flavour names a big issue for you?

cc @Bubu ^_^

[Bubu]

Hm, I thought I've answered this on IRC, but just in case .... ^_^

matrix.org's irc bridge was having trouble the last few days apparently, I never got your reply :-/

as for tags, all unstable releases are either marked as unstable (via github flag (can you track this?)) or I'll have a "beta" in the tag name

We can't track the github flag, but beta in tag name we can deal with. We'll figure something out.

are capitalized flavour names a big issue for you?

No, it just requires to manually set the output path, no big deal. Just an edge case that didn't come up before.

[mario]

@Bubu can you check shots and descs now?

[Bubu]

Screenshots and description don't show up as is. I have not much experience with this myself it might again be related to the flavour thing.

@tobiasKaminsky you implemented flavour specific fastlane metadata for fdroid, right? Could you have a look at this?

[tobiasKaminsky]

@Bubu I took your build receipt and enhanced it a bit: https://gitlab.com/fdroid/fdroiddata/merge_requests/2968

Currently it will not compile, due to a build error.

[mario]

@tobiasKaminsky you should probably make sure to only build non-beta versions.

Also what's the problem with building?

[tobiasKaminsky]

Auto update is set to stable version.
We can manually add beta versions, so that an user can select and install them.

[tobiasKaminsky]

Also what's the problem with building?

Latest beta cannot be build with ./gradlew assembleGeneric

/home/tobi/projekt/nextcloud/fdroid/fdroiddata/build/com.nextcloud.talk2/app/src/main/java/com/nextcloud/talk/application/NextcloudTalkApplication.java:40: error: cannot find symbol
import com.nextcloud.talk.utils.ClosedInterfaceImpl;
                               ^
  symbol:   class ClosedInterfaceImpl
  location: package com.nextcloud.talk.utils

According to drone, this seems to be fixed on master. Therefore I removed beta build on fdroid.

[tobiasKaminsky]

@Bubu https://gitlab.com/fdroid/fdroiddata/merge_requests/2968 is now ready for review & merge :tada:

[tobiasKaminsky]

@mario can we have a stable release, which supports "generic" flavor? Current stable 1.0.14 still have google deps and therefore cannot be published on fdroid.

[mario]

@tobiasKaminsky 19.03. for a stable release.

[mario]

cc @jospoortvliet if 19.03. suits you? That is, assuming nothing bad happens :P

[Bubu]

The fdroid keysharing issue has been solved. How do we proceed? add beta4 already? wait for release?

[mario]

@Bubu awesome! I'll tag a stable release either tomorrow or on saturday. Is that ok?

[tobiasKaminsky]

Just ping me @mario, then I can do the froid release and @bubu can merge it :tada:

[Bubu]

@tobiasKaminsky Way ahead of you! ;-)

https://gitlab.com/fdroid/fdroiddata/merge_requests/2968#note_63451053

[Bubu]

@mario feel free to close this or leave it open until the release is actually live on F-Droid.

[mario]

Will close, special thanks to @Bubu , @tobiasKaminsky and everyone else who participated in this issue!

[tobiasKaminsky]

@Bubu https://gitlab.com/fdroid/fdroiddata/merge_requests/2968/diffs#note_63634442
I fear that we need to change to "gradle=generic" .

[tobiasKaminsky]

It is now in Fdroid available :tada:
As I manually installed it at some point, and also installed gplay version at some point, I fear this is due to my messed up android, but does anybody have an idea how to fix this?
With root I deleted everything related to nextcloud talk and even wiped all fdroid data.

[lachlan-00]

The apk package is signed when built. Fdroid and Google apks have different signatures.

When you install a google play or custom signed version you need to uninstall the apk first then install the fdroid version.

[tobiasKaminsky]

Sorry, I guess I was a bit unclear, no talk app is installed anymore.
I verified this via:

OnePlus3:/ # cmd package list packages | grep -i nextcloud                                                                                                                                                                     
package:com.nextcloud.client
package:com.nextcloud.android.beta

[Bubu]

The error message says something about left over app data. You could do a find / -name talk or something maybe?

Anyway, I just installed and account importing from nextcloud worked! Very nice!

On March 20, 2018 12:10:11 PM GMT+01:00, Tobias Kaminsky [e-mail] wrote:

Sorry, I guess I was a bit unclear, no talk app is installed anymore.
I verified this via:

OnePlus3:/ # cmd package list packages | grep -i nextcloud             

package:com.nextcloud.client
package:com.nextcloud.android.beta

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/nextcloud/talk-android/issues/58#issuecomment-374559474

[mario]

@Bubu yay! I'm so happy that we waited! :)

[tobiasKaminsky]

@Bubu

With root I deleted everything related to nextcloud talk

I did exactly this, find / -name talk...

[DaCryptor]

After Googles "stunt" regarding Gapps and CTS (possible long-term lock-out for Custom ROM users in the future) it seems to be smart to find a solution to replace Google Cloud Messaging (GCM) soon.

[DudleyDursley]

And even today not everyone wants to install MicroG, just to get push notifications for one single app. But if I understand marios post correctly, there already is a plan to offer an alternative.

[mario]

@DaCryptor can you link me somewhere to read more on this?

[mario]

@DudleyDursley well, f-droid flavour is already out, I don't know when I'll get to thinking about this.

[testbird]

What kind of push notification method or other signaling is used in the f-droid version?

[mario]

@testbird absolutely none. I might implement polling mechanism eventually, but this will most probably kill the battery :-/

[testbird]

Ah, ok. And, would it be wrong to expect being able to receive calls if app is in background / foreground?
(Just to avoid disappoinments.)

[mario]

Once polling mechanism is in place, chances are that you will get call notifications while actively using the app, and not in other cases. On the other hand, chances of getting a notification while in background are rather slim I'm afraid.

This all comes with the additional drawbacks:

• the other end might hang up in 10-15 seconds window on how long it takes to poll so you will miss the call
• the battery will discharge at super fast rates :(

[testbird]

OK, thanks, now I understand the current state better. (Maybe inform about this limitation in to the F-Droid description?)

So, in lack of implementing a free push messaging client https://gitlab.com/foss-push/planning/ , piggybacking setup links into asynchronous email / massaging communication (https://github.com/nextcloud/talk-android/issues/96) might be an option that could avoid polling in nextcloud talk, while allowing to receive calls in foreground and background.

[DaCryptor]

@mario
Original article in German: https://www.computerbase.de/2018-03/google-play-store-apps-sperre/
Google-translation: https://translate.google.de/translate?sl=de&tl=en&js=y&prev=_t&hl=de&ie=UTF-8&u=https%3A%2F%2Fwww.computerbase.de%2F2018-03%2Fgoogle-play-store-apps-sperre%2F&edit-text=

Hopefully Google does not cut off the custom-rom-users one day...

[Gaeldrin]

Hello, I would have a question about this as I am using F-Droid without any Google anything (Android 7, LineageOS 14), but with a few extra APKs like WhatsApp and Facebook Messenger. The things is - Messenger seemingly polls the Facebook servers only when opened which is in accordance with what @mario said.

WhatsApp, however, receives all kinds of notifications basically in real time without me opening the app or turning off any battery optimization. Incoming calls received immediately. Battery usage is basically always under 5% on full cycle. After some search for any background info, I found this, maybe will help with ideas for the Talk service:
https://android.stackexchange.com/questions/43970/how-is-whatsapp-able-to-receive-messages-when-not-in-use

PS: I would second testbird's suggestion to mention this limitation in F-Droid's version if possible (unless it gets fixed soon) :)

[mario]

@Gaeldrin there are some apps that are automatically whitelisted by the phone vendors so they can do that, and that includes Whatsup. Unfortunately, we're nowhere near as "famous" as they are yet to be able to request this.

[hex-m]

@mario do you have a reference for this? @Gaeldrin mentioned he is using LineageOS so I don't see how the phone vendor could do that.

[mario]

@hex-m you're right, this is not related to Lineage. This is related to Oppo, Xiaomi, Asus, Huawei and a few others.

[violoncelloCH]

would it be possible to publish pre-releases to f-droid too? so they can be installed manually? I think there are many people using f-droid willing to test pre-releases...

[mario]

@Bubu could you add 1.2.0beta1 to F-Droid as a manual install?

[Bubu]

@mario I can add it tomorrow if that's not too late already?

[mario]

@Bubu it's not. There'll be a beta2 this week though :)

[Bubu]

Added beta1 now. I noticed the version name doesn't reflect the betaX status, it just shows 1.2.0 not sure if that's intentional? (It works this way as well, it just might be confusing for some people).

Just to clarify, you are planning to bump the VersionCode for beta2 and again for release I then?

[mario]

Yes. The version name lacking beta1 was just an omission on my part. Beta2
will have it.

On Mon, 14 May 2018 at 09:37, Marcus Hoffmann [e-mail]
wrote:

Added beta1 now. I noticed the version name doesn't reflect the betaX
status, it just shows 1.2.0 not sure if that's intentional? (It works this
way as well, it just might be confusing for some people).

Just to clarify, you are planning to bump the VersionCode for beta2 and
again for release I then?

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/nextcloud/talk-android/issues/58#issuecomment-388724499,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAWsq5tcgMx5pqfQAfaHlKQJUF9greEks5tyTRJgaJpZM4RZfK7
.

[Bellusterra]

I’m not a programmer so I don’t know that something like this would even be possible, but would there be a way for the polling frequency to be changed automatically based on if a user on the server opens their app then the polling frequency of each of the other users apps would also increase? Once all users apps go into ideal or are closed the polling frequency can drop back down to conserve battery power? I would guess that the current polling when the app is closed would have to be adjusted to the average time that it would take a user to open their app hit the contacts tab, scroll to the user they wish to call, and then actually call them, for this to actually even work.

[Bellusterra]

Or for larger groups it could even be dynamic where the system learns who you frequently call and when you open your app it increases the poll rate of those individuals apps on their devices.

[testbird]

Is someone interested to try exploring a proof-of-concept for push signaling in F-droid?

What kind of intent would a push notification have to emit for incoming nc-talk calls?

The idea would be to patch deltachat-android (IMAP-push based) to emit an intent (e.g. to nc-talk) on messages coming from a specific email contact.
https://github.com/deltachat/deltachat-android/blob/c5701745ebbd031c2aaa536ad47c194cc595c9f3/MessengerProj/src/main/java/com/b44t/messenger/NotificationsController.java#L377

[testbird]

(i.e. for allowing the nextcloud server to wake up the called client device, and signal the call, by sending an email to the user)

[bam80]

@testbird Would IMAP-push based notification require yet another persistent connection to IMAP server?
The idea is fine otherwise

[testbird]

If someone could provide a patch to the NotificationsController of deltachat-android (as in https://github.com/nextcloud/talk-android/issues/58#issuecomment-398816098) to emit an android "intent" to nextcloud-talk, this could be a proof of concept. It would work based on the same IMAP connection (email account) that is configured in deltachat. The nexcloud-talk server would just have to send a recognizable email, and deltachat would wake up the device and pass a message to nextcloud-talk.

The further idea is to convert the message reception and passing into a separate service (app) later (to be used by apps like deltachat, nextcloud-talk etc.). https://gitlab.com/foss-push/planning/wikis/home Even then it should be possible for it to maintain only a single IMAP connection, and passing received email-notifications, email-chat, and classic emails on to the corresponding apps.

[bam80]

@testbird what it's intended to do with these "push" emails - should they be deleted immediately upon receive? AFAIK if they are grouped to separate IMAP folder they would require additional connection anyway..

[testbird]

Deltachat currently only watches the INBOX, moves chat messages into the separate DeltaChat folder, and internally groups incoming emails from "approved contacts" into separate "chats" according to the sender and recipient group, there is no proper auto archive or delete yet (https://github.com/deltachat/deltachat-core/issues/120), however, it should be possible to delete the whole "nextcloud-talk notifications chat" from time to time if desired, as it would only contain wake-up notifications.

For a proof-of-concept and testing how IMAP-push could work, a patch to deltachat NotificationsController would seem like the easiest path.

(An idea could be to allow configuring a combination of sender address AND a subject, to trigger sending an intent containig the message to a configurable app. This would allow using the same account in nextcloud to send such messages to oneself, and even selecting a specific device or similar.)

[testbird]

@bam80 Some servers do allow to watch multiple folders with one connection (imap-notify) https://github.com/deltachat/deltachat-core/wiki/IMAP-strategy

(also added more details to my answer https://github.com/nextcloud/talk-android/issues/58#issuecomment-410291533)

[github-k8n]

I think there seems to be some notion that "Push" messages and specifically Google Cloud Messaging are something "special".

If I understood it correctly it should be like this:
Basically it is just the google app keeping a connection open with one of the nearby GCM servers.
That is basically the same thing that IMAP idle does.
Through this open connection, any notification can be "pushed" to the device.
I don't know of any side channel that would notify/wakeup the device, however if anyone has any details on that I'd be eager to know more...
The advantage that the GCM servers might have is that mobile phone proivders will not drop idle connections to those servers (which avoids having to reconnect if the connection is cut after a while..)
The second "advantage" is that you are bundling the notifications, i.e. only a single connection has to be maintained instead of multiple ones.

=> as testbird and others mentioned, IMAP idle gives me push notifications without battery drain.
The only battery drain comes if you do polling instead (i.e. opening new connections every x seconds...)

The ideal open source way would be to:

• Create a notification server that can be deployed anywhere.
• Create a library in android that handles the communication with that server (IP configurable on the phone)
• Create libraries that can be used by servers to use the notification server

(Basically recreating GCM but with a configurable server)

The disadvantage is that this requires support by the server and the apps so probably will not be popular...
-- the other solution would be to just do long-lived open connections in the app itself, but as others mentioned Google is actively trying to break things there by killing apps in the background... so the only way "GCM" is special is because google says so (and does not kill it running in the background)

[arendtio]

@mario I think waking the device when a call comes in is an essential feature for a 'talk' app and Nextcloud as a challenger of the large cloud corporations might want to use some alternative to the Google push services (people who use F-Droid are probably the same who use Nextcloud, and they do it for a reason). So this issue might be a bit more critical than it was qualified.

Nevertheless, I understand that writing a push alternative isn't simple and takes some time. A few years back I read this essay from the Conversations (XMPP chat app) author about push notifications and battery usage which I found very insightful, and as a regular Conversations user (installed through F-Droid) I can tell you that it has a superb energy footprint: https://gultsch.de/xmpp_2016.html

So maybe looking at the Conversations source code could prove to be useful.

[bes1002t]

As Huawei smartphones are shipped without google services, this issue might become important again. I know it is difficult to implement a push service, but I think for the future this will be very important.

Maybe there is already a open source push service alternative? The big disadvantage would be a battery that will be empty very fast if multiple apps are using play services alternatives.

[DJCrashdummy]

@bes1002t there are some projects, but nothing really ready for prime time (yet)... e.g. the 2 i "know" a little bit better are:

• FOSS-Push, which never gained track beside planing resp. brainstorming...
• OpenPush, by @Bubu