K-9: Re-enable Push

[cketti]

Figure out what needs to be done to make Push work reliably on as many devices as possible.

https://dontkillmyapp.com/ might contain some helpful information.

[beerisgood]

I can then test for AOSP (GrapheneOS) if needed.

[khaytsus]

I mentioned on IRC that I never had a push issue on Pixel 1 XL or Pixel 3 XL, stock ROM. I do have to disable battery optimization.

[beerisgood]

I mentioned on IRC that I never had a push issue on Pixel 1 XL or Pixel 3 XL, stock ROM. I do have to disable battery optimization.

Because stock use Google Push. All custom ROMs or ROMs without GAPPS can't use that

[khaytsus]

Yes sorry, I was a bit more terse than I intended to be. I'm happy to test any updates that would re-enable Push to test how they go on stock devices.

And as for using Google Push and Custom ROMs. Customs ROMs are great, I still use them on some older devices. But modern devices I don't, and I've observed the same trend in others. Said another way, stock devices are by far the greater majority of users more than ever. I would be a bit puzzled optimizing for custom ROMs in this regard.

[ArchangeGabriel]

Why is Google Push discussed here? This seems totally irrelevant to me as IMAP cannot use Google Push. Whatever K-9 did, does or will do, it did not, does not and will not rely on Google push AFAIK.

[wrapper]

+1 I'm having to manually sync on the latest beta 5.700

[gradius-v]

What about users that have all "battery-savers" disabled, why remove the PUSH option ? IMAP-Push never stopped working for me.

[olifre]

Push was always working fine for me on OnePlus 6 with the offiicial Android 9 and 10 ROMs, and battery optimizations disabled for the app. So I would also vote for a "switch" / option to re-enable this option for users who know what they are doing.

Leaving this aside: Is there any way users with a working setup can contribute to get this issue analyzed?

[beerisgood]

@olifre read https://github.com/k9mail/k-9/issues/4258#issuecomment-557914586

[olifre]

@beerisgood I read that (along with the full thread), but as @ArchangeGabriel , I'm not clever enough to figure out why Google push would be related to IMAP IDLE at all.

Is it because Google push enforces that the device keeps up a persistent TCP connection, which - coincidentally - also IMAP IDLE requires, such that the built-in Google stuff "by accident" fixes IMAP IDLE by forcing the TCP stack to stay active?
Could you enlighten us to clarify how users can help out?

[cketti]

Our GitHub issue tracker is not a discussion forum. We use it to coordinate development work. Unless you wish to work on an implementation, please refrain from posting here and use the mailing list instead.

[metbic]

When can we expect to get it back? It's most important to many people.

[beerisgood]

@metbic you can switch to FairEmail which works

[metbic]

@beerisgood Well, seems like I have to, since downgrading to 5.600 doesn't work with f-droid.

[licaon-kter]

@metbic that's an Android "feature" not limited to F-Droid, so export settings, uninstall, install older, import.

[backinabit]

From what I understand, the issue with push messaging being killed, is due to Android killing k9mail altogether while it is backgrounded. This isn't actually a new problem since the background service could always be killed under certain conditions, such as memory pressure, however, Android is now aggressively attacking background services in its quest for reduced power consumption. I have, in fact, always found k9mail push messaging to be somewhat unreliable for this reason.

The solution to this (both the case of memory pressure AND power savings) is to use a foreground service, which has the unfortunate side-effect of coming with a persistent notification. Foreground services are exempt from being killed by the power savings mechanism, and are higher priority than other background processes to protect them when under memory pressure.

Now naturally, you don't want to spam the user with an unnecessary persistent notification. The good news is that with notification channels, the user can be easily directed to configure the foreground service's persistent notification to be hidden, and without the side effect of killing all of the application's notifications. Doing this does NOT change the nature of the foreground service -- it remains exempt from being killed and maintains a higher priority than background services such that under memory pressure, it is not killed until the system really has no alternative.

See this for explanation regarding setting up the foreground service and notification;
https://www.spiria.com/en/blog/mobile-development/hiding-foreground-services-notifications-in-android/

EDIT: FairEmail uses this approach, and after using it for ~2 weeks (Pixel2 w/10) with hidden persistent notification and disabled battery optimizations, it has never missed a message. During the same period, k9mail of a version prior to push being disabled, also with battery optimizations disabled, missed every message.

[AndroidAppSeeker]

Can a frequent poll setting, say 5 minutes, be used as a work around ?

[nikwen]

$100 bounty on Bountysource for the person who successfully solves this issue. :tada:

---

Edit: @ttomasini chipped in another $20, making it $40 in total. :heart:
Edit 2: Another fine folk raised the bounty to $50! :star2:
Edit 3: Thanks to the awesome @hazzl we are now at $60! :cake:
Edit 4: Let's celebrate reaching the $100 milestone due to the amazing @sgiebels. :medal_sports:

[chcolin]

@nikwen And so? Can K9mail now run in the background and constantly check for incoming emails like FairEmail?

[femto-code]

K-9 version 5.600 (prior to disabled push feature) worked very well (Android 9) and notified the user thanks to Push about almost every mail in time.
Is there any information about IMAP IDLE being re-enabled respectively what is needed to accomplish this? The solution of starting a _foreground service_ mentioned by @backinabit is actually used in similar applications with positive effects (Telegram, FairEmail etc). By re-integrating this, K9 could definitively prove to keep up with other open source mail clients.

What is the difficulty at this point with the result that there is a reward set out (although I thinks it's great)? Why does it require a device-specific development effort?

[Lion-box]

I can confirm that v5.600 still works perfectly with IMAP IDLE on Sony XZ2c with Android 9
Never ever lost a single email notification.
The only tweak I used was excluding K9 from battery optimization.
If only people could manage not to open and keep in the background tens of stupid unuseful apps....

[KIMWW]

EDIT: FairEmail uses this approach, and after using it for ~2 weeks (Pixel2 w/10) with hidden persistent notification and disabled battery optimizations, it has never missed a message. During the same period, k9mail of a version prior to push being disabled, also with battery optimizations disabled, missed _every_ message.

The push issue has been going on for too long.
How good is FairEmail? Perhaps we should switch to FairEmail...

[khaytsus]

Perhaps we should switch to FairEmail...

If you want to switch, feel free. No need to troll here.

[KIMWW]

Perhaps we should switch to FairEmail...

If you want to switch, feel free.

Are you suggesting there won't be any improvements with push and we should therefore switch to FairEmail?? If it's not the case and you are unable to reply my question which was:

How good is FairEmail?

Then sniff your own ass khaytsuce

[cketti]

@KIMWW: This issue tracker is used to coordinate development. K-9 Mail related questions can be asked in our discussion forum.

Your question is not related to K-9 Mail. It's also highly subjective. So the easiest way to answer it is to just try FairEmail yourself.

Your statement towards khaytsus is absolutely unacceptable. It's the reason your GitHub account is now blocked from interacting with this repository.

[stevie77]

Sorry for asking here, but I do not know where else: I use K-9 in stable version 5.600 on Android 6 and Android 10. IMAP Push works on Android 6 but not on Android 10. Is this the expected behavior and therefore a new version 5.800 is needed? Or should K-9 5.600 work with IMAP Idle on a Galaxy S10e with Android 10?

[dereks]

I landed here trying to find an update since the June 2020 blog post:

https://k9mail.app/2020/06/01/Whats-Up-With-K-9-Mail

It says after Push is implemented there will be a new official release. Yet this ticket does not show progress nor an ongoing developer discussion about implementation details.

I cannot understand why Push takes over 2 months to implement. What is the blocker? Has anyone looked at it?

I may be willing to contribute a few hours of development time if nobody else wants to do it. Alternatively, how much of a bounty would it take to motivate you guys to finish Push? I'm a long-time user desperate for the upgrade, and would like to help.

Thank you for your continued work and excellent application.

[bjo81]

@dereks Keep in mind this is not a discussion forum.

[phil294]

@bjo81 exactly, and @dereks comment is very much on-topic, implementation specific and polite. If it is really the core developers' consensus that no communication and questions whatsoever besides absolutely critical development information is allowed on these issues, we really need that pointed out in the README. Otherwise I fear tension will continue to arise from these situations, as it is not the way issue trackers are normally handled on GitHub. Thanks!

This is not a discussion forum, I know.

[femto-code]

@dereks I had the same question over and over the past weeks and I am starting to have concerns whether I should move away from K9 although I agree - it is a great application with a massive potential!
I also would like to support the implementation like other users @AndroidAppSeeker or @Lion-box who have made considerable suggestions. But instead of supporting a constructive developer discussion about PUSH and the possible alternative implementation options (and yes @bjo81 / @phil294 - this is necessary during a development - as opposed to belittling or lecturing others (!) ) I have to realize that all comments are marked as "off-topic".
Surprisingly all posts with bounty appear as "highly valued" for the development?! Nevertheless, like @dereks aptly named this does not have serious effects on an ongoing developer discussion.
But back to the PUSH issue: Obviously the PUSH implementation in v5.600 worked well for many users (see "off-topic" posts above). Maybe we could check on this and have it tested in upcoming releases.
@phil294 Yes, we should prevent a heated - and therefore "off-topic" - discussion. I also cannot confirm the existence of that consensus like you said.

[dereks]

dereks> I may be willing to contribute a few hours of development time ... would like to help.

@dereks Keep in mind this is not a discussion forum.
Our GitHub issue tracker is not a discussion forum. We use it to coordinate development work.

Great, let's coordinate some development work on the IMAP PUSH feature. I am an Open Source developer volunteering to submit a Pull Request or contribute towards a bounty. Tell me how I can best contribute, even if it's just testing new alpha code.

And if I may respectfully offer some advice and echo what @phil294 said:

This is a public GitHub Issue Tracker database. This Issue Tracker is (usually) one of the top 3 links Google returns when you search for K-9 questions.

Millions of GitHub users will expect to comment here the way we do on other Open Source projects: bug reports, feature requests, technical support for errors, and even discussions that revolve around screenshots and comparisons to other IMAP clients. It is how public GitHub repos are used. Our GitHub accounts give us write permission to this forum by default, and it is unreasonable to think people will "just know" that you prefer people to subscribe to the Google Groups mailing list, or go to https://forum.k9mail.app/.

If you need a private developer discussion channel hidden away from The Mob, then use a private Issue Tracker. (Private repos are now free on GitHub). Or use Slack or IRC, i.e. something that is private.

If you just keep repeatedly telling people to go away, it will tarnish K-9's reputation as an Open Source project. People won't stop landing here from a web search and posting one-off comments or questions. The posts will continue to annoy you, and your response of "not welcome!" will continue to annoy users.

For me personally, I will not take 5 minutes to subscribe to a mailing list and/or create a new account on a 3rd party forum just to ask a quick question that should take 5 seconds. GitHub works the way it does for a reason.

Thank you again for working on K-9.

[webratte]

Are there plans when you will reanable push?
I would like to use/test the new designed UI of k9.

But push is really important to me.

Thank you.
BTW. The new UI is really nice 👍

[frank42hh]

So, tomorrow is this issue's first birthday. Anyone wants cake?

[nottheend]

😄👍

For some reason the notifications became bit more stable for me recently. Not sure why