K-9: Lost all email accounts

[andrenarchy]

This is already filed as a bug on google code but I couldn't find it here on github. This bug still is present in v5.006 – it just happened to me on Android 5.0.2 (CM12.0, One Plus One).

What exactly happened: all my email accounts were simply gone without any prior interaction. All user data of the app seems to be deleted. There wasn't any update to K9 or another app before it happened.

Are there any leads where this could come from?

[cketti]

I don't think this is a bug in K-9 Mail. Given our number of users there should be far more reports of this if it was.
But even if it is a bug in K-9 Mail there's not much we can do without a reliable way to reproduce this issue.

[andrenarchy]

It just happened again. Does k9 generate a log somewhere? (my phone is rooted)

[cketti]

https://github.com/k9mail/k-9/wiki/LoggingErrors

[andrenarchy]

Thx! It lost all accounts tonight again. I'll activate logging and wait for the next wipe. :)

[mikran]

I'm not opening this as I don't have any more information but I wanted to just say that this is happening to others too. I've had a new phone for about 3 weeks now and this has happened 3 times.

v.5.010 installed from F-Droid
Nexus 5x (Android 6.0.1)

[cdpxe]

I had the same problem yesterday: All account data was just lost when I started k9mail. The tool was then just showing its welcome message.

Maybe, just maybe, this was linked to the `extreme energy saving' mode I turned my phone into on that day.

[merlin-it]

Today the same happened on my device.
As a software developer I really understand how important an accurate bug description is. Even more important is the description how to reproduce the faulty behaviour, as mentioned before.
At the moment turning on error logging and waiting for the problem to reoccur seems the only way to get more details.

Anyway maybe some background information from users might help in narrowing down the problem. Most users will not have logging enabled and since we do not know how to reproduce it, logging would needed to be active 24/7.

In my case it happened over night. My device was in flight-mode without any network connection, as every night. No update was installed the last days. When I enabled wifi this morning and openend K-9, I was greeted with the setup-screen. Yesterday in the evening everything was still working fine.

I use K-9 Mail 5.010 on Android 6.0.1 / CyanogenMod 13.0-20160921-NIGHTLY-hammerhead on a Nexus 5.

Thanks to all K-9 developers for contributing to this really cool Mail-Client! This was the first really annoying bug I encountered in many years of heavy usage.

[philipwhiuk]

I agree - this sort of wildly intermittent bug is unlikely to be fixed before we address #1732

[cdpxe]

It just happened for the 2nd time. I started K-9 Mail this morning and it as only showing the welcome screen. All accounts were lost again. I use v5.010.

Unfortunately, I don't have any time to debug this now as I barely get enough sleep besides work. I just hope somebody provides some helpful hint here so that this super-annoying bug can be fixed.

[jdmulloy]

This keeps happening to me too. Very frustrating. I tried exporting my settings so I could just reimport them when it happened again, but the import fails. I'll enable logging, but right now K9 is useless.

[philipwhiuk]

@jdmulloy If you can reproduce that import failure (presumably you can) I'd love to try and fix that. If you could open a new issue and paste the stack trace / use the GitHub web UI attach your K-9 settings file (it doesn't have passwords in) / email it me that would be super :-)

[Whichcraft]

Same here. It just forgets the mail settings. Very frustrating. Had this 3-4 times already. I like the mail client, but this way its useless.

[Habbie]

This happened to me today. Tapped a 'new mail' notification, gave me 'Welcome to K9'.

[christianfl]

Happened today and about 3 or 4 times in the last 2 weeks. Never had that issue before. I am using LineageOS 14.1-20170509. I try to give more information, this bug is driving me crazy.

[ghost]

Happened to me about three times these last 2 weeks. Never had this problem before.
K9: 5.010
Android: Stock 7.1.2
Security patch level: May 5, 2017
Kernel: 3.10.73-g368902f
Build: N2G470

[con-f-use]

Just wanted to point out that it is still happening (as in to me, yesterday). A workaround would be nice, e.g. a hidden, programmatic backup of the config that gets restored when it happens.

[vlucchetti]

Has been happening for years, continues again today.

K9: 5.206
Android: 7.1.2 Pure Nexus
Security Patch: June 5, 2017
Kernel: Elemental X N6P-4.10
Build: N2G47W

[n00b42]

Happened several times already over last years, and once more last days.

K9: 5.206
Android: 6.0.1
ROM: CM 13.0-20161015-NIGHTLY-bacon
Device: OnePlus One

[cketti]

I had this happen to me once. The message databases of the accounts were still there and intact. Only the settings from the main database were mostly missing. Maybe caused by some race condition when writing settings.

Importing previously saved settings should restore things. The still existing message databases will be reused.

[realkinetix]

Hi! I have to post a 'me too' on this as it just happened for me on 5.208, and it has happened several times in the past.. I can generally tell something's up when I stop receiving notifications for mail for a day or so.

Nexus 6P running stock Android 8.0.0.

[Tobi823]

I had this issue today ... and I have a full log:

I'm using:
K-9 Mail Version 5.208 from F-Droid
LineageOS-14.1-20171211-NIGHTLY-bullhead without GApps
Nexus 5X + rooted (only rooted app ist AFWall+)
Central European Time, CET (UTC +1h)

When I open K-9 Mail today (18.12.2017 07:01) all my settings are gone.

These lines were logged at 17.12.2017 01:20:
12-17 01:20:15.180 I/k9 (31137): Preferences storage is zero-size, importing from Android-style preferences

indicating, that https://github.com/k9mail/k-9/blob/62df90d13be2cc54f35f22a284bba14370f23790/k9mail/src/main/java/com/fsck/k9/Preferences.java#L44 was triggered and deleted my settings.

Here is my log (I replace the real names of my two mail accounts)

k9log2.txt

Edit:
This is the unfilterd logcat output:
12-17 01:20:15.161 I/k9 (31137): Loading preferences from DB into Storage
12-17 01:20:15.162 I/ActivityManager( 7136): Low on memory:
...
12-17 01:20:15.162 I/ActivityManager( 7136): cch+4 S 0: com.fsck.k9 (pid 10082) cch-started-ui-services
12-17 01:20:15.162 I/ActivityManager( 7136): Free RAM: 457,868K
12-17 01:20:15.162 I/ActivityManager( 7136): Used RAM: 1,273,751K
12-17 01:20:15.162 I/ActivityManager( 7136): Lost RAM: 19,861K
12-17 01:20:15.180 I/k9 (31137): Preferences load took 19ms
12-17 01:20:15.180 I/k9 (31137): Preferences storage is zero-size, importing from Android-style preferences
12-17 01:20:15.182 I/k9 (31137): Committing preference changes
12-17 01:20:15.187 I/k9 (31137): Preferences commit took 5ms
12-17 01:20:15.240 I/k9 (31137): Registered: unmount receiver
12-17 01:20:15.243 I/k9 (31137): Registered: shutdown receiver
12-17 01:20:15.249 V/k9 (31137): Registering content resolver notifier
12-17 01:20:15.261 I/k9 (31137): Committing preference changes
12-17 01:20:15.275 I/k9 (31137): Preferences commit took 14ms

[cketti]

@Tobi823: Are there any log entries between 01:16:44.214 and 01:20:15.161? It looks like K-9 Mail's process was killed and the app restarted. It'd be good to know what caused this.

[Tobi823]

Here is the Logcat Output from 12-17 01:13:47.171 to 12-17 01:24:58.151.

logcat.txt
(I replace my wifi-mac, wifi-networks and my mail accounts with placeholders)

I found these interesting lines in the log:
12-17 01:20:13.771 E/lowmemorykiller( 386): Error writing /proc/10082/oom_score_adj; errno=22
12-17 01:20:13.952 I/WindowManager( 7136): WIN DEATH: Window{81cbded u0 com.fsck.k9/com.fsck.k9.activity.MessageList}
12-17 01:20:13.960 I/ActivityManager( 7136): Process com.fsck.k9 (pid 10082) has died
12-17 01:20:13.961 W/ActivityManager( 7136): Scheduling restart of crashed service com.fsck.k9/.service.PushService in 1000ms

[ghost]

Happened to me too, for the first time, after a couple of years using K-9 in Cyanogenmod/LOS. I don't have logs, I noticed only 3 days later.

K-9 Mail v5.208 from F-Droid
LineageOS 14.1
microG, no GAPPS

[hills]

Looks like I have experienced the same here. This morning, K-9 Mail (5.208) had reverted to as if it was newly installed.

• Doesn't seem there was an update to K-9 Mail or system update
• Phone had not rebooted; uptime was 5 days
• The preferences appear to still be present on disk; they haven't been deleted in whole
• This is the first time I've experienced this, in about 3 years of use

I tried basic things initially:

• Rebooting the phone
• "Delete cache"

But K9 Mail continues to present the "Welcome to K-9 Mail" screen. I followed the instructions in https://github.com/k9mail/k-9/wiki/LoggingErrors (unable to activate debugging logs as I can't get that far). Using adb I get the following:

--------- beginning of system
--------- beginning of main
12-29 14:09:17.797 I/k9      (16576): Loading preferences from DB into Storage
12-29 14:09:17.823 I/k9      (16576): Preferences load took 26ms
12-29 14:09:17.854 I/k9      (16576): Registered: unmount receiver
12-29 14:09:17.856 I/k9      (16576): Registered: shutdown receiver
12-29 14:09:17.863 V/k9      (16576): Registering content resolver notifier

I thought about filing a new issue, as I do not experience the "Preferences storage is zero-size, importing from Android-style preferences" above, however maybe that happened at the time of crash or corruption.

The preferences appear to be intact, with the most recently updated file being databases/preferences_storage. The file isn't corrupted; it's stable but it contains only a small amount of information (name, email address, signature)

Seems like my next steps are:

• find a way to enable more logging in K-9 Mail to establish why it rejects the preferences
• restoring from a backup

1|serranoltexx:/data/data/com.fsck.k9 # find -mtime -3 | xargs ls -dl                                                                                                                                      
drwxr-x--x 10 u0_a79 u0_a79     4096 2017-12-29 08:26 .
drwxrwx--x  4 u0_a79 u0_a79   229376 2017-12-29 12:32 ./cache
drwx------  2 u0_a79 u0_a79     4096 2017-12-29 12:32 ./cache/decrypted
drwx------  2 u0_a79 u0_a79     4096 2017-12-29 12:32 ./cache/temp
drwxrwx--x  6 u0_a79 u0_a79     4096 2017-12-29 02:58 ./databases
-rw-rw----  1 u0_a79 u0_a79 49369088 2017-12-29 07:50 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db
-rw-rw----  1 u0_a79 u0_a79   524288 2017-12-29 07:50 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db-journal
drwx------  2 u0_a79 u0_a79     4096 2017-12-29 06:17 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att
-rw-------  1 u0_a79 u0_a79    23695 2017-12-26 17:45 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80034
-rw-------  1 u0_a79 u0_a79    32768 2017-12-26 19:05 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80042
-rw-------  1 u0_a79 u0_a79    16574 2017-12-26 23:06 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80129
-rw-------  1 u0_a79 u0_a79    56755 2017-12-27 01:16 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80159
-rw-------  1 u0_a79 u0_a79    28617 2017-12-27 05:35 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80167
-rw-------  1 u0_a79 u0_a79    21547 2017-12-27 06:00 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80168
-rw-------  1 u0_a79 u0_a79    32768 2017-12-27 15:10 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80231
-rw-------  1 u0_a79 u0_a79    16952 2017-12-27 18:08 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80278
-rw-------  1 u0_a79 u0_a79    17877 2017-12-27 20:02 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80305
-rw-------  1 u0_a79 u0_a79    34308 2017-12-27 20:58 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80312
-rw-------  1 u0_a79 u0_a79    20318 2017-12-28 06:00 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80387
-rw-------  1 u0_a79 u0_a79    32768 2017-12-28 11:31 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80406
-rw-------  1 u0_a79 u0_a79    17418 2017-12-28 20:12 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80520
-rw-------  1 u0_a79 u0_a79    32768 2017-12-28 20:13 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80523
-rw-------  1 u0_a79 u0_a79    23842 2017-12-29 06:00 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80612
-rw-------  1 u0_a79 u0_a79   115092 2017-12-29 06:00 ./databases/2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att/80614
-rw-rw----  1 u0_a79 u0_a79    20480 2017-12-29 08:22 ./databases/preferences_storage
-rw-------  1 u0_a79 u0_a79    12824 2017-12-29 08:22 ./databases/preferences_storage-journal
lrwxrwxrwx  1 root   root         31 2017-12-29 08:26 ./lib -> /data/app/com.fsck.k9-2/lib/arm
drwxrwx--x  2 u0_a79 u0_a79     4096 2017-12-29 12:27 ./shared_prefs
-rw-rw----  1 u0_a79 u0_a79      208 2017-12-29 12:27 ./shared_prefs/unread_widget_configuration.xml.xml

serranoltexx:/data/data/com.fsck.k9 # sqlite3 ./databases/preferences_storage                                                                                                                              
SQLite version 3.9.2 2017-07-21 07:45:23
Enter ".help" for usage hints.
sqlite> SELECT * FROM preferences_storage;
2b04e23c-faf8-4421-a48f-a97461ea2f74.signatureUse.0|true
2b04e23c-faf8-4421-a48f-a97461ea2f74.name.2|Name
2b04e23c-faf8-4421-a48f-a97461ea2f74.name.0|Name
2b04e23c-faf8-4421-a48f-a97461ea2f74.description.2|xxxxxxxxx
2b04e23c-faf8-4421-a48f-a97461ea2f74.latestOldMessageSeenTime|1514493426000
2b04e23c-faf8-4421-a48f-a97461ea2f74.email.0|[email protected]
2b04e23c-faf8-4421-a48f-a97461ea2f74.signature.0|-- 
Name
2b04e23c-faf8-4421-a48f-a97461ea2f74.signatureUse.1|true
2b04e23c-faf8-4421-a48f-a97461ea2f74.description.1|yyyyyyyy
2b04e23c-faf8-4421-a48f-a97461ea2f74.signatureUse.2|true
2b04e23c-faf8-4421-a48f-a97461ea2f74.name.1|Name
2b04e23c-faf8-4421-a48f-a97461ea2f74.email.1|[email protected]
2b04e23c-faf8-4421-a48f-a97461ea2f74.signature.2|-- 
Name

2b04e23c-faf8-4421-a48f-a97461ea2f74.description.0|zzzz
2b04e23c-faf8-4421-a48f-a97461ea2f74.email.2|[email protected]
2b04e23c-faf8-4421-a48f-a97461ea2f74.signature.1|-- 
Name

MailService.previousInterval|-1

[hills]

I saw that databases/preferences_storage was far more complete in my backup. I took a bet on restoring only this file, and everything appears to have usefully returned to life.

I didn't restore the complete com.fsck.k9 directory, so it could be something is out of sync. But with the tools available to me right now it was easier to restore a single file than the whole directory. My backup is ~3 weeks old.

[asmw]

This happened twice in the last couple months to me.
Before that I've never seen the issue in months of usage and updating.
When I import a settings backup I have to re-enter my passwords and it seems to work again without re-downloading my inboxes.

Anything I can provide from a non-rooted phone to help when/if it happens again?

[Tobi823]

Even with an non-rooted phone you can provide an error log (https://github.com/k9mail/k-9/wiki/LoggingErrors). But you have to setup the Android Debug Bridge (https://developer.android.com/studio/command-line/adb.html) and increase your log file size (https://www.androidcentral.com/how-enable-developer-settings-android-42 then "Developer options" > Increase "Logger buffer sizes" to 16 Millions and enable "Store logger data persistently on device")

[crazy4chrissi]

My wife had the same issue 3 times now on her Nokia 3. I never had the problem on my Nokia 5. A little hard to debug when its not your own phone. The only thing I know is she installed some new app yesterday, and that her phone crashed completely yesterday: The screen suddenly went black and the phone seemed turned off (the battery was about 90% and charging at the time of the crash). She was unable to power it up and I managed to start it only after a couple of minutes and after pressing different combinations of buttons.... So her phone somehow crashed badly yesterday (never happened before) and today k9 has lost the settings (probably already yesterday). But k9 had lost its settings twice before without a crash like this. I will try to enable logging on her phone.

As the logs by @Tobi823 indicate, it seems to be caused by a crash, maybe due to low memory. This might also explain why it happens regularly on some devices, or why it happens only after a device has been used for a couple of months/years. It may depend on the regular memory usage of the phone, sudden power losses due to battery removal etc.

I am wondering, why the logs of @Tobi823 say that 457 MB of free RAM is low on memory. According to my Android settings, I have 468 MB free memory on average on my Nokia 5 and have no problems with this (2 GB total).

In general, I think k9 should not corrupt its settings when it crashes. It should store its settings in something that has a journal and recovers easily by itself after a crash. What does it currently use? Maybe better use an SQLite database than textfiles.
Edit: Okay, it already seems to use SQLite...

[crazy4chrissi]

In the directory listing from @hills, it seems that there is a rather big preferences_storage-journal file. And the journal file has different rights than the database file (no group-rights). Maybe recovery of the journal fails because of missing rights on the journal file?
At least one of the reasons for SQLite corruption mentioned here: https://www.sqlite.org/howtocorrupt.html

[ekbfh]

After update just lost all of accounts. Last version from market.

[hills]

Yesterday had the same happen again. It looks the same as my previous comment, above, but I have a little more log information.

The timeline is that on first using my phone in the morning (8:52am), the K9 widget on my home screen was set to a default.

serranoltexx:/data/data/com.fsck.k9/databases # ls -l
total 42076
-rw-rw---- 1 u0_a79 u0_a79  7110656 2018-03-15 17:14 1c38a6d4-4ab4-4f7f-822f-6f77bab1786b.db
-rwxrw---- 1 u0_a79 u0_a79   524288 2018-03-15 17:14 1c38a6d4-4ab4-4f7f-822f-6f77bab1786b.db-journal
drwx------ 2 u0_a79 u0_a79     4096 2018-03-15 16:42 1c38a6d4-4ab4-4f7f-822f-6f77bab1786b.db_att
-rw-rw---- 1 u0_a79 u0_a79 34664448 2018-03-17 02:37 2b04e23c-faf8-4421-a48f-a97461ea2f74.db
-rw-rw---- 1 u0_a79 u0_a79   524288 2018-03-17 02:37 2b04e23c-faf8-4421-a48f-a97461ea2f74.db-journal
drwx------ 2 u0_a79 u0_a79     4096 2018-03-16 23:20 2b04e23c-faf8-4421-a48f-a97461ea2f74.db_att
-rwxrw---- 1 u0_a79 u0_a79    81920 2017-03-03 02:11 4cd7248b-5b60-47bb-b871-af4d4bb37e23.db
-rwxrw---- 1 u0_a79 u0_a79    12824 2017-03-03 02:11 4cd7248b-5b60-47bb-b871-af4d4bb37e23.db-journal
drwx------ 2 u0_a79 u0_a79     4096 2017-03-03 02:11 4cd7248b-5b60-47bb-b871-af4d4bb37e23.db_att
-rwxrw---- 1 u0_a79 u0_a79    81920 2017-03-03 02:11 5c234f7c-7950-4ac9-a0d3-3d42f73a339f.db
-rwxrw---- 1 u0_a79 u0_a79    12824 2017-03-03 02:11 5c234f7c-7950-4ac9-a0d3-3d42f73a339f.db-journal
drwx------ 2 u0_a79 u0_a79     4096 2017-03-03 02:11 5c234f7c-7950-4ac9-a0d3-3d42f73a339f.db_att
-rw-rw---- 1 u0_a79 u0_a79    20480 2018-03-17 03:03 preferences_storage
-rw------- 1 u0_a79 u0_a79    12824 2018-03-17 03:03 preferences_storage-journal

The timestamp on preferences_storage suggests a time of a failure at 3:03. Here's the detailed timestamps:

serranoltexx:/data/data/com.fsck.k9/databases # stat preferences_storage                                                                                              
  File: `preferences_storage'
  Size: 20480    Blocks: 40      IO Blocks: 512 regular file
Device: b318h/45848d     Inode: 64813    Links: 1
Access: (660/-rw-rw----)        Uid: (10079/  u0_a79)   Gid: (10079/  u0_a79)
Access: 2018-03-17 01:51:31.522622048
Modify: 2018-03-17 03:03:15.555339382
Change: 2018-03-17 09:47:55.865639576

serranoltexx:/data/data/com.fsck.k9/databases # stat preferences_storage-journal
  File: `preferences_storage-journal'
  Size: 12824    Blocks: 32      IO Blocks: 512 regular file
Device: b318h/45848d     Inode: 64831    Links: 1
Access: (600/-rw-------)        Uid: (10079/  u0_a79)   Gid: (10079/  u0_a79)
Access: 2018-03-17 01:51:31.542643412
Modify: 2018-03-17 03:03:15.575360747
Change: 2018-03-17 03:03:15.575360747

I was then able to hunt down the adb logs from that time:

03-17 02:31:02.851   604  1472 I chatty  : uid=1000(system) Binder:604_E expire 3 lines
03-17 02:31:05.145   604  4190 I chatty  : uid=1000(system) Binder:604_19 expire 6 lines
03-17 02:31:23.094   604   711 I chatty  : uid=1000(system) Binder:604_3 expire 3 lines
03-17 03:02:29.644   604   687 I chatty  : uid=1000(system) MountService expire 3 lines
03-17 03:02:29.671   604   688 I chatty  : uid=1000(system) VoldConnector expire 7 lines
03-17 03:02:29.710   182 13349 D vold    : Starting trim of /data
03-17 03:02:29.827   182 13349 I vold    : Trimmed 53219328 bytes on /data in 110ms
03-17 03:02:29.925   182 13349 I vold    : Benchmarking /data/misc/vold/bench
03-17 03:02:56.674   182 13349 V vold    : Before drop_caches
03-17 03:02:57.189   604  4188 I chatty  : uid=1000(system) Binder:604_17 expire 4 lines
03-17 03:02:57.264   604  1468 I chatty  : uid=1000(system) Binder:604_C expire 5 lines
03-17 03:02:57.328   604  1450 I chatty  : uid=1000(system) Binder:604_A expire 4 lines
03-17 03:02:57.356   604  1437 I chatty  : uid=1000(system) Binder:604_9 expire 2 lines
03-17 03:02:57.434   604  4255 I chatty  : uid=1000(system) Binder:604_20 expire 1 line
03-17 03:02:57.471   604   700 I chatty  : uid=1000(system) ConnectivitySer expire 6 lines
03-17 03:02:57.549   604   958 I chatty  : uid=1000(system) Binder:604_4 expire 1 line
03-17 03:02:57.550   604  4221 I chatty  : uid=1000(system) Binder:604_1C expire 1 line
03-17 03:02:57.702   604   616 I chatty  : uid=1000(system) Binder:604_2 expire 1 line
03-17 03:02:57.767   604  4197 I chatty  : uid=1000(system) Binder:604_1A expire 2 lines
03-17 03:02:57.869   604  4034 I chatty  : uid=1000(system) Binder:604_13 expire 2 lines
03-17 03:02:57.922   604   615 I chatty  : uid=1000(system) Binder:604_1 expire 2 lines
03-17 03:02:58.437   182 13349 V vold    : After drop_caches
03-17 03:02:59.608   604 13370 I chatty  : uid=1000 system_server expire 4 lines
03-17 03:03:01.810   182 13349 I vold    : create took 26748ms
03-17 03:03:01.810   182 13349 I vold    : drop took 1763ms
03-17 03:03:01.810   182 13349 I vold    : run took 3262ms
03-17 03:03:01.810   182 13349 I vold    : destroy took 109ms
03-17 03:03:01.811   182 13349 D vold    : Starting trim of /cache
03-17 03:03:01.903   182 13349 I vold    : Trimmed 177926144 bytes on /cache in 92ms
03-17 03:03:01.926   182 13349 I vold    : Benchmarking /cache/misc/vold/bench
03-17 03:03:15.692   182 13349 V vold    : Before drop_caches
03-17 03:03:16.027   182 13349 V vold    : After drop_caches
03-17 03:03:17.212   182 13349 I vold    : create took 13765ms
03-17 03:03:17.213   182 13349 I vold    : drop took 335ms
03-17 03:03:17.213   182 13349 I vold    : run took 1124ms
03-17 03:03:17.213   182 13349 I vold    : destroy took 60ms
03-17 03:03:17.214   182 13349 D vold    : Starting trim of /efs
03-17 03:03:17.215   182 13349 I vold    : Trimmed 0 bytes on /efs in 1ms
03-17 03:03:17.218   182 13349 D vold    : Starting trim of /persist
03-17 03:03:17.218   182 13349 I vold    : Trimmed 0 bytes on /persist in 0ms
03-17 03:03:23.162   604   619 I chatty  : uid=1000(system) android.bg expire 9 lines
03-17 03:04:35.778   604   623 I chatty  : uid=1000(system) batterystats-sy expire 18 lines
03-17 03:14:59.942   604  4037 I chatty  : uid=1000(system) Binder:604_16 expire 1 line
03-17 03:18:18.339   604  1454 I chatty  : uid=1000(system) Binder:604_B expire 3 lines
03-17 03:28:18.130   604  4253 I chatty  : uid=1000(system) Binder:604_1E expire 3 lines
03-17 03:28:33.131   604   620 I chatty  : uid=1000(system) ActivityManager expire 11 lines
03-17 03:29:13.142 13644 13669 V FingerprintManager: FingerprintManagerService was null

I don't know much about the Android OS, but this looks like it could strongly hint the activity of the phone at that time. I assume drop_caches refers to /proc/sys/vm/drop_caches. Perhaps some system process operating on /cache triggers a behaviour in K9 which is not a crash but causes it to reset the preferences database?

I restored the files from backup:

serranoltexx:/data/data/com.fsck.k9/databases # kill 5015
serranoltexx:/data/data/com.fsck.k9/databases # rm preferences_storage*
serranoltexx:/data/data/com.fsck.k9/databases # cp /sdcard/preferences_storage* .

serranoltexx:/ # restorecon -Rv /data/data/com.fsck.k9/
SELinux: Loaded file_contexts contexts from /file_contexts.bin.
SELinux:  Relabeling /data/data/com.fsck.k9/databases/preferences_storage from u:object_r:app_data_file:s0 to u:object_r:app_data_file:s0:c512,c768.
SELinux:  Relabeling /data/data/com.fsck.k9/databases/preferences_storage-journal from u:object_r:app_data_file:s0 to u:object_r:app_data_file:s0:c512,c768.
SELinux:  Relabeling /data/data/com.fsck.k9/lib from u:object_r:app_data_file:s0 to u:object_r:app_data_file:s0:c512,c768.

and verified that this actually fixed the problem; I was able to launch K9 again and continue where I left off.

Whilst I had adb connected, I did some trivial research and found the command pm trim-caches. I tested if this trivially caused failure of k9, and it didn't, so I presume it's something more sinister.

serranoltexx:/data/data/com.fsck.k9/databases # pm trim-caches 192M           
serranoltexx:/data/data/com.fsck.k9/databases # pm trim-caches 256M           
serranoltexx:/data/data/com.fsck.k9/databases # pm trim-caches 512M           
serranoltexx:/data/data/com.fsck.k9/databases # pm trim-caches 768M           
serranoltexx:/data/data/com.fsck.k9/databases # pm trim-caches 1024M          
serranoltexx:/data/data/com.fsck.k9/databases # pm trim-caches 1500M          
serranoltexx:/data/data/com.fsck.k9/databases # pm trim-caches 2000M          

This is the second failure I've had in a few months. I've generally been running out of space on this phone more and more and, given the cache information above, perhaps there is a relationship between the two.

[msvi]

Not much information I can add, but after more than two years without problems I stumbled across the same bug this morning. K-9 mail showed the welcome screen and all settings and account information were lost (version 5.403).

[cdpxe]

@msvi: is your phone running out of free space? (this would somehow link to the info that came from previous posters). Also, my phone ran out of space. So maybe, this k9mail bug is related to that situation.

[msvi]

No, I have 3.3G left on internal storage and another 115G on the SD card.

[aooprojects]

just adding a me2 here to show that is a real issue that should be solved. happend to my phone ( stock image, non rooted Moto G4) again last week. I thought it might be a permission problem with the SD card that's formatted as internal storage as other Apps like file managers are complaining about that but then my mother brought her tablet with the exact same issue - all K9 settings/accounts gone and only the welcome screen shows. but this one doesn't even have an SD slot. a Nexus 10 with everything stock to the latest update (Marshmallow I think). she wouldn't be able to tinker with the system

[Irfy]

Happened to me too, for the first time after several years of usage.

K-9 5.600
Nexus 6P
Android 7.1.2 (LineageOS 14.1-20180219)

[kivi]

Happend to me too, first time after long time usage and I use it often.

Android 7.1.1
K9 5.600

[Whichcraft]

Just stop to use this piece of crap. Bluemail is a good replacement. Exchange, imap, all good.

[crazy4chrissi]

@Whichcraft Yes, you will never loose your settings with Bluemail, as they store your settings including password on their server [1] (Link is German only and from last year, not sure if they changed it). If you prefer that, go for it.

But I guess the typical users of K-9 want an open source solution that keeps their data private.

[1] https://www.kuketz-blog.de/blue-mail-android-mail-app-versendet-login-passwort-zum-e-mail-konto/

[sanistaal-bbc]

Happend to My k9 too for the first time in years. Sad that I will have to find another client now

[aooprojects]

maybe my experience sheds some light on this. it never happened to me again since I have a new phone and here's the major difference to my old phone: it has no micro SD slot so I couldn't format the SD card to be used as internal storage like I did on my old phone. I think this is a rather rare setup anyway and that's why it doesn't affect more users but exactly this seems to break things as the Android support also seemed rather sketchy at times as(also in file manager Apps) very few phones have micro SD support and the user base for this configuration is pretty much non-existing so Google doesn't put a lot of effort into this feature perhaps

[realkinetix]

Each phone I've had has not had an SD card slot. It really shouldn't be a direct cause of the issue.

[ygoe]

So I haven't found this issue in my search. Interesting that this is an open issue for years already. I'm using K9mail for years (don't know how many, up to 10 years maybe if it exists so long) on 3 or 4 devices and never had this issue before. It happened a few days ago, I don't know when. I also don't have any setup at hand to use the logging, especially since it isn't accessible when you don't have any accounts anymore.

Haven't read every comment here in detail, but was it assumed that K9mail being killed while writing something may corrupt the settings file? Why not write to a copy and rename when it's done? This has been a proven way to reliably write files. Also, if this bug cannot be fixed (for unknown reasons, it seems), an automatic backup of the affected file might be a good quick workaround to prevent users from losing access to their e-mail at any time.

I never experienced any data loss in this scale with another Android app all that time. Seeing I'm not the only one makes me seriously question the code quality of K9mail.

I'll have to set up everything again now and try exporting and importing the settings. Otherwise I'll have to find another e-mail app to rely on e-mail access. A quick search didn't bring up anything interesting. So I might have to consider Android unable to use e-mail. Or give Outlook a try again.

To complete my report: This is the latest version on Google Play (the version isn't accessible without accounts either) on Android 7.1.1, Sony Xperia Z5 compact, with enough free space in internal memory and SD card.

[BitBernd]

K9 lost its data again this morning

only infos i can give:

Version: 5.600
avrg RAM usage in last hours before wipe: 99MB
K9 still uses the same amount of storage space as before the "wipe" , ~500MB
Android Version: 9 with Security Update 1. July 2019

[fbrausse]

Same happened to me just now. Phone orderly shut itself down due to not enough power. After rebooting K9 v5.600 doesn't know any settings or accounts anymore. This phone is not rooted and has a physical microSD card.

[chrisdane]

Same here: normal restart of phone (not a restart after updates or battery drainage); opening k9mail -> Welcome screen. 2 buttons: "Import settings" and "Next" for setting up a new mail account.

• Fairphone 2
• OS: Fairphone Open 19.11.2 (Android 7.1.2)
• k9mail 5.600
• 2 mail accounts were configured in k9mail and used for ~1year

[aooprojects]

so it happened again last night! different phone from the last time I commented here. Android one device with Android 10. I had debug logging enabled as I've been burned several times and I jumped through the ADB hoops described here:
https://k9mail.app/documentation/debugging.html

but this was just wasted time as there's nothing in the log from last night. it starts this morning with this:
--------- beginning of main 06-11 10:55:05.605 6319 6319 I chatty : uid=10159(com.fsck.k9) expire 201 lin es 06-11 10:55:05.654 6319 11868 I chatty : uid=10159(com.fsck.k9) expire 55 line s 06-11 10:55:05.711 6319 11869 I chatty : uid=10159(com.fsck.k9) expire 11 line

which makes me wonder if K9 crashes so hard that it's restarted by Android so the PID changes and the log is lost forever. it seems to be impossible to provide information as long as K9 doesn't write permanent logs in case of an exception

[danielb42]

Happened to me 4 weeks ago and then again today on Galaxy S9/Android 10/current k9.

Since this defect is known for at least five years now I probably don't need to bother with pulling logs anyways..

[Diggensag]

Bump @cketti, this bug is still alive. The more info tag is 5 years old now.

[cketti]

@Diggensag: And so far nobody has been able to reliably reproduce the bug, including myself. If you believe you can fix the bug with the information provided so far, please go ahead. We'd love to get this fixed.

[ygoe]

I'm sorry, I'm not a specialist for Android development. But is this really all there is? The devs are unable to produce a reliable way to store data like every other app developer does? Maybe they should replace the entire mechanism that is related to that. The main point is: this bug isn't reproducible. And it looks like it never will be. It happens way too rarely (and collecting the necessary telemetry is impossible for the affected users) to ever find the pattern behind it. But the effect is catastrophal! This leads to a bad risk rating, justifying large measures. If you just go on waiting like this, that's just irresponsible IMO.

[con-f-use]

At least some mitigation would be nice. Add some code to detect that it has happend and offter to restore a backup or something.

[martin-riedl]

happened to me as well now ... and others I know as well recently.

[andresteinert]

Same here, happened today...

[aooprojects]

so it happened again last night and after the bad experience I described the last time I actually got up and tried to gather data as described in https://k9mail.app/documentation/debugging.html

I still have a screen open and here's what happened:
$ adb shell ps -A | grep k9 u0_a378 14683 689 5362976 146448 0 0 D com.fsck.k9

getting the log for that pid:

$ adb logcat -d --pid=14683 > k9-log-17-Nov-2020.txt

but it's useless:

$ ls -l k9-log-17-Nov-2020.txt -rw-rw-r-- 1 xxx xxx 0 Nov 17 23:24 k9-log-17-Nov-2020.txt

a zero byte file! so I suppose K9 crashed so hard it had to be restarted by Android and got a new PID

I've also got a complete device log from the logcat App but I won't post that publicly since it contains quite a bit of personal information

there are also no K9 useful logs in there but a couple of crashpad lines like:

11-07 20:17:26.003 30757:30757 F/crashpad] )iyJ2*?dqq.x-}MhR)]]&4F}M_8{z3R$_@BDX[ATOJP@8,}MY5pJ3Eb72N!O7{zW0Zfx{i}i6.De(kdFTi%K2b.x!Zl;J2@wgfJH+W12xmL.vb$:V[t+yENbI$A_J,9a+}o!|YkA1E-?Za8u+]Lrk\c{&AAa13"K*O&#!ExF#nx8M4MvbCbftfq_J{>g

wd*$X{(T=oKVaQ&!nbnIK57kJ.1m>gU~0mZ1_I'xPzHP}HjmxB2)_Nc~@@]g--gH'x&zxo&rDMH2bWX+ePcapWJ47+&,?VxA)'e3g0"eCnfB\#ULZbbrIO86?Lu/[R=1!MLOBLU+',+ZkF}vZ&ILjJYL"frE\5Fw#sE5fRJa+12lz-R9rD\:p++W#$>*h\4e6l>D\wBt'FXuaof.zAmKnb&~Gk!s^c

[11-07 20:17:26.003 30757:30757 F/crashpad]
!xc2i5bQ,)eN=+H\kFM[_{7nVL=HNfJhPO>,DI6K_3PeQf/H6tcLSK@zdoTu(w>?!)uEOu+hZCoJ\%me{'dUTV8Bs9(bqTAt()P_o?#HnG9v6}>-SfY9m>Me/T7Ejq>8hg-Yldt"GH8LU7;_2#!1^S,KE{!CY:ru]58,&d#Oldueh1#N}qX[IKfs2e^"|XhU<J"2eD/;:~s>PEKa?YL:BalR!WsO/Hrq!4JX^1x5Z4L}QboUcGBc8S2+4J>;Crp2~sg\j>+[|N
OP)OMEoR&d6WEcr+;,ZF-AVv;OOrxyB&/8,vdy"aIueT7'0lKx,q(jeH7D56('HUPz}PZf.x5Gs?EvT#3:qLe6pf6aq0]?8M){4:v_nv$!Myru)a8,$sn<+Aq7Gs]r)!rWja/]5-.oi_58g&c4}P1elc=!}5rP$mPjXt&jB85I(aP9zEK00-W`Dw3qiHMum^\q,Wk<#Iv_-Gra:CvefV6

`

not sure if it even has something to do with K9

my grep for k9 on this log file only matched stuff like:

etCGroupState():tofreeze=false, uid:10378 pkg:com.fsck.k9 --->>> ok reason:start perform not hasTimeout broadcast for action = android.os.action.DEVICE_IDLE_MODE_CHANGED at com.fsck.k9.mail.filter.PeekableInputStream.peek(PeekableInputStream.java:36) at com.fsck.k9.mail.store.imap.ImapResponseParser.readResponse(ImapResponseParser.java:36)

which came much later

we need better tools to debug this!

[fnune]

Happened to me today as well!