Andotp: [Testing] Password-based encryption

[flocke]

Please report all bugs encountered in the test version for password-based encryption here!
Provide a detailed description of how you encountered/reproduced the bug and a logcat!

Download of the test version:

• v6 (current, beta): https://cloud.shadowice.org/s/a5RE3UovB6BuFjf
• v5 (beta): https://cloud.shadowice.org/s/y1Ckm2Ar5ewGBEk
• v4 (beta): https://cloud.shadowice.org/s/0vdIcBNKBA6wZ4n
• v3 (beta): https://cloud.shadowice.org/s/0keZJG5jzBIG4st
• v2 (alpha): https://cloud.shadowice.org/s/Ek6dxFqOyg60nYh
• v1 (pre-pre-pre-alpha) https://cloud.shadowice.org/s/32vZoc5Qojd53OE

You will have to uninstall previous versions from F-Droid or Google Play before installing the test version.

[RichyHBM]

So far so good, and titanium seems to be working as expected

[donarrock]

Sadly for me the app doesn't even launch.

Device: LG G4 (H815)
ROM: latest official LineageOS (lineage-14.1-20171107-nightly-h815-signed.zip)
logcat: https://pastebin.com/5ecgMYHz

[RichyHBM]

@donarrock was this the first time you launched this version? The logcat doesn't make it look like its anything specific with andOTP code, but rather a missing binary.
I wonder if this build is just for arm and that LG is x86?

If you are able to, would you try checking out the custom encryption branch and building to your device yourself?

[flocke]

That shouldn't be a problem. The AVDs I usually use to test andOTP are mostly x86 machines and it always worked there ...
I'm gonna have a look at the logcat when I am back at home tomorrow.

[donarrock]

Thanks for the replies. It seems to be specific to this LineageOS build. I quickly tried a newer unofficial build, and there it didn't crash.

[flocke]

Version 2 is up: https://cloud.shadowice.org/s/Ek6dxFqOyg60nYh
This should be much more stable and you now can also change the authentication method while password-based encryption is enabled.

[RichyHBM]

I seem to be losing all tokens when I open the app with the latest version, I am also seeing the same exception from a missing binary as @donarrock was, so this may be a red herring, I can try installing a newer nightly for Lineage when one is available (usually fridays) however I also see that exception when I have it stored as keystore and the tokens are fine then.

Here is the full logcat https://pastebin.com/i3iyqvFw

[flocke]

That is very strange, I just checked again on my two devices and switching from KeyStore to password-based worked very well in both cases. I see the binary error as well some times, but it still works.

What I am more concerned about is the crypto error (BAD_DECRYPT). Something seems to be going wrong with the decryption of the files.
Can you tell me exactly the steps you took for migrating from KeyStore-based to Password-based encryption and at which point the error first appeared?

_Edit_: Still works after installing the latest LineageOS nightly (20180101) on my test device.

[RichyHBM]

OK, further testing it seems to happen when ever you close the app from the settings activity without having gone back to the tokens activity, if you go back to the main activity after switching to pass/pin based encryption its all fine. Maybe until you go back to the main activity the entries aren't getting saved with the new key? So when you reopen the app the entries are still encrypted with the keystore key?

So, from fresh install, open app, add a new token, go to settings, enable authentication (I used pin), set a pin, change encryption to pass/pin based, now close the app (swipe it from the tabbed apps, of what ever it's called), upon reopening the tokens won't be there

[flocke]

That sound about right. But in the new version the Settings Activity should close automatically after changing the encryption (and save the database when doing so).
Maybe I add a Toast to tell the user that some saving is going on and the app should not be closed.

[RichyHBM]

The settings page doesn't seem to close for me

What about an ongoing notification (on the status bar) whilst it is saving, I've seen a few apps do this and I think it prevents android from closing the app

[flocke]

Stupid question, but are you sure you installed the correct version?
Do you have the new CredentialsPreference when setting up the authentication? Or do you still have the old ListView + a separate password preference? It should already quit the SettingsActivity when you change/set the password/PIN (even if the encryption is still set to KeyStore, a bug in the current test version).

[RichyHBM]

What does the credentials preference look like? 😅
I do have the list view + password preference...

I built and installed from the latest code when you messaged on the group :s
I will try rebuilding

[flocke]

The new CredentialsPreference looks like this, combining both password and authentication selection in one preference:

[RichyHBM]

OK, yea I just had the wrong version 😅 all good now!

[barul42]

It seems to be working here, w/ password encryption enabled.
Added an account with the details I saved, quit the app and start it again, everything is still there ! :-D

[flocke]

Nice to know it works. I improved the switching between the encryption types a bit further and added a lot more fail-safes. I'll post a new test version in the evening and if there are no further bugs reported this will most likely be the final one.

[nomisma-qt]

running latest with password (PIN) encryption, added 2 entries. The coming 24 hours will tell. If necessary i can wipe cache and dalvik too, without upgrading the ROM (no new ROM avail at this time). I can also provide logs if something goes wrong. thx

[nomisma-qt]

update. with 5 entries stored using password (PIN) encryption, rebooted system to recovery, wipe dalvik + cache, reboot system:

All entries still there!

looks promising..

[flocke]

Just uploaded the (hopefully) final test version: https://cloud.shadowice.org/s/0keZJG5jzBIG4st
Switching between encryption modes and changing the password while password-based encryption is enabled should now be a lot more stable.

[RichyHBM]

Clicking on authentication to open the credentials preference with no tokens causes a crash due to not finding secrets.DAT file

[RichyHBM]

Hmm, also getting a crash on clicking authentication with tokens.

"Attempt to call Dialog.getCurrentFocus on a null object" from CredentialsPreference.java:225

I'm using a custom build from 5059abf

[flocke]

The error with the secrets.dat file is always thrown if no database is found, but that one is harmless.
The dialog error on the other hand could be the reason, let me look into that.

[nomisma-qt]

planning to test backup and recovery using latest test version. I won't bother with testing backup/restore or anything with keystore mode for now, i assume it's not resolved as it is an LOS issue.

addition, by now entries should have been lost with my heavy use. No entries lost so far. Test version 2 already a major relief. thx

[nomisma-qt]

backup and restore using encrypted .json file between test.v2 and test.v3 successful. Even with full deletion of app cache + storage and/or cache/dalvik wipe in between. Always successful.

[RichyHBM]

@nomisma-qt are you getting an app crash when you click on authentication on the v3 test app?

[nomisma-qt]

Yes. Trying to set authentication in settings in v3 = crash OR return to main display

I have to stop for today, as all crypto is exploding in the first global realtime hyperinflational collapse of the 21st century. Previous historical example is Weimar Germany.

Dec 24 total crypto market cap was $556B. As i write this it's 728. The process has 2 characteristics:

1: It will never stop
2: It will never reverse

I know this is not the right forum. But do not for a moment think this is a bubble. Do not think it's going away. I have been studying this on the highest level all my life. We are facing a catastrophic global event on a grand scale. Prepare accordingly. I am an expert, and have many answers. But at the same time i face destiny alone, and vulnerable.

thx for all your work. goodnight for now.

[flocke]

@nomisma-qt Could you get me a logcat? I assume it is the Dialog error as well.

[nomisma-qt]

Damn. Let me try. no promises.

[flocke]

I just tried on my 'production device' (I got v3 installed there as well to see if I get some problems with daily usage) and everything works fine here.

[nomisma-qt]

tell me the format and what log you need, i assume system. I run OS monitor. Second option is some FOSS from F-droid updated 6 months ago. What do you use?

[nomisma-qt]

I start andOTP, go to settings -> authentication. The app return to main screen.
I do it again, the app crashes.

For logs please give me the link to the logcat app that you are running. I haven't had time the last years to keep up.

[RichyHBM]

You need adb on your computer I think.

Here is my logcat, or atleast all of it untill the stacktrace goes into android land

01-03 22:18:05.837 6563-6563/org.shadowice.flocke.andotp E/AndroidRuntime: FATAL EXCEPTION: main
Process: org.shadowice.flocke.andotp, PID: 6563
java.lang.NullPointerException: Attempt to invoke virtual method 'android.view.View android.app.Dialog.getCurrentFocus()' on a null object reference
at org.shadowice.flocke.andotp.Preferences.CredentialsPreference.updateLayout(CredentialsPreference.java:225)
at org.shadowice.flocke.andotp.Preferences.CredentialsPreference.onBindDialogView(CredentialsPreference.java:133)
at android.preference.DialogPreference.showDialog(DialogPreference.java:300)
at android.preference.DialogPreference.onClick(DialogPreference.java:277)
at android.preference.Preference.performClick(Preference.java:994)
at android.preference.PreferenceScreen.onItemClick(PreferenceScreen.java:249)
at android.widget.AdapterView.performItemClick(AdapterView.java:310)

[RichyHBM]

Seem to have fixed it

if(getDialog() != null) {
    UIHelper.hideKeyboard(getContext(), getDialog().getCurrentFocus());
}

My guess is that when getDialog is called at the point at which the dialog is created it is null, but when the dialog is opened with no credentials set the leyboard doesnt show anyway so this fix doesnt change anything. It also hides the keyboard if you change from one of the other credentials to none

[nomisma-qt]

can't get ADB. Too busy. I have to stop my life and move abroad to the east starting tomorrow. For now my TOTP entries are staying. It's all i need. I been trading crypto since one bitcoin was worth $0.01

God bless you all for your help. Please head my warnings. We are facing the destruction of western civilization. Good luck to all.

[RichyHBM]

@nomisma-qt I think we have what we need now, good luck with your travels/move. If only I had of gotten into crypto earlier...

[nomisma-qt]

RichyHBM I know this is not the correct forum, and i will stop commenting here, but with my experience and knowledge of history, finance, money and psychology, i can assure you with absolute certainty, that it is not too late for you to position yourself.

Don't think it's too late. You don't need to buy a whole bitcoin to get into crypto. You can still buy a whole litecoin for under 300 dollars. You can still buy a whole digibyte for 8 cents.

The reason you feel it's too late for you and you just have to sit it out, is psychological. It's nothing to be ashamed of, there is nothing wrong with you. It is something everyone experiences. I have experienced it just like you, and everyone else.

What we are observing now in the crypto markets is something very significant. It is the biggest revolution and evolution in the field of money, finance and economics in the history of mankind so far, since man first crawled out of his cave.

Do not fear it, embrace it. If you have children they will thankyou if you do. I had to write this, even though it has nothing to do with this thread. Good luck to you wonderful people. And thankyou for all you do. Regards, mike.

[flocke]

@RichyHBM seems to be right, this is a very easy fix. Will provide a fixed build later today.

[RichyHBM]

@nomisma-qt oh don't worry, I have already positioned myself somewhat, I just wish I had of got some of those $0.01 BTC 😉

But yes, this probably isn't the right place to talk, feel free to message me on telegram if you want to talk more

[flocke]

New build with the crash fixed (and some other refactoring): https://cloud.shadowice.org/s/0vdIcBNKBA6wZ4n

[gigabytedevelopersinc]

Wow! So far so good!
No issues yet!
Works fine for me over here. When are you making a PR and merging this?

On Jan 4, 2018 18:21, "Jakob Nixdorf" [e-mail] wrote:

New build with the crash fixed (and some other refactoring):
https://cloud.shadowice.org/s/0vdIcBNKBA6wZ4n

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/andOTP/andOTP/issues/94#issuecomment-355343679, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AY_wpxkZ1j0xvaVnr-SHaGMQcBUA29dEks5tHQi2gaJpZM4RPOTu
.

[flocke]

I plan on merging this in the beginning of next week if no further issues come up. Since there are a lot of new strings to translate with this I need to give the translators some time afterwards so I expect to release the new version in about 2 weeks.

[gigabytedevelopersinc]

Cool. Haven't experienced any issues with it yet!

[gabm]

well I'm sorry - I'm not even able to start the app - if FCs immedately ... :/

here is a logcat: https://pastebin.com/iJ4f7nLu

[flocke]

@gabm I think I see where the problem is, I will upload a new version for you this evening with a potential fix.

[gabm]

@flocke I'll be happy to test ...

[flocke]

@gabm Could you try and see if this version fixes your crash: https://cloud.shadowice.org/s/y1Ckm2Ar5ewGBEk

[gabm]

unfortunately not... still crashes.. i can provide a logcat tomorrow...

On 8 January 2018 18:46:28 CET, Jakob Nixdorf [e-mail] wrote:

@gabm Could you try and see if this version fixes your crash:
https://cloud.shadowice.org/s/y1Ckm2Ar5ewGBEk

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/andOTP/andOTP/issues/94#issuecomment-356040192

[flocke]

Strange, I thought I knew what was happening. I was looking at your logcat and looking at my code and was sure I knew exactly what was causing it ... seems I was wrong. A logcat would be nice.

[flocke]

@gabm Could you get me a logcat with v5? This crash is the only thing preventing me from merging this ;)

[gabm]

@flocke here we go: https://pastebin.com/j4T9QHvC

looks the same problem to me?! maybe its a problem with my setup?!

-> I'm on the latest lineage os 14.1 nightly with microg

[RichyHBM]

I had a similar error when building on a wrong commit.. @gabm, assuming you have a backup could you fully uninstall andOTP, reboot your phone, and redownload/reinstall v5?

[flocke]

The strange thing is that all errors come from the loadEncryptionKeyFromKeyStore() function which should catch them ...

[RichyHBM]

That makes me think that something weird is going on with the install, thats why I suggest uninstalling completely first

[gabm]

I tried the uninstall procedure, but unfortunately without luck :/ still everything the same ..

On 9 January 2018 22:19:00 CET, Richy HBM [e-mail] wrote:

That makes me think that something weird is going on with the install,
thats why I suggest uninstalling completely first

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/andOTP/andOTP/issues/94#issuecomment-356417305

--
M.Sc. Matthias Gabriel
Fakultät für Elektrotechnik und Informationstechnik

Technische Universität Chemnitz
Reichenhainer Straße 70 | R. W440
09126 Chemnitz
Germany

Tel: [phone]
Fax: [phone]

matthias.[email protected] www.tu-chemnitz.de

[flocke]

Could you try wiping cache & dalvik in recovery after uninstalling the app and install it again after?

[flocke]

@gabm Another question: do you have a secure lockscreen (Pattern, Password, PIN)?

[gabm]

I uninstalled the app, cleared the cache, re-installed and still the same...

In theory I don't have a secure lockscreen set but I also tried this without luck..

Maybe you can give me some hints on where things might go wrong and later today or tomorrow I'll spin up the debugger and try to see what's going on... if u want..

[flocke]

The weird thing is that the app crashes inside a function that was specifically designed to catch all (KeyStore-related) exceptions and show the user an error message instead of just crashing ...

I don't think setting a secure lockscreen should be needed, at least I don't have one on my AVDs that I use to test andOTP. I will try without one later today on my real testing device.
Some of the stuff I found online relating to that error on the other hand suggests a secure lockscreen is required. They suggest to uninstall the app, unset the secure lockscreen (by setting it to swipe), set it to a secure variant again and re-install the app. Maybe you could try that.

[gabm]

They suggest to uninstall the app, unset the secure lockscreen (by setting it to swipe), set it to a secure variant again and re-install the app. Maybe you could try that.

no luck :/

[gabm]

as password-based encryption seems to solve problems for a lot of people, maybe u can release it anyway and we continue tracking a new issue against the new version?!

[flocke]

I'm gonna try something this evening, are you up for testing another build @gabm?
I'm just gonna try to catch the ProviderException when calling loadEncryptionKeyFromKeyStore() even though it isn't supposed to be thrown.

[gabm]

Of course I'm up for testing stuff :)

Here are my findings meanwhile, I attached the debugger and ran the code on my phone:

• when u catch the exception it passes gen.generateKeyPair();, but of course crashes later with a nullptr exception on the getCertificate call as it hasn't been generated...
• I seem to be affected by this: https://stackoverflow.com/questions/44469020/androidkeystore-keypairgenerator-crashes-on-small-number-of-devices -> in the comment someone states that he uses a Oneplus 2 (like me)
• the same error happens when I use the deprecated method using KeyPairGeneratorSpec

If I would be able to use password-based stuff instead of keypair encryption, maybe that would help me...

[RichyHBM]

I was going to say you could change the prefs with root, but that's probably going to be very complex with the password generation and stuff

What if as a first point of contact, on new installs, andOTP asked the user what encryption method to use, and only then would it do a keystore generation (if the user chooses the keystore)

AFAIK you don't have a way to launch the app in order to select password based encryption correct?

[flocke]

Yeah, I was planing something like this for the next release after the initial custom-encryption one.
But maybe I can implement it for this one already. It should be enough to show a message on the first launch and directly take the user to the SettingsActivity where he can change the encryption if he wants.

[gabm]

i think a simple A or B form would do the job..

all this leaves me a bit worried for the state of the encryption implementation in my phone..

[flocke]

I think taking the user to the setting is the best option. Just asking if he wants KeyStore or password-based wouldn't work as he also needs to set a password before enabling password-based encryption.
This is all already implemented in the SettingsActivity, so it's no use doing it again in the MainActivity.

[gabm]

yep makes sense..

[flocke]

@gabm Could you try this version: https://cloud.shadowice.org/s/a5RE3UovB6BuFjf
I tried a last thing to fix the crash, the encryption choice on first launch will come in the next version.

[gabm]

that one works! great!! .. I get an error message telling me that keystore initialisation didn't work, that any saved otp will get lost and that i should opt-in to use password based protection.

[flocke]

Nice, finally caught it all ...
Does the app work if you switch to password-based in settings? Tomorrow I will add/change the first-use dialog and probably merge the custom-encryption branch to master.

[gabm]

yep, looks good so far ...

[flocke]

It is done! I merged the custom-encryption branch to master, let's see about all those pull requests ...

[flocke]

Just an important update to all testers of the password-based encryption:

I decided to change the size of the encryption key generated from the password to match the size used in KeyStore-mode (128-bit). Due to that change you won't be able to upgrade directly from the test-version to the next release. You will have to backup, uninstall the test-version, install the release and restore.

[nomisma-qt]

Just wanted to say thanks for the most excellent work to flocke and everyone. andOTP is now the best 2FA client in the universe, hands down. Thankyou