Cardano-wallet: Password encryption specs

[Fell-x27]

Hi! are these methods covered by some abstract specs?

https://github.com/input-output-hk/cardano-wallet/blob/5af6e35a4f70ca8e1cebe413b19a99439e04addb/lib/core/src/Cardano/Wallet/Primitive/AddressDerivation/Icarus.hs#L421

https://github.com/input-output-hk/cardano-wallet/blob/5af6e35a4f70ca8e1cebe413b19a99439e04addb/lib/core/src/Cardano/Wallet/Primitive/AddressDerivation/Shelley.hs#L370

https://github.com/input-output-hk/cardano-wallet/blob/a4fd49e3f36596f68fbf7d327f89aa54e2e19f23/lib/core/src/Cardano/Wallet/Primitive/AddressDerivation/Byron.hs#L372

If yes, could somebody share any reference specs? If no, could somebody explain the code inside? I can read and use many programming langs, but Haskell is not the one of them :(

[KtorZ]

Hi @Fell-x27 , they aren't. That format is an implementation detail of cardano-wallet and cardano-crypto.
In brief:

Public keys are stored in the database simply as hex-encoded bytes. Public keys (abbrev. pub) are 32 bytes and stored alongside a chain code (abbrev. cc) of also 32 bytes that is used in key derivation. So, in db you have something that is roughly hex(pub|cc).

Privates keys (abbrev. prv) follow the same principles but are 64 bytes in length. They are stored encrypted; alongside their corresponding public key and chain code, as well as with a salted hash of the passphrase encrypting the key (for passphrase verification). So, in db you have something that is roughly hex(chacha(prv)|pub|cc|pbkdf2_sha512(passwd)).

[Fell-x27]

Hi @KtorZ ! Thanks!

hex(chacha(prv)|pub|cc|pbkdf2_sha512(passwd))

Just exported 2 encrypted private keys with the same mnemonic but different password from the wallet's db. Shelley.
256 symbols of hex. The first 128 symbols are the different, but the last 128 are the same. But your "rough" description says that last parts should be different. So can you describe more accurate encryption scheme for every case linked above?

Let me explain. I want to move "private" cryptography to the client from the server, and I need some kind of painless(for user, not for me) migration. The best way - just re-implement cardano-wallet storing/retrieving algorithm to

1. Do not force user to recreate wallet.
2. Have an ability of switching between "old" and "new" modes if needed.

I've tried to reverse cardano-wallet's code and that's the final point I've reached without any help. I know that the code above looks very simple, but I can't re-implement it without any clue or learning haskell syntax/structures just for this case.

[KtorZ]

Sorry, the information above comes from the top of my head. The passphrase is most likely stored before the encrypted key. So the first 128 characters are actually the passphrase hashes which would be different even if the passphrase were the same because of the random salt.

[Fell-x27]

chacha(prv)

What do we use as a key and nonce for chacha? I guess the key is passwd, or another hash of password(not the stored one), but what about nonce? I've found a string, 'serokellfore' (dark legacy?) that is acting like nonce for an old byron scheme probably, but what about Icarus and Shelley?

[KtorZ]

https://github.com/input-output-hk/cardano-crypto/blob/master/cbits/encrypted_sign.c#L53-L71

[Fell-x27]

Please, tell me that this c-code works the same way as cardano-wallet's implementation.

[KtorZ]

This C code is exactly the cardano-wallet's implementation.

[Fell-x27]

@KtorZ , thanks for the reference. I've implemented it. Works fine, does the same as C-code with the same params. I can encrypt and decrypt keys this way. But I can't decrypt keys from cardano-wallet database :)

The mnemonic: asthma seed food drink social income bar canal race spare omit fashion nut cushion green
The password (sha256-hashed "Q1w2E-345POIUYTd"): cc8bb592ac4ef4f1ded67bad5b86a33cc1e61d5a8ae8a55806a1af24ce9d7924

Salt and nonce are unknown and don't even matter.

my result: 60a57276fb96310bf6f6686886b39d4088ab76da62b48e1e0fb6c3cb920a0ad549a0b0b1b93407c292ea68ff43f3f18a75c7de50253e144e328d85b588835e28870e80ab658196a8cb534eca6d9a4b403e410158c9c4960068571f39d5d3bb3931cbe787ce0bc0d52a94edc2e97ec7e2a1975b81c3974424f7b18a9841095399

cardano-wallet's result:
57a5325a56a26372dd6170cd98d61c29d909a16a354b314a57090d4c52680240a50b8292c12bc3cd566cf9e2a0e932ff6697fdf49395ef20b6dbd377fd14e8d2870e80ab658196a8cb534eca6d9a4b403e410158c9c4960068571f39d5d3bb3931cbe787ce0bc0d52a94edc2e97ec7e2a1975b81c3974424f7b18a9841095399

Looks pretty similar.
But I got different results after decryption. For my version I got my keys. For cardano-wallet's one I got binary mess.

I'm sure that algorithm is ok. So the only way to get binary mess - call memoryCombine with a _wrong password_.

Does cardano-wallet do any changes with the password before encryption? Some hashing?

[Fell-x27]

Well, the haskell code says that yes.

serializeXPrv (k, h) = ( hex . unXPrv . getKey $ k , hex . getHash $ h )

What hash function is inside getHash?

[KtorZ]

@Fell-x27 how exactly do you convert the recovery phrase into a root key at the moment?

edit The public key look the same though, so your approach is probably correct.

[Fell-x27]

@KtorZ , The problem is very simple as I can see.

My case: "memoryCombine(key, my_pre_hashed_pass)"
Your case: "memoryCombine(key, hash(my_pre_hashed_pass))"

So, the last step is repeating "your" hashing. to get the real pass used for encryption with the cardano-wallet.

[KtorZ]

Which wallets? Old legacy Byron or new recent wallets?

[Fell-x27]

Both I think.
For example showed above it was a recent wallet shelley/icarus like. But I will support old legacy byron either, so if there is a some difference, would be nice to know about it.

[Fell-x27]

I see what you're talking about . There is some difference for it:

serializeXPrv ((ByronKey k _ (Passphrase p)), h) = ( hex (unXPrv k) <> ":" <> hex p , hex . getHash $ h )

But it touches only key preparing stage. Instead of hex(prv) we get hex(prv):hex(pass), am I right? But getHash function is the same and I need to know what type of hash it provides to reproduce it properly :)

[KtorZ]

There's no primary transformation of the passphrase done before encryption on the Shelley/Icarus wallets except that the passphrase is encoded as UTF-8 bytes.

[Fell-x27]

I'm not a haskell programmer, but isn't this code:
hex . getHash $ h
the same as 'get hex of hash of h'?

[KtorZ]

Yes and no. To give you some explanation about these bits of code:

• This occurs when serializing the encrypted key and the hash of the encryption passphrase to the database.

• The wallet does not store the passphrase unencrypted but, a salted hash of it and even in-memory, we do not generally manipulate the passphrase more than needed so it's converted into a hash and manipulated as such. We could use a simple "string" for that, but Haskell being Haskell, we use a dedicated data-type called Hash which wraps a bytes buffer. getHash just unwraps it.

• hex is a little helper that encodes the bytes buffer as a base16 string; that's because SQLite does not support storing raw bytestring easily, so we typically store bytes buffer as base16 strings.

Yet again, we do not store the encryption passphrase, but a hash of it. Obviously, if you try decrypting the encrypted key with that hash, it'll fail. The hash is simply used to verify the actual passphrase when trying to decrypt the key. Since pretty much any bytestring (of the right length) is a valid Ed25529 key, you could in principle try decrypting with anything and you would get a valid Ed25519 key (although not the one you expect). Hence, being able to verify that the provided passphrase is the right one is essential.

[Fell-x27]

Wait. Maybe I read the stored data wrong way.

There is a stored data in the database.

57a5325a56a26372dd6170cd98d61c29d909a16a354b314a57090d4c52680240a50b8292c12bc3cd566cf9e2a0e932ff6697fdf49395ef20b6dbd377fd14e8d2870e80ab658196a8cb534eca6d9a4b403e410158c9c4960068571f39d5d3bb3931cbe787ce0bc0d52a94edc2e97ec7e2a1975b81c3974424f7b18a9841095399

It's 128b length. Last 64b is xpub. So we have only 64b encrypted part.
57a5325a56a26372dd6170cd98d61c29d909a16a354b314a57090d4c52680240a50b8292c12bc3cd566cf9e2a0e932ff6697fdf49395ef20b6dbd377fd14e8d2

MemoryCombine _output length == input length_.

So If you encrypt 64b prv (96b xprv without 32b of chaincode, because we moved it to a public part), there is no place for password hash attachment.

1) If wallet stores password hash there, it encrypts _not the 64b prv_, but what does it encrypt exactly? Entropy?
2) Is encrypted data an array of bytes, or utf-8 hex-represented string?
3) Which byterange should be sliced-off and decrypted actually in this case?

I'm sure the problem is just a tiny difference in the data preparing, no more. Not in the encryption algo etc. It's so close.

[KtorZ]

1. I guess you're referring to the root column of the private_key table. The passphrase hash is stored in another column called hash.

2. It's an array of bytes, encoded as a base16 string.

3. If you're referring to the root column, then it seems like you've done the right thing.

[Fell-x27]

It's an array of bytes, encoded as a base16 string.

I meant what form it has when passed to MemoryCombine? Does wallet pass prv as hex-string, or as byte-array?

[KtorZ]

byte-array.

[Fell-x27]

But password is passed as a string? If password is "qwerty", this string will be passed, not it's str->byte->hex representation, right?

[KtorZ]

The string is passed via the HTTP API, encoded as JSON. JSON data must be UTF-8 encoded, so strings in JSON are basically an array of bytes, UTF-8 encoded. So I am not sure to get your question?

[Fell-x27]

raw text: qwerty123^&*
urlencoded for HTTP: qwerty123%5E%26*
bytearray raw version: [113, 119, 101, 114, 116, 121, 49, 50, 51, 94, 38, 42]
hex raw version: 7177657274793132335e262a
bytearray urlencoded for HTTP vesrion: [113, 119, 101, 114, 116, 121, 49, 50, 51, 37, 53, 69, 37, 50, 54, 42]
hex urlencoded for HTTP version: 7177657274793132332535452532362a

There are 6 ways to represent the same string. But all of them will cause different encryption result...

Listen, could you suggest some IDE+debugging tool for haskell? I think it would be more efficient to just trace the wallet's code, than waste your time here :)

[Fell-x27]

@KtorZ , I did it. I. Did. It. :rocket:

I've found the problem. It was my fault. I compiled my memory_combine version with wrong chacha rounds count. Just re-checked original code, fixed my own and successfully decrypted my keys from cardano-wallet database with my password.

Thank you for your patience!

[KtorZ]

:pray: Glad to hear.