Cardano-node: [BUG] - BALANCE CHECK FAILURE - support missing for ed25519e keys

[AndrewWestberg]

Exernal

Summary
cardano-cli fails to convert an ITN private key that is of type ed25519e. It only works for ed25519

Steps to reproduce
Steps to reproduce the behavior:

1. run

$ cardano-cli shelley stake-address convert-itn-key --itn-signing-key-file BCSH0_owner.prv --out-file itn_bcsh0_owner.staking.skey

1. see error (modified error message for security)

Error deserialising signing key: "\248\237\249\CAN#\228\v\230\171X\234w\SUB\248\227\DC4\fQ\r\ETX`k\169\251\254g\170\196C)(u\205J\182;}\243\GS\150\143\242\239H\133\SYNib\150P\238U2\223r\ACK3\SUB\NAK\SOf\GS4}"

Also note that no test code coverage was done for ed25519e keys.
https://github.com/input-output-hk/cardano-node/blob/master/cardano-cli/test/Test/ITN.hs

Expected behavior
Expect that key converts ok.

System info (please complete the following information):

• OS: Ubuntu
• Version 20.04
• Node version 1.14.2

[ilap]

You should not expose your private key, pls edit the issue by deleting the "Error deserialising signing key:...." as soon as possible

Also, ed25519e_sk can be easily converted to ed25519_sk by stripping out the last 32 bytes of the bytecodes or the last 64 characters of the hexa string of the private key (not he bech32 coded byt the bytes as hexa string).

[AndrewWestberg]

You should not expose your private key, pls edit the issue by deleting the "Error deserialising signing key:...." as soon as possible

Also, ed25519e_sk can be easily converted to ed25519_sk by stripping out the last 32 bytes of the bytecodes or the last 64 characters of the hexa string of the private key (not he bech32 coded byt the bytes as hexa string).

Not a real key

see error (modified error message for security)

[ilap]

All good then, sorry have not read it through properly.

[dcoutts]

E xtended signing keys cannot be converted to equivalent non-extended signing keys. It would need direct support in the api and cli. The new typed api should make adding support for this not too tricky.

[kevinhammond]

To follow on, this is in progress, and will be available shortly

[gacallea]

FWIW: converting extended keys with jcli is a working workaround: https://github.com/gitmachtl/scripts/tree/master/cardano/shelley-testnet#claiming-rewards-from-the-itn-testnet-with-only-skpk-keys

[AndrewWestberg]

Thanks @kevinhammond and @dcoutts . That makes sense with the error I was seeing after converting the key which is

Error while submitting tx: Failed to submit Shelley transaction: ApplyTxError [LedgerFailure (UtxowFailure (MissingVKeyWitnessesUTXOW (WitHashes (fromList [KeyHash a4b0885........]))))]

[AndrewWestberg]

FWIW: converting extended keys with jcli is a working workaround: https://github.com/gitmachtl/scripts/tree/master/cardano/shelley-testnet#claiming-rewards-from-the-itn-testnet-with-only-skpk-keys

As Duncan states above, this is not equivalent. It can be used to check the balance, but you are unable to claim rewards.

[gacallea]

my apologies, I have misunderstood this issue then.

[disassembler]

I thought this wasn't going to be an issue when we prioritized getting ITN key converter in, but we linked some official QA scripts in our documentation that created keys with the wrong type, which increase the priority of this as we have stake pool operators that used these scripts to create their keys. https://github.com/input-output-hk/jormungandr-qa/blob/master/scripts/createAddress.sh#L25

As such, raising this to medium priority.

[ilap]

E xtended signing keys cannot be converted to equivalent non-extended signing keys. It would need direct support in the api and cli. The new typed api should make adding support for this not too tricky.

Yes, you are right, I mixed up /w Bip32 to Extended.

But, I do not think that the extended can be converted to non-extended signing keys as there is a SHA-512 is involved in non-extended key and that non-extended key is just a seed for keypair generation (SHA512 and set/clear bits).

[ilap]

Yes, you are right, I mixed up /w Bip32 to Extended.

Checked, even Ed25519Bip32 cannot be converted to ITN's Ed25519Extended as they set the 3rd bit of the key[31] which makes half of the Bip32s non extended key compatible.

[arussell29]

The fix for the ITN extended key is very important for me. I will not be able to withdraw my ITN rewards without it. Thank you.

[ilap]

The fix for the ITN extended key is very important for me. I will not be able to withdraw my ITN rewards without it. Thank you.

That's ok, they can create some cli command for handling ext keys, I was referring only the conversion between keys.

[stakepool247]

same issue here as well, hope this issue is resolved soon.

[gacallea]

Many pool operators are affected by this, including me. I hope this issue is fixed soon as well, or all hard work on the ITN would go to waste for so many of us.

[stakepool247]

Is there any progress on this? Any deadline?

[daolavi]

Hopefully, you guys will sort it out soon. I'm keeping an eye on this ticket.

[aalkilani]

Any updates on this?

[dcoutts]

The cardano-cli now supports extended payment keys. It should now be relatively straightforward to support the conversion.

[AndrewWestberg]

Can we add some Shelley Mainnet and High priority tags to this one? This would be very bad to launch without this after promising that no ITN funds would be left behind.

[MarcelKlammer]

This should be a high priority now.

[ilap]

Can we add some Shelley Mainnet and High priority tags to this one? This would be very bad to launch without this after promising that no ITN funds would be left behind.

As I understand, the following would like to be applied for the ITN pool reward keys created as Ed25519Extended keypairs:

1. The ITN's pool staking/rewards keys and therefore the rewards in it are used as Shelley mainnet staking keys and its rewards
   as pledge (owner staking key) or as rewards (operator's staking key).
2. The ITN's pool staking/rewards keys and therefore the rewards in it are to be moved (as reward account balance withdrawal transaction) to some wallet on Shelley mainnet that can be used
   a. as owner or operator keys/wallet staking keys and/or pledge/rewards (similar to point 1.) or
   b. as just a simple wallet for spending i.e. sending them to anybody.

1. Migration ITN rewards as staking key to Shelley mainnet as owner or operator's staking keys.

Some part of the issue is related to this
But, I highly discourage you to use ITN keys as owner or operator's staking keys on Shelley mainnet. It has a lot of disadvantages.
In other hand, the withdrawal ITN rewards on Shelley is part of the issue/problem. See, the Withdrawal explanation later.

2. Migration to some operator, owner or spending wallet

2.a migrate to op/owner wallet

I would recommend this instead of point 1. and I will definitely do it:

1. create two wallets in Daedalus (one for owner (only pledge) and one for operator (only for rewards) if the operator is the same /w the owner), then
2. extract reward staking keys from the same mnemonics used for above wallet(s), using cardano-address using the derivation path for the reward staking key and
3. use those keys temporary for signing and using in certificates.
4. move funds from the ITN extended keys (i.e. the account style reward address) to these wallet addresses (similar to those in the F&F exercises) by creating, signing and sending txs when ths feature i.e. reward refund is available on Shelley, see point 2.b
5. delete or move offline the keys extracted.

For staking key extraction, we need confirmation from the dev team that the keys of the m/1852'/1815'/0'/2 derivation path is used as reward account staking keys in Shelley.

UPDATE2: I have just checked Yoroi's key derivation for staking keys and they use the m/1852'/1815'/0'/2/0 (address instead of chain) for staking keys. E.g:

rootKey
    .derive(harden(Purpose.CIP1852))   // 1852' Purpose
    .derive(harden(CoinTypes.CARDANO)) // 1815' Coin
    .derive(harden(0));                // 0'      Account
    .derive(ChainDerivation.CHIMERIC)  // 2       Chain 
    .derive(0)                         // 0       Address
    .to_public();

2.b migrate to some spending wallet

For the ITN rewards: I will jut simply move those ITN rewards to any wallet address.
Check details with cardano-cli.

Note: I did similar on ITN (I created keys from mnemonic using some tools e.g. jcli and some node script), and what I only had to do now in MC4's for recovering my pool reward is: using the 15-length mnemonics in Daedalus 1.6.0-STN5 to retrieve my pool rewards similar to the delegation rewards.

Withdrawal

The issue is that the rewards in Shelley are on the account reward addresses generated from some public key (simple ed25519_pk like keys) of the extended key differ. But, the public key is valid everywhere therefor it can be used as and account reward address in Shelley.

So, to withdrawal, you only need to create a withdrawal transaction on Shelley

1. at least one input (tx:id) that have some funds for paying the fee
2. Contains an output (address to send) and finally
3. reward account address and the reward in lovelace (i.e. reward address and reward) as bech32address+lovelace

End sign (witness) this transaction /w

1. the input's the private key and
2. privatekey part of the ITN's extended key.

From cardano-cli: The reward withdrawal as StakeAddress+Lovelace where StakeAddress is the Bech32-encoded stake address followed by the amount in Lovelace."

UPDATE: But, I did not expect this ^, I would have expected a normal Shelley reward account address i.e. blake2b_224(account address prefix | ed25519_pk)

DISCLAIMER: BUT, DO NOT TRY TO DO THIS, UNTIL YOU 100% UNDERSTAND WHAT YOU ARE DOING.

[steakandbake]

Experiencing same issue. Bump for visibility.

[AndrewWestberg]

This issue has been resolved by using a special build of cardano-cli

$ git checkout -b jordan/cli-handle-extended-itn-keys
$ git branch --set-upstream-to origin/jordan/cli-handle-extended-itn-keys jordan/cli-handle-extended-itn-keys
$ git pull
$ cabal clean
$ cabal update
$ cabal build all
$ cp $(find $(pwd) -type f -name "cardano-cli") ~/.local/bin/cardano-cli-itn

Then, extract the keys

$ cardano-cli-itn shelley key convert-itn-extended-key --itn-signing-key-file pool_owner.prv --out-file pool_owner.itn.skey
$ cardano-cli-itn shelley key convert-itn-key --itn-verification-key-file pool_owner.pub --out-file pool_owner.itn.vkey

These keys can be used like regular staking keys. You can use the regular cardano-cli commands for withdrawing rewards. You'll need another account to pay the fees when withdrawing rewards. Once you've extracted the keys, they are good to use with the regular 1.18.0 version of the node for mainnet.

[Jack-Jhon]

cardano-cli-itn

Hello, I did not find this branch in the carada-node repository, can you link it to me? thank!

[disassembler]

it's in master (and node 1.18.0 and 1.19.0)