Cardano-wallet: Different behavior regarding resolved inputs when listing transactions with and without query parameters

Created on 18 May 2020  路  7Comments  路  Source: input-output-hk/cardano-wallet

Context

Reported by an exchange on Slack.

__Quote:__

"We need to check input, output values in a transaction. However, if we use start, end parameters in /transactions api, input values don't appear in the response. If we fetch all the transactions without start and end parameters, they appears. But we don't want to fetch all the transactions belonging the wallet neither due to the performance concerns."

| Information | - |
| --- | --- |
| Version | Unknown |
| Platform | Unknown |
| Installation | Unknown |

Steps to Reproduce

see #1685

Expected behavior

Responses are coherent with and without start / end

Actual behavior

Responses are different.

Resolution

  • Resolve transaction inputs in the database lookup, when fetching inputs and outputs ==> #1685

QA

  • Added a regression test to capture the scenario described below in comment as TRANS_REG_1670
picture KtorZ

All 7 comments

Going quickly through the code, one of the reason might be:

    -- This relies on DB.readTxHistory returning all necessary transactions
    -- to assemble coin selection information for outgoing payments.
    assemble

So, we the history returned is used both as the result, but also as an index for resolving transaction inputs. Therefore, if a transaction input refers to a known transaction that is discarded by the range parameters, then it won't be present in the return history and will fail the lookup.

Making assemble fetch transactions from the database would probably fix the issue, but at what cost? This would require at most 1 request per input (so possibly up to 80/90 requests per transactions) although it'd still be better than fetching the entire history in memory every time.

Another alternative could be to list transactions as already resolved from the database with an inner join; haven't quite thought about it but it seems doable from the top of my head and should not induce that much overhead.

cc @piotr-iohk : if this intuition is correct :point_up: , we should be able to reproduce this by making two transactions, the second one using at least one output of the first one, and then, by listing all transactions and listing all transaction with a range only including the second transaction.

KtorZ picture KtorZ on 18 May 2020

Here is an example case from my test wallet

cardano-wallet transaction list <wallet_id> --order ascending --start 2020-05-14T05:53:31Z

Ok.
[
    {
        "inserted_at": {
            "time": "2020-05-14T05:53:31Z",
            "block": {
                "height": {
                    "quantity": 4159520,
                    "unit": "block"
                },
                "epoch_number": 192,
                "slot_number": 14426
            }
        },
        "status": "in_ledger",
        "amount": {
            "quantity": 186803,
            "unit": "lovelace"
        },
        "inputs": [
            {
                "id": "42f493c8502dac8a6641311c5bd7cb7667aaf7c8173fa9cdec9f108a48e66bcd",
                "index": 1
            },
            {
                "id": "42f493c8502dac8a6641311c5bd7cb7667aaf7c8173fa9cdec9f108a48e66bcd",
                "index": 0
            },
            {
                "id": "2ef34f9c05094c4062a421ba3c848749540eb9b52843d735b35948f17818d5d2",
                "index": 1
            }
        ],
        "direction": "outgoing",
        "outputs": [
            {
                "amount": {
                    "quantity": 251730278,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhsudH4ffDjF4aPeLMSNm7jY8sjrC6KMYJEQuLVw6K9BDwPborASAmnYNJaMN2XFS4HntbFB8qE8hjBikLRbHQ2x5QmYJfU2"
            },
            {
                "amount": {
                    "quantity": 1110000,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrht2XS7fa7c9rmppP7uKh9CDmHEv7NoB1ZQX5cxoY211qgJwGTvdCraHpohTkEcqjoc11NRzFTLVksqF2RnSPHKx73JvfPmd"
            }
        ],
        "depth": {
            "quantity": 17379,
            "unit": "block"
        },
        "id": "e5af89b9cdd7d35c47e177a995360edfabb1f923561e3190e8c08a85ec1822a9"
    },
    {
        "inserted_at": {
            "time": "2020-05-14T05:54:31Z",
            "block": {
                "height": {
                    "quantity": 4159523,
                    "unit": "block"
                },
                "epoch_number": 192,
                "slot_number": 14429
            }
        },
        "status": "in_ledger",
        "amount": {
            "quantity": 170894,
            "unit": "lovelace"
        },
        "inputs": [
            {
                "amount": {
                    "quantity": 251730278,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhsudH4ffDjF4aPeLMSNm7jY8sjrC6KMYJEQuLVw6K9BDwPborASAmnYNJaMN2XFS4HntbFB8qE8hjBikLRbHQ2x5QmYJfU2",
                "id": "e5af89b9cdd7d35c47e177a995360edfabb1f923561e3190e8c08a85ec1822a9",
                "index": 0
            }
        ],
        "direction": "outgoing",
        "outputs": [
            {
                "amount": {
                    "quantity": 250449384,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhtCfXvRBS6CVME47uACg5YtHDWb7jJFiJfA2BhLvRJW61XTfcRoeHxNVDGkh6kVyrrJNaH63JXrAq7PonT5Po4xxoBFRrM2"
            },
            {
                "amount": {
                    "quantity": 1110000,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrht2XS7fa7c9rmppP7uKh9CDmHEv7NoB1ZQX5cxoY211qgJwGTvdCraHpohTkEcqjoc11NRzFTLVksqF2RnSPHKx73JvfPmd"
            }
        ],
        "depth": {
            "quantity": 17376,
            "unit": "block"
        },
        "id": "6fe2c36e306e85427f82343b0bc0d603340d85973eebf14b93c787f7d2f52b2a"
    },
    {
        "inserted_at": {
            "time": "2020-05-14T06:12:51Z",
            "block": {
                "height": {
                    "quantity": 4159578,
                    "unit": "block"
                },
                "epoch_number": 192,
                "slot_number": 14484
            }
        },
        "status": "in_ledger",
        "amount": {
            "quantity": 10170894,
            "unit": "lovelace"
        },
        "inputs": [
            {
                "amount": {
                    "quantity": 250449384,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhtCfXvRBS6CVME47uACg5YtHDWb7jJFiJfA2BhLvRJW61XTfcRoeHxNVDGkh6kVyrrJNaH63JXrAq7PonT5Po4xxoBFRrM2",
                "id": "6fe2c36e306e85427f82343b0bc0d603340d85973eebf14b93c787f7d2f52b2a",
                "index": 0
            }
        ],
        "direction": "outgoing",
        "outputs": [
            {
                "amount": {
                    "quantity": 240278490,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhssS2V42ombNCtJDxaJhh5WNU7GfYH6BurPajM16MbpSic5gJeVUYni4heRtz8JbCg23ATnnwiTXVRVKKjSvGYqQtLLnoc5"
            },
            {
                "amount": {
                    "quantity": 10000000,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhse3psjrnwHFhow2Gn7Bsks6omysUvLM6EtnPz2QpHX5AQkwe3hnmmbQdUG67TMZPmqHEeugj7MTVRwGzpm7uy3xt34KUSV"
            }
        ],
        "depth": {
            "quantity": 17321,
            "unit": "block"
        },
        "id": "a61cd9ac1bea8878052680828702dc4021ff7a77ca6bd0b1876966bcc5c53e81"
    }
]

As you can see the first transaction(e5af89b9cdd7d35c47e177a995360edfabb1f923561e3190e8c08a85ec1822a9) is missing the input values and addresses.

If I fetch the transactions without --start parameter here is the corresponding part,

[
    {
        "inserted_at": {
            "time": "2020-05-14T05:53:31Z",
            "block": {
                "height": {
                    "quantity": 4159520,
                    "unit": "block"
                },
                "epoch_number": 192,
                "slot_number": 14426
            }
        },
        "status": "in_ledger",
        "amount": {
            "quantity": 186803,
            "unit": "lovelace"
        },
        "inputs": [
            {
                "amount": {
                    "quantity": 1110000,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrht2XS7fa7c9rmppP7uKh9CDmHEv7NoB1ZQX5cxoY211qgJwGTvdCraHpohTkEcqjoc11NRzFTLVksqF2RnSPHKx73JvfPmd",
                "id": "42f493c8502dac8a6641311c5bd7cb7667aaf7c8173fa9cdec9f108a48e66bcd",
                "index": 1
            },
            {
                "amount": {
                    "quantity": 251907081,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhtASth3EBJqqscGuxq6Lh4rs88XxqTHWge9WwUo6NPSWE2ausuiQWwdk2LjLzqBdTpasrfhFYWyy3DVn3TT3QcAixSJj3cc",
                "id": "42f493c8502dac8a6641311c5bd7cb7667aaf7c8173fa9cdec9f108a48e66bcd",
                "index": 0
            },
            {
                "amount": {
                    "quantity": 10000,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrht2XS7fa7c9rmppP7uKh9CDmHEv7NoB1ZQX5cxoY211qgJwGTvdCraHpohTkEcqjoc11NRzFTLVksqF2RnSPHKx73JvfPmd",
                "id": "2ef34f9c05094c4062a421ba3c848749540eb9b52843d735b35948f17818d5d2",
                "index": 1
            }
        ],
        "direction": "outgoing",
        "outputs": [
            {
                "amount": {
                    "quantity": 251730278,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhsudH4ffDjF4aPeLMSNm7jY8sjrC6KMYJEQuLVw6K9BDwPborASAmnYNJaMN2XFS4HntbFB8qE8hjBikLRbHQ2x5QmYJfU2"
            },
            {
                "amount": {
                    "quantity": 1110000,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrht2XS7fa7c9rmppP7uKh9CDmHEv7NoB1ZQX5cxoY211qgJwGTvdCraHpohTkEcqjoc11NRzFTLVksqF2RnSPHKx73JvfPmd"
            }
        ],
        "depth": {
            "quantity": 17378,
            "unit": "block"
        },
        "id": "e5af89b9cdd7d35c47e177a995360edfabb1f923561e3190e8c08a85ec1822a9"
    },
    {
        "inserted_at": {
            "time": "2020-05-14T05:54:31Z",
            "block": {
                "height": {
                    "quantity": 4159523,
                    "unit": "block"
                },
                "epoch_number": 192,
                "slot_number": 14429
            }
        },
        "status": "in_ledger",
        "amount": {
            "quantity": 170894,
            "unit": "lovelace"
        },
        "inputs": [
            {
                "amount": {
                    "quantity": 251730278,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhsudH4ffDjF4aPeLMSNm7jY8sjrC6KMYJEQuLVw6K9BDwPborASAmnYNJaMN2XFS4HntbFB8qE8hjBikLRbHQ2x5QmYJfU2",
                "id": "e5af89b9cdd7d35c47e177a995360edfabb1f923561e3190e8c08a85ec1822a9",
                "index": 0
            }
        ],
        "direction": "outgoing",
        "outputs": [
            {
                "amount": {
                    "quantity": 250449384,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhtCfXvRBS6CVME47uACg5YtHDWb7jJFiJfA2BhLvRJW61XTfcRoeHxNVDGkh6kVyrrJNaH63JXrAq7PonT5Po4xxoBFRrM2"
            },
            {
                "amount": {
                    "quantity": 1110000,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrht2XS7fa7c9rmppP7uKh9CDmHEv7NoB1ZQX5cxoY211qgJwGTvdCraHpohTkEcqjoc11NRzFTLVksqF2RnSPHKx73JvfPmd"
            }
        ],
        "depth": {
            "quantity": 17375,
            "unit": "block"
        },
        "id": "6fe2c36e306e85427f82343b0bc0d603340d85973eebf14b93c787f7d2f52b2a"
    },
    {
        "inserted_at": {
            "time": "2020-05-14T06:12:51Z",
            "block": {
                "height": {
                    "quantity": 4159578,
                    "unit": "block"
                },
                "epoch_number": 192,
                "slot_number": 14484
            }
        },
        "status": "in_ledger",
        "amount": {
            "quantity": 10170894,
            "unit": "lovelace"
        },
        "inputs": [
            {
                "amount": {
                    "quantity": 250449384,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhtCfXvRBS6CVME47uACg5YtHDWb7jJFiJfA2BhLvRJW61XTfcRoeHxNVDGkh6kVyrrJNaH63JXrAq7PonT5Po4xxoBFRrM2",
                "id": "6fe2c36e306e85427f82343b0bc0d603340d85973eebf14b93c787f7d2f52b2a",
                "index": 0
            }
        ],
        "direction": "outgoing",
        "outputs": [
            {
                "amount": {
                    "quantity": 240278490,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhssS2V42ombNCtJDxaJhh5WNU7GfYH6BurPajM16MbpSic5gJeVUYni4heRtz8JbCg23ATnnwiTXVRVKKjSvGYqQtLLnoc5"
            },
            {
                "amount": {
                    "quantity": 10000000,
                    "unit": "lovelace"
                },
                "address": "DdzFFzCqrhse3psjrnwHFhow2Gn7Bsks6omysUvLM6EtnPz2QpHX5AQkwe3hnmmbQdUG67TMZPmqHEeugj7MTVRwGzpm7uy3xt34KUSV"
            }
        ],
        "depth": {
            "quantity": 17320,
            "unit": "block"
        },
        "id": "a61cd9ac1bea8878052680828702dc4021ff7a77ca6bd0b1876966bcc5c53e81"
    }
]
psychoplasma picture psychoplasma on 18 May 2020
👍1

Indeed I can reproduce:

I have 3 outgoing txs on my wallet:
id1 -> inserted at: 2020-05-18T14:15:36Z
id2 -> inserted at: 2020-05-18T14:26:56Z
id3 -> inserted at: 2020-05-18T14:31:56Z

TC0: Listing all transactions. => The amount is shown for every input for each tx.

cardano-wallet-byron transaction list 05bb19daeb9e3aaca992856fb21f64ded1632aec | jq '.[] | .inputs'

[
  {
    "amount": {
      "quantity": 657187,
      "unit": "lovelace"
    },
    "address": "37btjrVyb4KEWxuxXEVFoj7HqjS6Q6nVkAHVFa33QR8QuqScKxD4p9Aw3PYfXQj724wWNPBGnJLWEiW9SYR5c4EwB8rCRRysab3pYamT8Wwun45WGd",
    "id": "bf6f9b0f2067314e2c507750f47744bf82bd0f1a7562000747f90d16aee619ea",
    "index": 0
  }
]
[
  {
    "amount": {
      "quantity": 828665,
      "unit": "lovelace"
    },
    "address": "37btjrVyb4KCckzCvuVxt7MizbjCDc2N9LbppckfeMDJffPae6ngk3Ms551hesD2fr5wbyFCmHSAFLbvTwTMdieKjT8D6GsEZ4ZZixTfzYrGXuvF7P",
    "id": "89c680afc1e3ca4cad2f442d99f9a00a2b921283c7a64624f1b575e60c1f2ad2",
    "index": 0
  }
]
[
  {
    "amount": {
      "quantity": 1000000,
      "unit": "lovelace"
    },
    "address": "37btjrVyb4KDBr7TwHm5HivAt8r8Jk4eXDERDyj6G8s9Rw6Prmr5RrB9dnqmk1cmmXQCWURyLLfa4BNKfrJDpTsaxSvWwviHXH4bRnLJuxm16Fzvca",
    "id": "0b089cb29fddb79307ee36dcd5ce3728d3fea794782901c7d2550bd7fc8548ec",
    "index": 0
  }
]

TC1: List txs with _--start 2020-05-18T14:15:36Z_ ( start before tx id1) => Amount for tx id1 is missing.

cardano-wallet-byron transaction list 05bb19daeb9e3aaca992856fb21f64ded1632aec --start 2020-05-18T14:15:36Z | jq '.[] | .inputs'
[
  {
    "amount": {
      "quantity": 657187,
      "unit": "lovelace"
    },
    "address": "37btjrVyb4KEWxuxXEVFoj7HqjS6Q6nVkAHVFa33QR8QuqScKxD4p9Aw3PYfXQj724wWNPBGnJLWEiW9SYR5c4EwB8rCRRysab3pYamT8Wwun45WGd",
    "id": "bf6f9b0f2067314e2c507750f47744bf82bd0f1a7562000747f90d16aee619ea",
    "index": 0
  }
]
[
  {
    "amount": {
      "quantity": 828665,
      "unit": "lovelace"
    },
    "address": "37btjrVyb4KCckzCvuVxt7MizbjCDc2N9LbppckfeMDJffPae6ngk3Ms551hesD2fr5wbyFCmHSAFLbvTwTMdieKjT8D6GsEZ4ZZixTfzYrGXuvF7P",
    "id": "89c680afc1e3ca4cad2f442d99f9a00a2b921283c7a64624f1b575e60c1f2ad2",
    "index": 0
  }
]
[
  {
    "id": "0b089cb29fddb79307ee36dcd5ce3728d3fea794782901c7d2550bd7fc8548ec",
    "index": 0
  }
]

TC2: List txs with _--start 2020-05-18T14:20:00Z_ (before tx id2) => Amount for tx id2 is missing.

cardano-wallet-byron transaction list 05bb19daeb9e3aaca992856fb21f64ded1632aec --start 2020-05-18T14:20:00Z | jq '.[] | .inputs'
Ok.
[
  {
    "amount": {
      "quantity": 657187,
      "unit": "lovelace"
    },
    "address": "37btjrVyb4KEWxuxXEVFoj7HqjS6Q6nVkAHVFa33QR8QuqScKxD4p9Aw3PYfXQj724wWNPBGnJLWEiW9SYR5c4EwB8rCRRysab3pYamT8Wwun45WGd",
    "id": "bf6f9b0f2067314e2c507750f47744bf82bd0f1a7562000747f90d16aee619ea",
    "index": 0
  }
]
[
  {
    "id": "89c680afc1e3ca4cad2f442d99f9a00a2b921283c7a64624f1b575e60c1f2ad2",
    "index": 0
  }
]

TC3: List txs with _--start 2020-05-18T14:30:00Z_ (before tx id3) => Amount for tx id3 is missing.

cardano-wallet-byron transaction list 05bb19daeb9e3aaca992856fb21f64ded1632aec --start 2020-05-18T14:30:00Z | jq '.[] | .inputs'
Ok.
[
  {
    "id": "bf6f9b0f2067314e2c507750f47744bf82bd0f1a7562000747f90d16aee619ea",
    "index": 0
  }
]

Conclusion: amount is not displayed for the first outgoing transaction that is shown after --start parameter.

Can be observed for wallet on byron testnet: onion game wish tuna smooth category clock language hour subway mirror soccer.

piotr-iohk picture piotr-iohk on 18 May 2020
👍1

Similar issue, but for incoming transactions: #573.

rvl picture rvl on 20 May 2020

Looking at my previous example https://github.com/input-output-hk/cardano-wallet/issues/1670#issuecomment-630233182 the original issue is no longer the case. I.e. the amount is being shown when one uses start to narrow listed transactions.

I noticed however that there is some discrepancy on transaction input amounts before and after the fix. I mean transactions on the wallet that is restored on recent master are different from tx on the wallet restored on revision before the fix (https://github.com/input-output-hk/cardano-wallet/commit/031835950a38384b289f3adc43c671a00ef9000b)

Looking at the transaction from my previous example 7ddde107f4691e0db21462324a41c7d1d719979b4c4861e2486af1937ac0ac97:

Before the fix:
input amount = 657187
output1 = 155
output2 = 485654

After the fix:
input = 100 (which doesn't seem right)
output1 = 155
output2 = 485654

Screenshot from 2020-05-27 15-31-25

piotr-iohk picture piotr-iohk on 27 May 2020

@piotr-iohk , should be fixed by #1697 now :ok_hand:

KtorZ picture KtorZ on 29 May 2020
👍1

lgtm.

piotr-iohk picture piotr-iohk on 29 May 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

gitloverr picture gitloverr  路  6Comments
got3nks picture got3nks  路  9Comments
vicepool picture vicepool  路  7Comments
KtorZ picture KtorZ  路  7Comments
KtorZ picture KtorZ  路  11Comments