Cardano-wallet: Can't submit metadata via wallet API

[ghost]

Context

Problem: When trying to submit a transaction using cardano-wallet API, if the transaction includes metadata the metadata is not posted to the blockchain along with the transaction.

Documentation: https://input-output-hk.github.io/cardano-wallet/api/edge/#operation/postTransaction

Results: After submitting a successful transaction (ie, no errors about malformed metadata or any other issues that would prevent a transaction from completing), the json object returned from the transaction does not include the metadata (see below except from a successful transaction that included metadata but was not posted with the transaction: a45c4c721ca071154805a018c8c05dd0aefd12da645830787c1eb9edfbe21790):

["metadata"]=>
  NULL

| Information | Information |
| --- | --- |
| Version | 2020.11.26 (git revision: ec22c1ba68e3483c0a2cdfba90d751d3a5f0d47f)
| Platform | Ubuntu 18.04 |
| Installation | Built with Guild instructions: https://cardano-community.github.io/guild-operators/#/Build/wallet |

Steps to Reproduce

Using the language of your choice, compose a function that meets the requirements of the documentation to send a successful transaction that includes metadata and then check either the transaction ID on the blockchain and see if it has any metadata (if it does, then, my bad, I guess this is user error, and please tell me what you did to get it to work), or dump the raw transactions json when it happens and see if there is any meta data. If it doesn't have any and your transaction was a success, then welcome to the frustrating hell i've been in for the last few days.

See above

Expected behavior

The expected behavior would be to see the metadata posted along with the transaction to the blockchain.

Actual behavior

What actually happens is that the transaction goes through successfully but the metadata is not posted/included with the transaction.

["metadata"]=>
  NULL

---

Resolution

---

QA

[rvl]

Hi @coconutpool - thanks for the report, and sorry that it's not working for you.

I can't reproduce your issue with cardano-wallet v2020.11.26.

For example, I have sent you a transaction with metadata, and this is what was the API response was:

{
  "status": "pending",
  "withdrawals": [],
  "amount": {
    "quantity": 5172893,
    "unit": "lovelace"
  },
  "inputs": [
    {
      "amount": {
        "quantity": 20000000,
        "unit": "lovelace"
      },
      "address": "addr1q9pwwavztgpmp8ffyk7fqav054aw8q2hcp8u8n2qprnzqvd0mm0qynsd0prs52d0sat7laugfxaean9kapcvxrmmgqxs39mz6r",
      "id": "a4c2f40b65a0513750a98164b03a85f27480f78b12eed92a4392b6781cd70bc2",
      "index": 0
    }
  ],
  "direction": "outgoing",
  "outputs": [
    {
      "amount": {
        "quantity": 14827107,
        "unit": "lovelace"
      },
      "address": "addr1qyr79mneaja767gag9efu7p7w6pqaq4jlp8jwzudnxhthp90mm0qynsd0prs52d0sat7laugfxaean9kapcvxrmmgqxsmcn72a"
    },
    {
      "amount": {
        "quantity": 5000000,
        "unit": "lovelace"
      },
      "address": "addr1qx5fqf6fksplrlh6x367y5xwpr5wah9szdpa0vr3c5a75e5l0sanuyzjdehfw7zgnl8fwth0njh56x8rjmsra4up03ds22t9mp"
    }
  ],
  "expires_at": {
    "time": "2020-12-07T13:12:04Z",
    "epoch_number": 234,
    "absolute_slot_number": 15780433,
    "slot_number": 55633
  },
  "pending_since": {
    "height": {
      "quantity": 5046605,
      "unit": "block"
    },
    "time": "2020-12-07T11:12:01Z",
    "epoch_number": 234,
    "absolute_slot_number": 15773230,
    "slot_number": 48430
  },
  "metadata": {
    "1": {
      "string": "https://github.com/input-output-hk/cardano-wallet/issues/2371"
    }
  },
  "id": "2e79ed9542eaef39267eac92e1ff32935cf5c264c154d0a6514decba10db1456"
}

Also, please note that cardano explorer does not show transaction metadata. But the next version will.

However, the cardano-wallet transactions API should always show metadata.

If the transaction metadata is malformed, invalid, or too large, the wallet should always return an error, and no transaction should be submitted.

Here is some wiki documentation on transaction metadata - perhaps that helps.

Also, you can do more testing with the cardano shelley testnet:

• Faucet
• Testnet explorer

Could you provide the json data you are POSTing to the transactions endpoint, minus the spending password? For example:

{
  "passphrase": "xxxxxx",
  "payments": [
    {
      "address": "addr1qx5fqf6fksplrlh6x367y5xwpr5wah9szdpa0vr3c5a75e5l0sanuyzjdehfw7zgnl8fwth0njh56x8rjmsra4up03ds22t9mp",
      "amount": {
        "quantity": 5000000,
        "unit": "lovelace"
      }
    }
  ],
  "metadata": {
    "1": {
      "string": "https://github.com/input-output-hk/cardano-wallet/issues/2371"
    }
  }
}

[ghost]

I got it. It took awhile, but I didn't realize just how limited and specific the metadata has to be in order to submit it via the wallet.

I've discovered that while you can submit ANYTHING that's JSON via the command line (cardano-cli) you can ONLY submit very specialized formatted metadata from the wallet.

Personally I think that's dumb. I've been told it's to prevent "spam" from meta transactions, but thats seems like a pretty lame excuse considering I can submit literally anything as long as its in JSON format from the command line.

Please fix this. Allow anything that can be submitted from the command line to be submitted to the wallet API.

[rvl]

With cardano-cli --json-metadata-no-schema, it will convert "any" json into the on-chain metadata format. The conversion is automatic, but the results on chain might not be what you expected!!

Use the programming language of your choice to convert your data to and from the wallet tx metadata format. The tx metadata format used by the cardano-wallet API maps directly to how transactions are encoded. There is also cardano-cli --json-metadata-detailed-schema which uses the same json format as cardano-wallet.

You will find there are some metadata string length limits, and an overall transaction size limit. These limits are designed to make abuse less simple, that's all. The metadata size limits are cardano tx validation rules, nothing to do with the wallet.

Please check the response codes returned from the post transaction API. If there is invalid metadata in a payment request, it should be immediately rejected by the wallet, and no transaction should ever reach the chain.

[KtorZ]

As @rvl pointed out, one can convert any JSON into binary on-chain metadata, but you cannot convert any binary on-chain metadata back to JSON. So instead of converting any arbitrary JSON value and risking breaking client applications which would be rightfully expecting to see the same metadata reported to them as they posted it, we've opted for a safer approach and use JSON to _describe_ what the metadata should be using a succinct vocabulary.

It's not dumb, it's actually quite clever; and also a lot more reliable than converting any arbitrary JSON inputs without being able to anticipate how they'll come back to you.

Please fix this. [..] but thats seems like a pretty lame excuse considering I can submit literally anything as long as its in JSON format from the command line.

No this is the fixed way already. If you're happy with the command-line, use the command-line.

Since there's no issue, I'll now close this.

[ghost]

I don't want to use the command line, that's why I am specifically using the wallet API and not the command line (I mean yes, technically I can use the command line, but that's not what I'm trying to do here).

How do I "describe" the metadata then? What does that even mean? Like do I say I'm think of a number between 1 and 10 instead of giving an actual number? Like come on, help me out here so I can make my app work like it's supposed to since I have to work within the constraints of what the wallet allows.

[KtorZ]

You have linked the API documentation above so I assume that you have read it and in particular the bits regarding metadata in transactions. The key take-aways are:

• Metadata should be defined as object where top-level keys are positive integers.
• Each object pointed by a top-level key must be made using one of the 5 available constructors: int, string, bytes, list, map.

For example, trying to send the number 14 as metadata, one may pass:

{ 
    "0": { "int": 14 }
}

For more examples and details, have a look at the wiki about transaction metadata.

Do you have an example of "naked JSON metadata" you're trying to send?

Like come on, help me out here so I can make my app work like it's supposed to since I have to work within the constraints of what the wallet allows.

The constraints do not come from the wallet, the constraints about metadata comes from the protocol. The wallet gives you an interface which gives you a way to represent anything that is allowed by the protocol. Any "naked JSON" can be converted to a description as expected by the wallet, for instance:

{ "cardano": [ "rocks" ] } 

becomes

{ "0": { "map": [ { "k": { "string": "cardano" }, "v": { "list": [ { "string": "rocks" } ] } } ] } }

What you have to understand is that JSON is NOT the format used on-chain. The on-chain format is called CBOR, and CBOR is much more expressive than JSON. It is not a simple 1:1 translation. CBOR gives you way to do fancy things such as using list as keys for maps, which is totally impossible in JSON. Hence this description format which allows to work around the limitations of JSON itself.

[ghost]

I get that now, it's not super clear that that is literally the only thing it accepts in that format. Especially since having done transactions with the CLI and getting entirely different results I think you can understand how frustrated I feel.

So help me out and show me how you tell somebody what "14" means or what it's related to if all I can say is in the meta is "int:14".

Am I just supposed to assume that the person interacting with this meta already knows what it is and what it's for?

Show me how you would post your public key and tell somebody that's what they were looking at and that it's not just a random string of characters.

[KtorZ]

@coconutpool I can't write your application for you. If you have specific questions about the format or the structure of metadata themselves, we will answer them. The wiki is full of examples, and include an example for each of the primitive.

Show me how you would post your public key and tell somebody that's what they were looking at and that it's not just a random string of characters.

1. I'll start by looking at CIP-0010 to see if there's already an existing metadata prefix registered for the type of usage I am after.

2. If not, I'll make a PR to "reserve" this prefix in CIP-0010 and explain the motivation behind my application.

3. Let's say that I reserved the prefix 1337 to identify public keys for MyAwesomeApplication, I'll then define my binary metadata in CBOR as the following CDDL:

{ 1337: bytes .size 32 }

which can be expressed in JSON to cardano-wallet as:

{ "1337": { "bytes": <public-key> } }

where <public_key> is to be replaced with a textual representation of the public key, in base 16. For example:

{ "1337": { "bytes": "48656c6c6f2c2043617264616e6f2148656c6c6f2c2043617264616e6f21ffff" } }

The goal of registering your prefix in CIP-0010 is to explain and document for others what semantic you intend to give to values associated to a chosen prefix. In this case, the semantic would be simple: any metadata which uses 1337 as a prefix should then reference a 32 bytes public key.

There's no guarantee that people would respect that, but your application may disregard any non abiding metadata.

[ghost]

I don't need you to do anything for me, I just want to understand what I'm working with since it's obviously only clear to you and your documentation is frustratingly vague to the non-devloper. Also it would be helpful to maybe include a link to the wiki in the API docs since it is slightly more helpful than what you post there and I didn't know about that until now.

So if I wanted to explicitly tell somebody who didn't know that that string was a public key, I would do a "map" where I could say that "k" was "public key" and "v" was the key string? Is that correct?

And am I limited to only one type of map, int, etc per meta or can I mix them as long as they are in the format?

[ghost]

If you want average people to build apps and dApps on Cardano you're going to need to seriously improve the documentation because it sucks.

[KtorZ]

Ok thanks for your feedback.

[ghost]

I appreciate everyone's help and I get it now. I would like to take a second to make a suggestion after having a minute to look back on how frustrating things were. I think the most confusing part of the whole thing is in the examples of how the metadata is formed (as seen in the API docs, not the wiki). The problem to me was that it all looks like "placeholder" data. When i read it and i see "string" or "key" or some other generic term, I read it as "replace this" with your string, int, list, whatever that you want to use. I dont read it as "this is the data type that belongs here, dont edit this, change the other parameter." do you see what i'm saying and how that lead to endless confusion about how to format the metadata? I think if the docs made it more clear how that works, it may make it easier for others to understand as well. That's all I have to say about this. Thanks.

[rvl]

Hi @coconutpool - yes I see what you're saying. In the API docs, there is not adequate space to provide an explanation, examples, etc. This needs to be fully documented in the user guide, and then the API docs should link to that.