Ethers.js: Error: invalid utf8 byte sequence; invalid continuation byte

[gitpusha]

Hi,

I am getting the following error whilst trying to parse an event using ethers.js interface.

This event emits a string, I assume this has something to do with the problem because before I added the string to the event data emission there was no problem with parsing the log.

The event in solidity:

    event LogExecutionFailure(
        address indexed executor,
        uint256 indexed executionClaimId,
        address payable indexed user,
        IGelatoTrigger trigger,
        IGelatoAction action,
        string executionFailureReason
    );

My ethers script that throws the error:

const abi = [ "event LogExecutionFailure(address indexed executor, uint256 indexed executionClaimId, address indexed user, address trigger, address action, string executionFailureReason)"]

const iface = new ethers.utils.Interface(abi);

const topicExecutionFailure = ethers.utils.id(
      "LogExecutionFailure(address,uint256,address,address,address,string)"
);

const filterExecutionFailure = {
      address: "0x21F7908734A147e084DBEe58C616fd59CE2A0a02",
      fromBlock: parseInt(searchFromBlock),
      topics: [topicExecutionFailure]
};
const logsExecutionFailure = await provider.getLogs(
      filterExecutionFailure
);
for (const log of logsExecutionFailure) {
      const parsedLog = iface.parseLog(log);
....
ERROR throws here
}

Here is a link to the tx that emitted this event on 0x21F7908734A147e084DBEe58C616fd59CE2A0a02 on Kovan

As you can see the correct string ("ErrorTransferFromUser") is shown as part of the event data on Kovan:

Please help me find out why my ethers interface throws when trying to parseLog(log).

[ricmoo]

Can you include the contract source?

If you change the encoding from text to hex in that entry, you will see there are 4 bytes in front; which are non-printable and invalid UTF-8 bytes.

The bytes are obviously the length of the string, and it looks like it is emitting a a full EIP-838 4-byte selector (the 0x08c379a0; i.e. id("Error(string)"), so everything is properly being shifted as a revert error output, but I'm not sure how you can trigger this to be logged in Solidity as an error like this without using the require(..., message) syntax.

I'm a bit concerned that the length; its length should be congruent to 4 mod 32, but it is congruent to 0 mod 32, which means there is no way to detect it, and I'm curious where it is stealing/padding the extra bytes from/to... :s

[ricmoo]

Hmmm... Are you maybe calling a function in another contract, which is reverting, and you are using the returned bytes as a string? That would explain the padding and the extra length prefix. Some sort of ad-hoc try...catch? I'm guessing you would have to be using assembly for this though, and not stripping the Error(string) selector off?

[gitpusha]

Hmmm... Are you maybe calling a function in another contract, which is reverting, and you are using the returned bytes as a string? That would explain the padding and the extra length prefix. Some sort of ad-hoc try...catch? I'm guessing you would have to be using assembly for this though, and not stripping the Error(string) selector off?

Sorry for the delay.

Man you are a genius. This is exactly what I am doing.

I want to use solidity's new high-level try catch to catch revert string reasons

See Source here on line 395

I intentionally bubble the revert string reason I intend to catch with try catch Error(string memory reason up from here (line 87)

Lastly, I emit the caught revert string reason in the event here on line 417

What should I do to help ethers.js decode this? It does work on kovan.etherscan event logs already.

I appreciate your help. You are a true master of your art!

[ricmoo]

Cool, I didn't realize the try..catch thing was ready. :)

I've not used it before, but my guess is you'll want to do something like:

// Change this to bytes instead of string?  Maybe? Otherwise you'll need to process as a string in place... Same code, just keep using string.
} catch Error(bytes memory revertReason) {
  assembly {
    executionFailureReason := add(revertReason, 36)  
  }
}

This is just a guess though, you may need to better examine the byte code that is bubbled back, but I think you will want to skip over the 32 bytes of the revertReason bytes and then the first 4 bytes of the data; after that should be a valid length-prefixed string...

You would also probably want to add some safety by checking revertReason.length % 32 == 4 and check that the first 4 bytes are the Error(string) selector.

This is all largely guess-work... If it works or you figure out how to do it, let me know. I'll include cookbook recipes. :)

[ricmoo]

(you might have to skip over 68 bytes instead of 36 to get past a string link too... Maybe try 68 before 36? I'm more confident in that number now that I think about it)

[gitpusha]

yeah, it's ready since solc 0.6 release - here is the documentation on it.

There is a specific clause to catch lowLevelData see:

So I might wanna use that.

So let me get this straight. You would recommend to not convert the revertReason bytes I catch here to a string and bubble that string up to here?
Instead, you recommend just bubbling the bytes up. Catching them as bytes. And then performing some bytes manipulation in order to get a valid length-prefixed string. And then emit that string in the event?

*Edit: I cannot bubble the bytes memory revertReason that gets returned by a low-level delegatecall up as bytes. Won't compile. The payload for the revert() op has to be a string apparently.

The problem I basically have is that in my chain of execution I have this low-level delegatecall that I need to make. And that does not auto-bubble up revert reason strings. Which is annoying for me because I want to catch the revert reason string from above in the try catch Error(string memory revertReason) below. So I tried to hack my way around this by converting the returndata that is returned by the low-level delegatecall into a string in hopes of then being able to catch that revertReason string with the high-level try catch Error(string). This seems to have worked. But as you rightly spotted, my string is not formatted correctly.

So I guess I need to perform some string manipulation here after all.

Should I maybe just emit the string I catch as-is in the event and find a way to make ethers.js do some string manipulation before normal decoding of event signature and data?

[ricmoo]

Oh, so I think the delegate call is circumventing the catch (string ...) part, because the delegate call is getting back the selector followed by the ABI encoded bytes. You will probable need to decompose that first, using a similar assembly block like above.

I think this is the problem. You are getting an encoded raw bytes back and casting it to a string, which it isn't... You should decode it here and call revert with the extracted string.

Make sense?

[gitpusha]

Hi, yes that makes sense to me.

I think you should also know about the solidity 0.6's new array slicing syntax that is available for calldata arrays see here.

This is great for decoding bytes arrays without the need to resort to inline assembly.

However, I probably cannot use it here because I am getting a bytes memory returned from the low-level delegatecall.

Still, I figured you should be notified. This new syntax might make life easier for you too, here and there.

So back to my problem:

I cannot do this:

(bool success, bytes memory revertReasonBytes) = address.delegatecall(payload);
string memory revertReasonString = abi.decode(revertReasonBytes, (string));
if (!success) revert(revertReasonString);

because the revertReasonBytes still have a signature and a length in them, correct?

So I would have to slice through the bytes first, to get to that string portion and then decode that as string.

I will try it out myself, I just wanted to bounce my question by you anyhow because I might learn something again from your answer.

Thanks for all your help.

[gitpusha]

(you might have to skip over 68 bytes instead of 36 to get past a string link too... Maybe try 68 before 36? I'm more confident in that number now that I think about it)

Update:
This seems to have worked well!

I was able to use the string now in ethers.js

This code does the trick now, as you prescribed.

Do you think this implementation is ok?

I left out this bit:

You would also probably want to add some safety by checking revertReason.length % 32 == 4 and check that the first 4 bytes are the Error(string) selector.

because I catch any evm exceptions during the decoding anyway.

Can you elaborate on this bit revertReason.length % 32 == 4, please? I dont quite get this sanity check. Why == 4?

Thanks again for your great help.

[ricmoo]

Awesome! If Solidity now supports the slice operation, I think that should work too. Maybe try slicing from 36?

The 4 is from the selector.

If you call a Solidity function normally, it returns ABI encoded data, which is always packaged up into 32 byte slots. So the valid return of any Solidity function with be congruent to 0 mod 32. When an error occurs, it prepends the normal ABI encoded data (0 mod 32) with a 4 byte selector (currently only sighash("Error(string)") is supported but the EIP allows for arbitrary error types in the future, so it's best to check the first 4 bytes match Error(string)), so that makes the number of bytes returned congruent to 4 mod 32. :)

Make sense? It's a pretty nitty-gritty detail of how the underlying VM works and how Solidity ABI interacts with it.

[gitpusha]

Hi,
Yes it does, but it only supports calldata, not memory arrays.

Ok, so all returndata is padded to 32 bytes in solidity, except when the Error selector is prepended, got it.

Thanks for the granular explanation! I am closing this issue. Thanks for your brilliant help.