Parity-ethereum: Windows: `cannot access native cert store`

[Johanpmeert]

2020-02-25 06:28:32 Starting Parity-Ethereum/v2.7.2-stable-d961010f63-20200205/x86_64-pc-windows-msvc/rustc1.41.0
2020-02-25 06:28:32 Keys path C:UsersjohanAppDataRoamingParityEthereumkeysethereum
2020-02-25 06:28:32 DB path C:UsersjohanAppDataLocalParityEthereumchainsethereumdb906a34e69aec8c0d
2020-02-25 06:28:32 State DB configuration: fast
2020-02-25 06:28:32 Operating mode: active

====================

stack backtrace:
0: 0x7ff79df7ea66 -
1: 0x7ff79e93389d -
2: 0x7ff79e9336b1 -
3: 0x7ff79f61e3bb -
4: 0x7ff79f625398 -
5: 0x7ff79e044c3d -
6: 0x7ff79e04b90f -
7: 0x7ff79e7d8185 -
8: 0x7ff79e6da84e -
9: 0x7ff79e69e5e8 -
10: 0x7ff79f63f077 -
11: 0x7ff79f641e5c -
12: 0x7ffb961b7bd4 - BaseThreadInitThunk

Thread 'fetch' panicked at 'cannot access native cert store: Custom { kind: InvalidData, error: BadDER }', srclibcoreresult.rs:1188

[niklasad1]

Hey, this is most likely related to that your native certification store could not be found.
Can you check that it is configured (https://docs.microsoft.com/en-us/windows-hardware/drivers/install/local-machine-and-current-user-certificate-stores)?

My guess is that rustls-native-certs panics in fetch.

Does the node crash after this error?

[sammyp42]

I have the same issue.

[niklasad1]

@ltfschoen stated;

And I also found this issue which appears to have been caused by "rustls-native-certs", which is a dependency of "hyper-rustls" #11515

This issue will likely be resolved by updating hyper-rustls to v0.20

[niklasad1]

The sad part is that hyper-rustls depends on hyper 0.13 which in turn depends on tokio 0.2 and futures 0.3 which is quite significant to change if we want to be consistent in the codebase i.e, we are still using futures 0.1.

[adria0]

@sammyp42 @Johanpmeert, some questions:

1) Do you have certificates in your personal certificate store, it's the current user described in https://docs.microsoft.com/en-us/dotnet/framework/wcf/feature-details/how-to-view-certificates-with-the-mmc-snap-in ?
2) Please, might you send more detailed openethereum logs? You can generate them with -ldebug flag activated.

edit: I tested openthereum with a fresh Windows10 Enterprise installation and works ok.
edit2: Might you try to use --usd-per-eth 200 in the command line and check if the bug is also triggered, please?

[sammyp42]

@sammyp42 @Johanpmeert, some questions:

1. Do you have certificates in your personal certificate store, it's the _current user_ described in https://docs.microsoft.com/en-us/dotnet/framework/wcf/feature-details/how-to-view-certificates-with-the-mmc-snap-in ?
2. Please, might you send more detailed openethereum logs? You can generate them with -ldebug flag activated.

edit: I tested openthereum with a fresh Windows10 Enterprise installation and works ok.
edit2: Might you try to use --usd-per-eth 200 in the command line and check if the bug is also triggered, please?

I downloaded the latest version to retest and get similar issue, but error appears in different part of code now (I replaced my userid with {user}):

Loading config file from C:\Users\{user}\AppData\Roaming\Parity\Ethereum\config.toml
2020-07-19 20:54:27  main INFO openethereum::configuration  Using a fixed conversion rate of Ξ1 = US$200.00 (23809524 wei/gas)
2020-07-19 20:54:27  main INFO openethereum::run  Starting OpenEthereum/v3.0.1-stable-8ca8089-20200601/x86_64-pc-windows-msvc/rustc1.43.1
2020-07-19 20:54:27  main INFO openethereum::run  Keys path h:\chains\parity\Ethereum\keys\ethereum
2020-07-19 20:54:27  main INFO openethereum::run  DB path h:\chains\parity\Ethereum\chains\ethereum\db\906a34e69aec8c0d
2020-07-19 20:54:27  main INFO openethereum::run  State DB configuration: fast
2020-07-19 20:54:27  main INFO openethereum::run  Operating mode: active


====================

   0: <unknown>
   1: <unknown>
   2: <unknown>
   3: <unknown>
   4: <unknown>
   5: <unknown>
   6: <unknown>
   7: <unknown>
   8: <unknown>
   9: <unknown>
  10: <unknown>
  11: <unknown>
  12: <unknown>
  13: BaseThreadInitThunk
  14: RtlUserThreadStart


Thread 'fetch' panicked at 'cannot access native cert store: Custom { kind: InvalidData, error: BadDER }', C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\hyper-rustls-0.18.0\src\connector.rs:29

This is a bug. Please report it at:

    https://github.com/openethereum/openethereum/issues/new

````

nothing exciting in the logs either (and I used the -ldebug flag):

2020-07-19 20:54:27 main INFO openethereum::configuration Using a fixed conversion rate of Ξ1 = US$200.00 (23809524 wei/gas)
2020-07-19 20:54:27 main INFO openethereum::run Starting OpenEthereum/v3.0.1-stable-8ca8089-20200601/x86_64-pc-windows-msvc/rustc1.43.1
2020-07-19 20:54:27 main INFO openethereum::run Keys path h:chainsparityEthereumkeysethereum
2020-07-19 20:54:27 main INFO openethereum::run DB path h:chainsparityEthereumchainsethereumdb906a34e69aec8c0d
2020-07-19 20:54:27 main INFO openethereum::run State DB configuration: fast
2020-07-19 20:54:27 main INFO openethereum::run Operating mode: active
````

I have a whole bunch of certs under _current user_. It's as if some security setting is blocking access to something, but not sure where to look, doesn't seem UAC, or SmartScreen related.

[adria0]

Please, can you check what happens if you run it with --usd-per-eth 200 @sammyp42 ?

[sammyp42]

Please, can you check what happens if you run it with --usd-per-eth 200 @sammyp42 ?

I used the following cmd line options to generate the output provided:

openethereum.exe -ldebug --log-file log.txt --usd-per-eth 200

[adria0]

@sammyp42, please could you try if this version works?

https://github.com/openethereum/openethereum/tree/adria0/rusttls-171

if you do not want to comple it use this:

https://wetransfer.com/downloads/25911a283b14ac90f22fa3c5e18ce01820200721143657/4daa9a7206025f6888a2cbea4d1c576820200721143730/53ce18

[sammyp42]

@sammyp42, please could you try if this version works?

https://github.com/openethereum/openethereum/tree/adria0/rusttls-171

if you do not want to comple it use this:

https://wetransfer.com/downloads/25911a283b14ac90f22fa3c5e18ce01820200721143657/4daa9a7206025f6888a2cbea4d1c576820200721143730/53ce18

I downloaded the binary and it works!

[adria0]

@vorot93

The problem we have now with Windows users is that if the windows certificate store contains a X509 certificate that is not able to process, the client panics. This is solved in hyper-rustls 0.20.0 but this requires introducing futures 0.3, as @niklasad1 said.

If we use 0.17.1 that is the previous version of the current 0.18.0 the only change is that the root certificates are hardcoded, and @sammyp42 verified that it works.

We can wait to #11823 and use tokio-compat and hyper-rustls 0.20.0 but I'm not sure how many changes this will require.

[vorot93]

Let's take the plunge and upgrade to tokio-compat.

[adria0]

Let's take the plunge and upgrade to tokio-compat.

@vorot93 I've been checking what I need to change, it seems that mainly this is in FetchClient. The code is quite complex, please take a look at https://github.com/openethereum/openethereum/blob/master/util/fetch/src/client.rs

FechClient is used in

• PriceInfo
• Private-Tx
• Miner. NotifyWork for WorkPoster
• HashFetch (not used now)
  
  
  
  • Client updater
  
  
• parity_hashContent RPC API (ParitySet)

We can rewrite the logic for FetchClient and BodyReader but is not clear for me if:

a) what is the criteria for selecting which functions move to async (we also use async traits there?). Mainly talking about FetchClient'sBodyReader here.
b) That the logic we rewrite will work exactly in the same way that the previous one. We need to try to make sure that we are goit going to break WorkPoster and PrivateTx in their current deployments.. The first, it's easy, but I'm not sure how we can test PrivateTx well.

I think that is better just to rollback to 0.17.1.

[adria0]

(closed by mistake)