Prysm: Node crash and stop

[wimel]

🐞 Bug Report

Description

Following the instructions on participate have this error:

FATAL main: Could not create validator at path:  error=open /.shardwithdrawalkeya9aaf4df5d1b.tmp898422365: permission denied

The node crash and stop ### Has this worked before in a previous version? I don't know Yes, the previous version in which this bug was not present was: .... ## 🔬 Minimal Reproduction After clone the repo and runing `./prysm.sh validator accounts create` with a simple password `123456789` the node crash and stop ## 🔥 Error

unable to store key
github.com/prysmaticlabs/prysm/validator/accounts.NewValidatorAccount
        validator/accounts/account.go:68
github.com/prysmaticlabs/prysm/validator/accounts.CreateValidatorAccount
        validator/accounts/account.go:155
main.main.func1
        validator/main.go:106
gopkg.in/urfave/cli%2ev2.(*Command).Run
        external/in_gopkg_urfave_cli_v2/command.go:167
gopkg.in/urfave/cli%2ev2.(*App).RunAsSubcommand
        external/in_gopkg_urfave_cli_v2/app.go:381
gopkg.in/urfave/cli%2ev2.(*Command).startApp
        external/in_gopkg_urfave_cli_v2/command.go:248
gopkg.in/urfave/cli%2ev2.(*Command).Run
        external/in_gopkg_urfave_cli_v2/command.go:85
gopkg.in/urfave/cli%2ev2.(*App).Run
        external/in_gopkg_urfave_cli_v2/app.go:261
main.main
        validator/main.go:196
runtime.main
        GOROOT/src/runtime/proc.go:203
runtime.goexit
        GOROOT/src/runtime/asm_amd64.s:1373
could not initialize validator account
github.com/prysmaticlabs/prysm/validator/accounts.CreateValidatorAccount
        validator/accounts/account.go:156
main.main.func1
        validator/main.go:106
gopkg.in/urfave/cli%2ev2.(*Command).Run
        external/in_gopkg_urfave_cli_v2/command.go:167
gopkg.in/urfave/cli%2ev2.(*App).RunAsSubcommand
        external/in_gopkg_urfave_cli_v2/app.go:381
gopkg.in/urfave/cli%2ev2.(*Command).startApp
        external/in_gopkg_urfave_cli_v2/command.go:248
gopkg.in/urfave/cli%2ev2.(*Command).Run
        external/in_gopkg_urfave_cli_v2/command.go:85
gopkg.in/urfave/cli%2ev2.(*App).Run
        external/in_gopkg_urfave_cli_v2/app.go:261
main.main
        validator/main.go:196
runtime.main
        GOROOT/src/runtime/proc.go:203
runtime.goexit
        GOROOT/src/runtime/asm_amd64.s:1373

## 🌍 Your Environment **Operating System:**

  
Ubuntu Bionic Beaver (18.04 LTS) (GNU/Linux 4.15.0-99-generic x86_64 )
  

What version of Prysm are you running? (Which release)

  
master
  

Anything else relevant (validator index / public key)?

[farazdagi]

Thanks for reporting.
When creating account what is the default path used (I assume you use the default path)?

[farazdagi]

• The #5707 should fix this kind of errors (by requiring directory to exist)
• Another relevant PR is #5592

@wimel can you please retry and let me know if the problem persists

[wimel]

• The #5707 should fix this kind of errors (by requiring directory to exist)
• Another relevant PR is #5592

@wimel can you please retry and let me know if the problem persists

Hi!! sorry for delay and thanks. Yes I use the default path (add image to check).
With the latest release I don't have any problems, _at the moment_ :1st_place_medal:

[farazdagi]

Ok, thank you! Closing the issue, as it it works for you now.

[wimel]

Ok, thank you! Closing the issue, as it it works for you now.

After the upgrade some errors appear, the node don't stop for this error, just add more info in case it help you

2020-05-03 08:36:01] ERROR sync: Failed to close stream error=read tcp4 10.19.98.141:13000->1XX.2XX.3XX.4X:13000: read: connection reset by peer
[2020-05-03 08:36:02]  INFO initial-sync: Waiting for enough suitable peers before syncing required=3 suitable=2
[2020-05-03 08:36:07]  INFO initial-sync: Waiting for enough suitable peers before syncing required=3 suitable=2
[2020-05-03 08:36:08]  WARN powchain: Subscription to new head notifier failed error=websocket: close 1006 (abnormal closure): unexpected EOF
[2020-05-03 08:36:12]  INFO initial-sync: Waiting for enough suitable peers before syncing required=3 suitable=2
[2020-05-03 08:36:14]  INFO sync: Peer has sent a goodbye message Reason=fault/error peer=16Uiu2HAkv5d1HxGf8V9gfGyaspz3APUzwz3B6W2K5CG141UQASxB
[2020-05-03 08:36:14] ERROR sync: Failed to close stream error=read tcp4 10.19.98.141:13000->1XX.2XX.3X.4XX:13000: read: connection reset by peer
[2020-05-03 08:36:17]  INFO initial-sync: Waiting for enough suitable peers before syncing required=3 suitable=2
[2020-05-03 08:36:18] ERROR powchain: Could not connect to powchain endpoint error=websocket: bad handshake (HTTP status 503 Service Unavailable)
could not dial eth1 nodes
github.com/prysmaticlabs/prysm/beacon-chain/powchain.(*Service).connectToPowChain
        beacon-chain/powchain/service.go:340
github.com/prysmaticlabs/prysm/beacon-chain/powchain.(*Service).waitForConnection
        beacon-chain/powchain/service.go:382
github.com/prysmaticlabs/prysm/beacon-chain/powchain.(*Service).run
        beacon-chain/powchain/service.go:605
github.com/prysmaticlabs/prysm/beacon-chain/powchain.(*Service).Start.func1
        beacon-chain/powchain/service.go:237
runtime.goexit
        GOROOT/src/runtime/asm_amd64.s:1373
[2020-05-03 08:36:22]  INFO initial-sync: Waiting for enough suitable peers before syncing required=3 suitable=2
[2020-05-03 08:36:24]  INFO powchain: Connected to eth1 proof-of-work chain endpoint=wss://goerli.prylabs.net/websocket
[2020-05-03 08:36:27]  INFO initial-sync: Waiting for enough suitable peers before syncing required=3 suitable=2
[2020-05-03 08:36:30]  WARN powchain: eth1 client is not syncing

[wimel]

Hi @farazdagi another error, and node crash and stop. Do you think is good to open another ISSUE?

This is the output

Using go1.14.2 on Ubuntu 18.04.4 LTS - 4.15.0-99-generic x86_64

[farazdagi]

Re-openning for investigation:

• the first issue with inability to connect to ETH1 chain
• for the second issue @wimel it seems that error out put is trimmed (missing head section with error), if it (or another issue) reappears - please feel free to post a comment.

[farazdagi]

• The ETH1 connectivity issue is due to our ETH1 nodes being overloaded with requests i.e. there's no issues in code, you just need to point to good running/non-overloaded ETH1 nodes (infura or local geth) by specifying the http-web3provider and web3provider flags. It needs to be properly documented though.
• When it comes to the second issue, @wimel when/if it re-occurs, please, create another bug report -- it will be easier to track it separately.