Venus: [bug] Asset failed to be downloaded

[NatoBoram]

Describe the bug

Command FILECOIN_USE_PRECOMPILED_RUST_PROOFS=true go run ./build/*.go deps fails.

To Reproduce

1. sudo apt install snapd pkg-config
2. sudo snap install go --classic
3. curl https://sh.rustup.rs -sSf | sh
4. go get -u -v -fix github.com/filecoin-project/go-filecoin
5. cd ~/go/src/github.com/filecoin-project/go-filecoin
6. FILECOIN_USE_PRECOMPILED_RUST_PROOFS=true go run ./build/*.go deps
7. See error

Expected behavior

Dependencies should be installed.

Screenshots

./scripts/install-rust-proofs.sh
using precompiled rust-proofs
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5685  100  5685    0     0  24399      0 --:--:-- --:--:-- --:--:-- 24399
./scripts/install-rust-proofs.sh: line 10: jq: command not found
./scripts/install-rust-proofs.sh: line 18: jq: command not found
curl: (3) <url> malformed
basename: missing operand
Try 'basename --help' for more information.
curl: (3) <url> malformed
asset failed to be downloaded
failed to find or obtain precompiled rust-proofs, falling back to local
~/go/src/github.com/filecoin-project/go-filecoin/proofs/rust-proofs ~/go/src/github.com/filecoin-project/go-filecoin
cargo 1.32.0-nightly (5e85ba14a 2018-12-02)
    Updating crates.io index
    Updating git repository `https://github.com/zcash-hackworks/sapling-crypto`
    Updating git repository `https://github.com/dignifiedquire/rust-gperftools`
    Updating git repository `https://github.com/filecoin-project/merkle_light`
    Updating git repository `https://github.com/gtank/blake2-rfc`
   Compiling filecoin-proofs v0.1.0 (/home/nato/go/src/github.com/filecoin-project/go-filecoin/proofs/rust-proofs/fileco
in-proofs)
error: failed to run custom build command for `filecoin-proofs v0.1.0 (/home/nato/go/src/github.com/filecoin-project/go-
filecoin/proofs/rust-proofs/filecoin-proofs)`
process didn't exit successfully: `/home/nato/go/src/github.com/filecoin-project/go-filecoin/proofs/rust-proofs/target/r
elease/build/filecoin-proofs-5c0848e970cff4bd/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'Unable to find libclang: "couldn\'t find any valid shared libraries matching: [\'libclang.so\
', \'libclang-*.so\', \'libclang.so.*\'], set the `LIBCLANG_PATH` environment variable to a path where one of these file
s can be found (invalid: [])"', src/libcore/result.rs:1009:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.

~/go/src/github.com/filecoin-project/go-filecoin
cp: cannot stat 'proofs/rust-proofs/target/release/paramcache': No such file or directory
cp: cannot stat 'proofs/rust-proofs/target/release/libfilecoin_proofs.a': No such file or directory
cp: cannot stat 'proofs/rust-proofs/target/release/libfilecoin_proofs.pc': No such file or directory
Command './scripts/install-rust-proofs.sh' failed: exit status 1
exit status 1

Desktop:

nato@hp-server
--------------
OS: Ubuntu 18.04.2 LTS x86_64
Host: HP G62 Notebook PC 0598120000202710010020100
Kernel: 4.15.0-46-generic
Uptime: 2 hours
Packages: 649
Shell: bash 4.4.19
Terminal: /dev/pts/3
CPU: Intel i3 M 370 (4) @ 2.399GHz
GPU: Intel Core Processor
Memory: 947MiB / 3744MiB

Additional context

Related to #2000.

---

Fix :

sudo snap install jq

[schomatis]

Hey @NatoBoram! Thanks for the detailed report. There are a couple of issues to unpack here, but the one you may care about the most is that jq is not installed by default in many Linux distributions (it's used here to form the URL from where the script will download the compiled Rust binaries).

[schomatis]

Additionally,

1. jq should be listed as a dependency, or at least we could mention this problem in the troubleshoot guide.

2. It sounds to me that the script should stop at the first error to make it a bit more clear to the user what's going on (the curl was issued with a malformed URL and after that we can't expect any sane behavior).

3. There's the LLVM dependency, but that's also expected if building from source (since downloading the binaries failed).

[joshgarde]

Running Ubuntu 16.04 LTS here and got the same jq error, however install continued for me without it

[schomatis]

Bumping the priority of this a bit to give it more visibility, it's something that will probably come up repeatedly.

[NatoBoram]

Installing jq solved this problem. I'll let you close it when https://github.com/filecoin-project/go-filecoin/pull/2007/ is merged. Thanks!

[rosalinekarr]

As a temporary measure, let's update the README to recommend installing jq on linux systems.

In the future, we should try to remove this dependency entirely.