Rusoto: Rusoto fails to build if it is used from other app with `features = ["ec2"]`

Created on 14 Oct 2016  路  6Comments  路  Source: rusoto/rusoto
  • Rusoto 0.18.0, with features = ["ec2"]
  • rustc 1.12.0
  • Arch Linux x86_64

Rusoto 0.18.0 fails to build when this crate is used from other app.

% cd hello-rusoto
% tail -2 Cargo.toml
[dependencies]
rusoto = {version = "0.18", features = ["ec2"]}
% cargo build
   Compiling rusoto v0.18.0
error: failed to run custom build command for `rusoto v0.18.0`
process didn't exit successfully: `/home/tatsuya/rust-projects/hello-rusoto/target/debug/build/rusoto-43d906a2aede9cc8/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'Could not convert JSON in "/home/tatsuya/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_codegen-0.2.7/botocore/botocore/data/ec2/2015-10-01/service-2.json" to Service: Syntax(MissingField("operations"), 16977, 1)', ../src/libcore/result.rs:788
note: Run with `RUST_BACKTRACE=1` for a backtrace.

However, both of the followings succeeded:

  • features = ["s3"] was given instead of ec2

Or

  • Ran cargo build --features ec2 inside Rusoto's source tree.

Please see more details in the following logs.

Versions

% rustc --version
rustc 1.12.0 (3191fbae9 2016-09-23)

% cargo --version
cargo 0.13.0-nightly (109cb7c 2016-08-19)

% cat /etc/os-release
NAME="Arch Linux"
ID=arch
PRETTY_NAME="Arch Linux"
...

% uname -a
Linux mini-arch 4.7.6-1-ARCH #1 SMP PREEMPT Fri Sep 30 19:28:42 CEST 2016 x86_64 GNU/Linux

Steps to Reproduce

Use Rusoto from an app, with features = ["ec2"] specified.

$ cd ~/rust-projects
% cargo new hello-rusoto --bin
     Created binary (application) `hello-rusoto` project
% cd hello-rusoto
% echo 'rusoto = {version = "0.18", features = ["ec2"]}' >> Cargo.toml
% cargo build
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading rusoto v0.18.0
 ...

 Downloading rusoto_codegen v0.2.7
 ...

   Compiling rusoto_codegen v0.2.7
   Compiling rusoto v0.18.0
error: failed to run custom build command for `rusoto v0.18.0`
process didn't exit successfully: `/home/tatsuya/rust-projects/hello-rusoto/target/debug/build/rusoto-43d906a2aede9cc8/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'Could not convert JSON in "/home/tatsuya/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_codegen-0.2.7/botocore/botocore/data/ec2/2015-10-01/service-2.json" to Service: Syntax(MissingField("operations"), 16977, 1)', ../src/libcore/result.rs:788
note: Run with `RUST_BACKTRACE=1` for a backtrace.

But it seems the JSON file _does_ contain operations field.

% cat /home/tatsuya/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_codegen-0.2.7/botocore/botocore/data/ec2/2015-10-01/service-2.json \
    | jq '.operations[].name'
"AcceptVpcPeeringConnection"
"AllocateAddress"
"AllocateHosts"
...

"UnassignPrivateIpAddresses"
"UnmonitorInstances"

% cat /home/tatsuya/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_codegen-0.2.7/botocore/botocore/data/ec2/2015-10-01/service-2.json \
    | jq '.operations | length'
203

It will not fail if features = ["s3"] is specified.

### change to `features = ["s3"]`
% sed -i -e 's/ec2/s3/g' Cargo.toml
% cargo build
   Compiling rusoto v0.18.0
   Compiling hello-rusoto v0.1.0 (file:///home/tatsuya/rust-projects/hello-rusoto)
    Finished debug [unoptimized + debuginfo] target(s) in 12.69 secs

### change back to `features = ["ec2"]`
% sed -i -e 's/s3/ec2/g' Cargo.toml
% cargo build
   Compiling rusoto v0.18.0
error: failed to run custom build command for `rusoto v0.18.0`
process didn't exit successfully: `/home/tatsuya/rust-projects/hello-rusoto/target/debug/build/rusoto-43d906a2aede9cc8/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'Could not convert JSON in "/home/tatsuya/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_codegen-0.2.7/botocore/botocore/data/ec2/2015-10-01/service-2.json" to Service: Syntax(MissingField("operations"), 16977, 1)', ../src/libcore/result.rs:788
note: Run with `RUST_BACKTRACE=1` for a backtrace.

It will not fail if I run cargo build --features ec2 inside Rusoto's source tree.

% cd ~/rust-projects
% git clone --recursive [email protected]:rusoto/rusoto.git
Cloning into 'rusoto'...
remote: Counting objects: 11166, done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 11166 (delta 8), reused 4 (delta 4), pack-reused 11134
Receiving objects: 100% (11166/11166), 15.34 MiB | 558.00 KiB/s, done.
Resolving deltas: 100% (8967/8967), done.
Submodule 'codegen/botocore' (https://github.com/boto/botocore.git) registered for path 'codegen/botocore'
Cloning into '/home/tatsuya/rust-projects/rusoto/codegen/botocore'...
Submodule path 'codegen/botocore': checked out 'ad1cbbc20e6687c7f60710a53ca9e17d84a18820'

% cd rusoto
% git checkout v0.18.0
% cargo build --features ec2
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading serde_codegen v0.8.9
 Downloading aster v0.27.0
 Downloading serde_codegen_internals v0.8.9
 Downloading quasi v0.20.1
 Downloading quasi_codegen v0.20.0
   Compiling itoa v0.1.1
   Compiling utf8-ranges v0.1.3
   ...

   Compiling rusoto_codegen v0.2.7 (file:///home/tatsuya/rust-projects/rusoto/codegen)
   Compiling rusoto v0.18.0 (file:///home/tatsuya/rust-projects/rusoto)
    Finished debug [unoptimized + debuginfo] target(s) in 181.0 secs
bug
Source
picture tatsuya6502

All 6 comments

Thanks for the detailed bug report. I reproduced it with one of my projects using Rusoto.

I compiled my WIP branch of pun-on-rust-demo and everything went fine. Deleted the cargo.lock file in my project and these new packages came along:

Downloading serde v0.8.12
 Downloading regex-syntax v0.3.7
 Downloading matches v0.1.3
 Downloading serde_codegen v0.8.12
 Downloading serde_codegen_internals v0.10.0
 Downloading syn v0.9.0
 Downloading quote v0.3.3

And I then see the same error:

thread 'main' panicked at 'Could not convert JSON in "/Users/matthew/.cargo/registry/src/github.com-1ecc6299db9ec823/rusoto_codegen-0.2.7/botocore/botocore/data/ec2/2015-10-01/service-2.json" to Service: Syntax(MissingField("operations"), 16977, 1)', ../src/libcore/result.rs:788

My gut says the new versions of serde and friends caused this: they were just released a few days ago.

matthewkmayer picture matthewkmayer on 14 Oct 2016
👍1

Looks like you were right.

I cloned serde locally, checked out serde 0.8.11, added serde_codegen, serde, and serde_codegen_internals to "paths" in ~/.cargo/config, and was able to compile my app using Rusoto ec2 0.18.0 successfully.

Switching back to serde 0.8.12 resulted in the MissingField("operations") failure again.

davideagen picture davideagen on 15 Oct 2016

Thanks guys for looking into this!

So for now, we can workaround this by adding the followings to Cargo.toml,

[dependencies]
rusoto = {version = "0.18", features = ["ec2"]}
serde_codegen = "= 0.8.11"

and then run cargo update.

% pwd
/home/tatsuya/rust-projects/hello-rusoto

% cargo update
    Updating registry `https://github.com/rust-lang/crates.io-index`
    Updating quote v0.3.3 -> v0.2.3
    Updating serde_codegen v0.8.12 -> v0.8.11
    Updating serde_codegen_internals v0.10.0 -> v0.9.0
    Updating syn v0.9.0 -> v0.8.7

% cargo build
   Compiling quote v0.2.3
   Compiling syn v0.8.7
   Compiling serde_codegen_internals v0.9.0
   Compiling serde_codegen v0.8.11
   Compiling rusoto_codegen v0.2.7
   Compiling rusoto v0.18.0
   Compiling hello-rusoto v0.1.0 (file:///home/tatsuya/rust-projects/hello-rusoto)
    Finished debug [unoptimized + debuginfo] target(s) in 95.47 secs

% egrep '^ "(serde.*|quote|syn) ' Cargo.lock | sort | uniq
 "quote 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)",
 "serde 0.8.12 (registry+https://github.com/rust-lang/crates.io-index)",
 "serde_codegen 0.8.11 (registry+https://github.com/rust-lang/crates.io-index)",
 "serde_codegen_internals 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)",
 "serde_json 0.8.2 (registry+https://github.com/rust-lang/crates.io-index)",
 "syn 0.8.7 (registry+https://github.com/rust-lang/crates.io-index)",

serde 0.8.12 itself seems OK, but serde_codegen and friends need to be kept in these specific versions.

tatsuya6502 picture tatsuya6502 on 15 Oct 2016

I have at least narrowed this down to a problem with serde_codegen producing some really crazy code. Opened an issue here: https://github.com/serde-rs/serde/issues/590

adimarco picture adimarco on 18 Oct 2016
👍1

I released serde_codegen 0.8.14 with a fix and confirmed that [dependencies] rusoto = {version = "0.18.0", features = ["sqs"]} now compiles.

dtolnay picture dtolnay on 19 Oct 2016
🎉1

I just published a new release 0.18.1 that fixes this.

adimarco picture adimarco on 19 Oct 2016
🎉1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

jyn514 picture jyn514  路  5Comments
lholznagel picture lholznagel  路  4Comments
pantaloons picture pantaloons  路  4Comments
srijs picture srijs  路  6Comments
softprops picture softprops  路  6Comments