Wasm-pack: `wasm-pack test` running tests multiple times

Created on 18 Jan 2019 · 10Comments · Source: rustwasm/wasm-pack

🐛 Bug description

My tests seems to be running multiple times in Firefox instead of just once.

$ wasm-pack test crates/virtual-dom-rs --firefox --headless

  [1/6] 🦀  Checking `rustc` version...
  [2/6] 🎯  Adding WASM target...
  info: component 'rust-std' for target 'wasm32-unknown-unknown' is up to date
  [3/6] 🌀  Compiling tests to WASM...
      Finished dev [unoptimized + debuginfo] target(s) in 0.47s
  wasm-bindgen 0.2.31 (b04f60cf2)
  [4/6] ⬇️  Installing wasm-bindgen...
  [5/6] Getting geckodriver...
  [6/6] Running tests in Firefox...
      Finished dev [unoptimized + debuginfo] target(s) in 0.44s
       Running /Users/chinedufn/Development/opensource/percy/target/wasm32-unknown-unknown/debug/deps/virtual_dom_rs-9c24f0dc38062301.wasm
  no tests to run!
       Running /Users/chinedufn/Development/opensource/percy/target/wasm32-unknown-unknown/debug/deps/create_element-c831bfe39cb431f1.wasm
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
Waiting for test to finish...

  test create_element::click_event ... ok
  test create_element::div_with_properties ... ok
  test create_element::nested_divs ... ok

  test result: ok. 3 passed; 0 failed; 0 ignored
       Running /Users/chinedufn/Development/opensource/percy/target/wasm32-unknown-unknown/debug/deps/diff_patch-ac2fa5a869b98ee0.wasm
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
Waiting for test to finish...

  test diff_patch::replace_with_children ... ok
  test diff_patch::append_sibling_text_nodes ... ok
  test diff_patch::append_text_node ... ok
  test diff_patch::text_node_siblings ... ok
  test diff_patch::append_children ... ok
  test diff_patch::remove_attributes ... ok
  test diff_patch::truncate_children ... ok
  test diff_patch::replace_child ... ok

  test result: ok. 8 passed; 0 failed; 0 ignored
       Running /Users/chinedufn/Development/opensource/percy/target/wasm32-unknown-unknown/debug/deps/events-35fa407b92336d6e.wasm
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
Waiting for test to finish...

  test events::on_input ... ok
  test events::on_input_custom ... ok

  test result: ok. 2 passed; 0 failed; 0 ignored
|
no tests to run!
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
running 3 tests

test create_element::click_event ... ok
test create_element::div_with_properties ... ok
test create_element::nested_divs ... ok

test result: ok. 3 passed; 0 failed; 0 ignored
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
running 8 tests

test diff_patch::replace_with_children ... ok
test diff_patch::append_sibling_text_nodes ... ok
test diff_patch::append_text_node ... ok
test diff_patch::text_node_siblings ... ok
test diff_patch::append_children ... ok
test diff_patch::remove_attributes ... ok
test diff_patch::truncate_children ... ok
test diff_patch::replace_child ... ok

test result: ok. 8 passed; 0 failed; 0 ignored
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
running 2 tests

test events::on_input ... ok
test events::on_input_custom ... ok

test result: ok. 2 passed; 0 failed; 0 ignored

🤔 Expected Behavior

Tests run once

👟 Steps to reproduce

git clone https://github.com/chinedufn/percy
wasm-pack test crates/virtual-dom-rs --firefox --headless

🌍 Your environment

Include the relevant details of your environment.
wasm-pack version: 0.6.0
rustc version: $ rustc 1.33.0-nightly (2d3e909e4 2018-12-22)

PR attached bug user report wasm-pack test

Source

chinedufn

All 10 comments

@chinedufn thanks for filing this! we'll take a look soon- i've assigned @fitzgen because he knows the most about the test command. i'll attempt to repro now and comment here with my experience. could you let us know what operating system youre on? it might not matter but it's always nice to have all the possible info!

ashleygwilliams on 18 Jan 2019

oh also- looks like you are running test on https://github.com/chinedufn/percy ? this may be an odd side effect of running the command in the workspace (again not totally sure, but some relevant info for sure!)

ashleygwilliams on 18 Jan 2019

can reproduce on my machine!

Ashleys-MacBook-Pro-2:percy ag_dubs$ wasm-pack test crates/virtual-dom-rs --firefox --headless

  [1/6] 🦀  Checking `rustc` version...
  [2/6] 🎯  Adding WASM target...
  info: component 'rust-std' for target 'wasm32-unknown-unknown' is up to date
  [3/6] 🌀  Compiling tests to WASM...
      Finished dev [unoptimized + debuginfo] target(s) in 0.28s
  [4/6] ⬇️  Installing wasm-bindgen...
  [5/6] Getting geckodriver...
  [6/6] Running tests in Firefox...
      Finished dev [unoptimized + debuginfo] target(s) in 0.25s
       Running /Users/ag_dubs/rust/percy/target/wasm32-unknown-unknown/debug/deps/virtual_dom_rs-9e58ab35df2eb75a.wasm
  no tests to run!
       Running /Users/ag_dubs/rust/percy/target/wasm32-unknown-unknown/debug/deps/create_element-c363d3307c12c5d1.wasm
Executing bindgen...
Visiting http://127.0.0.1:51206...

  test create_element::click_event ... ok
  test create_element::div_with_properties ... ok
  test create_element::nested_divs ... ok

  test result: ok. 3 passed; 0 failed; 0 ignored
       Running /Users/ag_dubs/rust/percy/target/wasm32-unknown-unknown/debug/deps/diff_patch-bea8606093b0cd09.wasm
Executing bindgen...
Visiting http://127.0.0.1:51242...

  test diff_patch::replace_with_children ... ok
  test diff_patch::append_sibling_text_nodes ... ok
  test diff_patch::append_text_node ... ok
  test diff_patch::text_node_siblings ... ok
  test diff_patch::append_children ... ok
  test diff_patch::remove_attributes ... ok
  test diff_patch::truncate_children ... ok
  test diff_patch::replace_child ... ok

  test result: ok. 8 passed; 0 failed; 0 ignored
       Running /Users/ag_dubs/rust/percy/target/wasm32-unknown-unknown/debug/deps/events-2dad164255426644.wasm
Executing bindgen...
Visiting http://127.0.0.1:51279...

  test events::on_input ... ok
  test events::on_input_custom ... ok

  test result: ok. 2 passed; 0 failed; 0 ignored
|
no tests to run!
Running headless tests in Firefox with `/Users/ag_dubs/Library/Caches/.wasm_pack/geckodriver-daada96429827dae/geckodriver`
running 3 tests

test create_element::click_event ... ok
test create_element::div_with_properties ... ok
test create_element::nested_divs ... ok

test result: ok. 3 passed; 0 failed; 0 ignored
Running headless tests in Firefox with `/Users/ag_dubs/Library/Caches/.wasm_pack/geckodriver-daada96429827dae/geckodriver`
running 8 tests

test diff_patch::replace_with_children ... ok
test diff_patch::append_sibling_text_nodes ... ok
test diff_patch::append_text_node ... ok
test diff_patch::text_node_siblings ... ok
test diff_patch::append_children ... ok
test diff_patch::remove_attributes ... ok
test diff_patch::truncate_children ... ok
test diff_patch::replace_child ... ok

test result: ok. 8 passed; 0 failed; 0 ignored
Running headless tests in Firefox with `/Users/ag_dubs/Library/Caches/.wasm_pack/geckodriver-daada96429827dae/geckodriver`
running 2 tests

test events::on_input ... ok
test events::on_input_custom ... ok

test result: ok. 2 passed; 0 failed; 0 ignored

Ashleys-MacBook-Pro-2:percy ag_dubs$

ashleygwilliams on 18 Jan 2019

what operating system you're on

MacOS

Definitely stealing this dropdown for more details idea!

this may be an odd side effect of running the command in the workspace

Sounds good - other than that migrating my workspace tests to wasm-pack has been seamless.

chinedufn on 18 Jan 2019

👍1

cool (i'm also on OSX and yeah details tags are handy)!

is this a behavior you just started seeing happen in the workspace with 0.6.0? if that's the case, i would imagine that means the workspace is the culprit! (it'd be helpful to know if you just started seeing this and it wasn't happening before on a single crate, just to help eliminate some stuff)

ashleygwilliams on 18 Jan 2019

So it turns out that I was previously using the wasm-bindgen and wasm-bindgen-test binaries directly since all of my projects use workspaces.
I just started migrating to wasm-pack this morning.

Sorry that's all I've got on that front - happy to provide any other info that might be useful though!

chinedufn on 18 Jan 2019

👍1

no worries, no more info is a type of info still :) - glad to have you using wasm-pack! would love to hear if you have any other suggestions that may have come up during your migration!

ashleygwilliams on 18 Jan 2019

👍1

If I strace -f -e trace=execve wasm-pack test --firefox --headless I only get wasm-pack exec'ing cargo test one time, which makes me think this is the wasm-bindgen-test-runner doing something funky maybe. Or a bad interaction between wasm-pack and wasm-bindgen-test-runner somehow.

fitzgen on 18 Jan 2019

👍1

Hmmm thank you to both of you for looking into this!

So a few quick things that I'm noticing in case it helps.

The first time we see a batch of tests we're also printing the path to the .wasm binary, the second time we are not:

Running /Users/chinedufn/Development/opensource/percy/target/wasm32-unknown-unknown/debug/deps/diff_patch-ac2fa5a869b98ee0.wasm
Running headless tests in Firefox with `/usr/local/bin/geckodriver`
Waiting for test to finish...

  test diff_patch::replace_with_children ... ok

Running headless tests in Firefox with `/usr/local/bin/geckodriver`
running 8 tests

test diff_patch::replace_with_children ... ok

There is also slightly different copy. Waiting for test to finish... vs. running 8 tests so that could also be a clue.

Second thing that I see is this pipe character in the output

  test result: ok. 2 passed; 0 failed; 0 ignored
|
no tests to run!

From a quick search of the codebase the only pipe that I can find that gets printed out is in src/progressbar.rs

cd wasm-pack
rg '\|'
// ... snip ...
src/progressbar.rs
93:                .tick_chars("/|\\- ")

Of course - absolutely no idea if any of this is at all relevant - just pointing out anything that might help y'all think about where this could be stemming from.

chinedufn on 18 Jan 2019

👍1

I suspect it is just the output being printed twice, and not the tests actually being run twice. Will test this hypothesis later today.

fitzgen on 22 Jan 2019

Was this page helpful?

0 / 5 - 0 ratings