In CONTRIBUTING.md:
You can run tests directly with node:$ ./node ./test/parallel/test-stream2-transform.js
This bit is a little problematic and can trip up new folks.
parallel and sequential) but not others (such as message).--FLAGS comment, then that needs to be taken into account on the command line or else the test won't work.We could clarify all this, or we could just remove that bit of text entirely and instruct people to always use test.py. That second option seems like the better one to me. Discuss.
@nodejs/testing
By the way, this definitely comes up for real users.
I don't know, it's still useful info for people trying to debug segfaults. The test runner only gets in the way then.
Given that the first method suggested (directly above) is to run the test with tools/test.py, I think we could just say that this alternative method doesn't always work without going into extensive detail.
If we are going to have more documentation about what the test runner does, it might be worth putting that somewhere else (like test/README.md) and just linking to said info from CONTRIBUTING.md.
Maybe extend doc/guides/writing-tests.md a little?
The way to run a single test using test.py was not obvious to me at first (test.py parallel/test-stream2-transform), I would welcome this in the documentation.
@joaocgreis It's immediately above the text we're talking about in this PR:
If you are updating tests and just want to run a single test to check it, you can use this syntax to run it exactly as the test harness would:$ python tools/test.py -v --mode=release parallel/test-stream2-transform
There's probably room to improve that text and/or make it more noticeable.
@joaocgreis FYI, since https://github.com/nodejs/node/pull/9694, you can use the full path, e.g. tools/test.py test/parallel/test-stream2-transform.js, which should make it more obvious.
@Trott my bad, thanks for pointing it out.
I agree with @gibfahn above, we could remove this from CONTRIBUTING.md, leaving only instructions to run with test.py and a link to test/README.md. There, add some more information (a note about --FLAGS, a a note about message and https://github.com/nodejs/node/blob/master/test/message/testcfg.py, ...).
Most helpful comment
I don't know, it's still useful info for people trying to debug segfaults. The test runner only gets in the way then.