Rspec-core: Make --random the default in rspec 3

[myronmarston]

The --random option is awesome and I use it on all my projects. I imagine the main reasons it's not universally used is that it's relatively new (and thus, lots of folks don't know about it) and on older projects that do have accidental ordering dependencies, it could take a bunch of effort to fix all the ordering dependencies.

When we added it in a minor rspec 2.x release it made sense not to have it on by default (since it could have caused lots of breakage for people when upgrading) but I think we should make it the default in rspec 3.

Thoughts?

[dchelimsky]

I actually think it's not a good default because it's confusing for someone who is unaware. I'd rather raise awareness by mentioning it in the readme and possibly making it the default in generated artifacts (like spec/spec_helper.rb), but not within rspec-core itself.

[alindeman]

It's the default for minitest, and has definitely saved me from a non-obvious bug or few.

I think it's a good idea: I'd maybe rather folks be a little more confused upfront than write bad specs? Or is that not a good position to hold?

[myronmarston]

I guess it's partially a question of the intention of the defaults: are they the settings recommend by the rspec core team? Are they meant to be optimized to reduce confusion for newcomers? Something else?

[dchelimsky]

Generally, I think defaults should follow the principle of least surprise.

[justinko]

I'm afraid if we don't introduce randomization as the default, it will never be widely adopted. Users will not "seek out" the different ways to order their examples...

Many users are not even aware of "order-dependent tests"; I wasn't until Myron introduced this potential feature for RSpec many months ago.

[pjg]

I'm +1 on making random the default. I feel like enforcing better/more reliable tests is the right approach. Even though it's harsher on the user... and will definitely introduce a wave of "My specs are failing randomly after upgrading to latest RSpec" posts/questions.

[probonogeek]

Randomness breaks the "specs as documentation" that you get when running specs in nested format (which I do a lot) and I also think it makes debugging all that much harder. We've been converting over nearly 3000 specs from rspec1 to rspec2 (yes, I know we are late) and I would have flipped out if I had to deal with random specs every time. I would agree with David, teach users about the functionality, don't force it.

[rodrigotassinari]

+1 on making random the default, same reasons of @pjg

[revans]

I also think random order should be the default for rspec. I use it on all my projects.

I think it violating principle of least surprise is debatable. As a testing framework, having it not test order-dependency violates that principle for me. It [rspec] should be guiding the users/developers towards best practices.

[wallace]

+1 for random default

[chrismcg]

There's two ways I could be surprised here.

Say random is on by default, I could be writing a spec and it fails because I accidentally introduced an order dependency. This may be confusing to me until I understand specs are been run in a random order but the failure happens close to where I was coding so I can hopefully fix it straight away.

Say random is off by default. I'm writing the same spec with the same order dependency but I don't trigger a failure because specs are running in the same order. I'm happy with my feature and some months later something happens to trigger a different spec order, now the spec blows up but it's not going to be as clear why or as easy to find and fix.

The second scenario has happened to me months and even years later, so I'd like to see random being on by default so I don't have to switch it on myself.

[rosenfeld]

-1 - for some integration tests it makes sense to me to rely on prior tests to be successful.

Take a look at this discussion I've started on Buster.js as well:

https://github.com/busterjs/buster/issues/194

[jwo]

I am team-random on this one.

I've had specs only work in a certain order before (my fault, obviously), and I like that minitest randomizes the order for you. And, I'm one of the people that didn't even know you could randomize rspec until recently -- that's why defaults are so important.

I submit that the test for defaults shouldn't be, "keep the number of complaints to a minimum", it should be "have the best product" In my opinion, that's randomizing the tests to promote test isolation.

[nwjsmith]

+1 for random default. RSpec already encourages good tests, this would just further that.

[jrgifford]

+1 for random default.

[tonyc]

I like @dchelimsky's idea, new projects will have it as a sensible default (and people won't have order-dependent tests), but "legacy" users will still get their old, order-dependent tests passing on an upgrade. Seems like a reasonable balance.

I can see both sides of the issue - I like the idea of making sure your tests aren't order-dependent, but wouldn't it make it harder to catch ordering-related test failures if each run is randomly ordered?

On the surface, it seems that making specs randomly ordered aren't a guarantee of order-independency (If that makes sense). It seems like it could also lead to "phantom" CI failures if an obscure order-related bug is introduced that fails a build, and then everybody else thinks tests are running clean.

[pjg]

BTW I think that after making random the default, an option like --ordered could be added, which should satisfy the naysayers ;)

[sobrinho]

-1

I think that should be an option, but not the default behavior, following the principle of no surprises by default.

[DCarper]

Generally speaking I think things should default to best practice.

I think the ordering should default to random. Is anyone going to argue that it's not better going forward? If it breaks old test suites and people don't want to fix the issues themselves, they can always specify the file system's ordering

Also, to me personally I don't think it would be out of place to include this as a configurable option in spec_helper as opposed to a command line option, lots of which is focused on formatting and specifying which examples to run. This might help to increase awareness of the option itself.

One other thought on making it part of spec_helper instead of the command line utility is that the order you want to run tests in probably doesn't change? You probably make the decision to run in random order or not. Whereas the output formatting / filtering / tags / things like debugger change nearly almost every time you use the rspec command.

[rosenfeld]

Integration tests can run really slow under some circumnstances. You can get your specs running faster if you assume some pre-conditions created by your previous examples instead of recreating that situation all over again (which could be quite slow) or by the use of lots of mocks (which can make your examples more complex and more error-prone as well).

[cdemyanovich]

@probonogeek How does random order break the documentation format for you? I just tried running specs with --format documenation --order random. It appears that example groups are randomized and then examples within the groups are randomized. Which examples go with which groups is preserved, though.

Since this change is proposed for a major release and not a minor or patch one, I favor making randomness the default. Hopefully, people are consciously upgrading to a major release after reading announcements and/or change logs and basing their decision on the value that the new release will add to their project.

[darrinholst]

+1 on random being the default. It would have helped us recently when our ci workspace got moved to another mount and caused the specs to be run in a different order which uncovered a test dependency. Would have been nice to fail fast.

[srushti]

What I would like to have is, to be able to rerun the specs in an order that just failed. Basically, otherwise when I screw up and I have test A screwing up the system, which causes test B to fail, the test results will only show test B failing while the problem will actually be in test A.

If I can re-run it in the same order it'll be easier to narrow down the problem.

[hgmnz]

+1. In general the framework should encourage good tests. By setting random to default, we are saying "You must write tests that teardown correctly"

[myronmarston]

What I would like to have is, to be able to rerun the specs in an order that just failed. Basically, otherwise when I screw up and I have test A screwing up the system, which causes test B to fail, the test results will only show test B failing while the problem will actually be in test A.

You can already do this. Rspec prints out the random seed that it used at the end of the test run, and you can re-run with --seed 1234 (or whatever) to reproduce the bug exposed by the randomization.

[jeremygpeterson]

+1 to make it the default for new projects.

[lmcalpin]

+1 for random default. tests should not depend on order and if the framework encourages that, that's a good thing.

[seancribbs]

If you are setting up integration tests by the results of previous tests in the same group, you're doing it wrong and your suite will one day break spectacularly. What if you run an example in isolation to debug it, and then it breaks in a different way? I think that's more surprising than finding the second breakage earlier. Besides, we have shared contexts for examples that need the same setup.

Needless to say, I'm very :+1: for random by default, as long as the seed is properly set on JRuby automatically (in basho/riak-ruby-client I have to seed with the current time).

[goshakkk]

+1 for making --random the default.

[lsegal]

-1.

There's a disparity between "ideal conditions" and pragmatic results here. A perfect test suite that has no inter-dependency issues is the ideal case, and we should _strive_ for that, but we don't need to assume it's "perfect code or bust". Furthermore, random by default punishes new users and doesn't really affect those who strive for perfection. Experienced users who care about independent tests can easily opt into --random, but new users will be seriously confused by tests that fail randomly. The comment about using --seed N to lock into a test run is nice, but I seriously doubt a new user would be able to figure this out on their own. Adding randomness to test runners completely throws "the element of least surprise" out the window.

There's also the problem of making tests more brittle by assuming they are independent. This causes maintainers to get false positives from users running specs on other machines, and also CI machines, at _random_ intervals. The random part means the maintainer can't immediately identify if the failure is a legitimate regression, or an isolated test build issue that only affected that one user/build. That's a lot of extra strain on maintainers who often get bug reports and notifications about broken builds. This is why I personally run tests in ordered mode, but occasionally do a few _local_ tests with --random to make sure the tests are well formed. That way, I can guarantee that what works locally will work everywhere while still cleaning up the tests if there are inter-dependency issues. I don't think a problem with your build environment should cause random failures unless you explicitly ask for that kind of testing. That just seems wrong to me.

I should also add (and reiterate what was mentioned above) that --random doesn't even guarantee that you will identify inter-dependency issues. It just means that at some point in the future, you _may_ identify the issue. That point in the future may be minutes, hours or even days, and that's a problem, because you must always be ready to take time out of development to debug issues that might be completely unrelated to the code you are currently working on. I don't know if that's a good default behaviour.

[DCarper]

Encouraging bad test practice is wrong.

@isegal I define brittle exactly the opposite as you do. If I modify one test and that breaks another, that is extremely brittle to me, not the other way around ;/

Making it easier for the less experienced to write dependent and brittle tests is wrong.

Writing dependent integration tests for performance bonuses is also not right in my opinion.

Speaking to the "Defaults should follow the least surprises theory"... Walking that argument down the far end of the spectrum, if one doesn't want surprises, then one shouldn't be using 3rd party code altogether.

The only legitimate reason for not defaulting to random is breaking existing test suites at which point I say just specify the default ordering or don't upgrade.

[JoshCheek]

-1

Same reason as @probonogeek My --format documentation order gets mixed up. I sometimes rely on that ordering. E.g: 'it does a when A', 'it does b when B', 'it does c for everything else'

[cmaujean]

+1 for default randomness. randomizing test order ensures that tests aren't order dependent. "Least surprise" adherents should be reminded that the real surprise is finding out that your tests are order dependent when you weren't intending them to be. A simple "Tests will be run in random order by default" in the README should be enough to mitigate any surprise you might have, and reading the documentation is a key part of properly using any library.

On the down side, I love documentation mode, and generally have that set in my .rspec file. This would kill some of the value of that, unless everything was run randomly and then the documentation was spit out in the proper order. I would be happy with a compromise such as this.

[rosenfeld]

Let me do a wild guess here (although I truly believe the guess is valid): how many of you on the pro-random side are actually writing full integration specs for JavaScript intensive applications (like one-page apps) and are using jQuery live events?

For those under such scenario, I'd really appreciate an article on how you write full integration specs for your complex applications (with drag-and-drop, etc) while not relying on examples running order.

Also, I'm pretty curious on how much your integration test suite takes to complete so that you can run your tests in any order. How many seconds/minutes/hours would that be?

[ahawkins]

random default. Mini test has caught order dependent bugs before

[rosenfeld]

Then, suppose that you'll need about 1s for setting up a context (reload the page, set up database, etc) and have 10 single examples that would take about 1s to run in a certain order. You'd spend 10s to run the same examples just to make them independent from each other.

If I write my specs like these I'll stop writing specs as they would take forever to run my specs...

[lsegal]

@DCarper, we're talking about brittle in different contexts: running vs writing. The question is which do you want to optimize for? Given that tests are run far more often than they are written, I would think you'd optimize to have them run robustly. In other words, what you run is what you get. That tends to cause much fewer surprises in my experience. In your case, if you modify one test and it breaks another, you have at least isolated the inter-dependency to a specific set of tests. That would not be the case for --random, where two sequential runs could fail for _wildly_ different reasons. If my tests are going to fail at all, I'd rather have them fail in the more deterministic manner-- and at least only when I've changed something, not just because I've re-run the tests a week later.

Also, not using --random does _not_ imply that you are encouraging bad tests. It's not a black and white issue. As I pointed out, you don't need to tie your build process to your test results, you can ensure good tests in parallel with reliable test results.

[lsegal]

@rosenfeld you can write before blocks that perform your fixture setup for a group of tests. That shouldn't be affected by ordering.

[justinko]

FYI everyone, RSpec randomization isn't "true" randomization. The groups are randomized, and the examples within each group are randomized. To put it another way, examples are only randomized within their group.

If RSpec supported "true" randomization, before(:all) wouldn't work.

[rosenfeld]

I was not saying it was impossible. Just that it would be painfully slow for a JavaScript intensive application that also need lots of records to exist in the database...

[justinko]

Let me clarify why I said that.

If RSpec supported "true" randomization, it _would_ break the documentation formatter. The current randomization implementation does not affect the formatters in any way.

[DCarper]

@lsegal we were talking about two different things. I still think that if test B passes when run after test A, but fails if run after test C, that's brittle.

It seems like people are saying, "Our tests are slow, we'll fix that by sacrificing test quality."

Performance aside, is anyone actually arguing that it's better practice to have order dependent tests?

I also love the argument that all of the extra false positives one will get from order-dependent tests represent far worse surprises than finding out rspec defaults to a random ordering at some point in the TDD / Rspec learning curve.

[lsegal]

@DCarper you seem to be arguing on boolean logic here. Writing good software always involves a balance of competing interests. The question isn't whether order dependent tests are or aren't better practice; I don't think you would get pushback on that question. The problem is that there are cost/benefit questions to all things that require extra effort, and not all projects prioritize this best practice as worth the effort of more complicated debugging. For example, documenting your code is also generally considered a "best practice", but most developers would not consider a tool that defaulted to asking you to document every method in your codebase as an acceptable default-- not because documentation is unimportant, but because the cost is usually much higher than the benefit.

The real question is whether --random _effectively_ encourages the best practice while also making it easy enough to deal with when things go wrong. Is there any actual data in the wild about how randomized tests affect _code quality_? Note: not build quality, _code quality_. As I mentioned before, independent tests is a best practice that every developer should absolutely _strive_ for-- but I'm not personally convinced that it's a practice developers should _start with_ as a priority above other practices, like, say, spending that effort to improve test _coverage_ rather than _build quality_.

[DCarper]

@lsegal I see what you're saying and I totally agree, all best pratices / rules / whatever, are definitely a matter of balance and contextual feasibility.

Random ordering isn't directly about code quality, it's about test quality which in turn affects code quality.

For the less experienced, it's encouraging good test-practice instead of _less than good_. If someone does reach the point where they they're making a conscience decision to trade quality for performance, that's fine, but at least let them opt into that decision.

I think defaulting to random is completely consistent with the Ruby / Rails way. Provide and push as many best practices for free and let the individual decide when they don't make sense.

[radar]

+1

I ran into an issue with Spree's tests just yesterday where one test was relying on another test running before it to set up data for it. I think this is a sign for a broken test.

Indicating to people that it's random during the run should be enough. A seed is also a great idea.

[gicappa]

-1

IMO rspec shouldn't teach anything or strive to inspire peoples' tests to perfection.

It should just be a awesome tool (as it is) for any kind of people: for the ones who use it right by the book but for all the others who mess up integration unit acceptance and whatever kind of test you name.

[oldfartdeveloper]

+1

Though I'm in sympathy with the contrarians that argue that it might be too costly in certain situations, I fall on the side that it will save over the long run in most cases. That being said, I agree that this "feature" needs to be documented prominently where noobe's will see it with an opportunity to opt out of --random.

I also like the idea of the default being assigned in spec_helper.rb

[captainpete]

Random by default catches more bugs, which is good.
Random by default does not catch _all_ the dependency bugs and it's dangerous to assume that it will, by default.
This proposed default encourages folk to write tests that do not have side-effects.
There's an opt out.

+1

[dnagir]

No, for local development. And here's why:

• least surprise - I want to see specs in the order I write them
• lost time to match output to the specs, I'll have to spend more time to "fix" randomisation
• unreliable failures/successes - yes, you'll find the _shared state_ issues, but the time from previous point is longer than this, not worth it.
• slower TDD - because we have to deal with random output

But I would say Yes for CI server since all the points above don't make sense there.

Most people use RSpec primarily as a TDD tool. Making random a default will make them less productive.

As a result, I think it should be off by default.
There's no big deal enabling it.

I hope this won't become a drama similar to returning boolean in rails :)

[radar]

No @dnagir, I (respectfully/rspectfully?) disagree with everything you said. Here's why.

least surprise - I want to see specs in the order I write them

You only want to do this because you're used to this now. Personally, I don't alphabetise my development. One day I'll be working on product_spec.rb and then the next adjustment_spec.rb. Alphabetisation is not how _normal_ people work. Your argument is silly.

lost time to match output to the specs, I'll have to spend more time to "fix" randomisation

This is time you would inevitably lose anyway because your specs are poorly written. Consider the randomisation as battle testing for your specs. If they are written as they should be, then they would not fail based on the order. Read this very carefully: _No spec file should depend on setup from another spec file to execute_. They should be executable _individually_.

unreliable failures/successes - yes, you'll find the shared state issues, but the time from previous point is longer than this, not worth it.

With the --seed option you will be able to run the specs in an identical order, allowing you to track down failures caused by random specs.

slower TDD - because we have to deal with random output

How? The same specs will run, it'll only be the order that's changed. By changing the order of three operations, taking 1, 3 and 5 seconds long each, doesn't magically make it take any more/less than 9 seconds. Basic math, really.

But I would say Yes for CI server since all the points above don't make sense there.

I actually _agree_ with this point. I want my shit to break on CI because CI is sanity-checking my code and preventing "Works on _my_ machine!" style responses.

Most people use RSpec primarily as a TDD tool. Making random a default will make them less productive.

Simply untrue. Time will be saved by people learning to write specs _properly_ and not spent tracking down failures in their specs because this one time they chose to run that spec file individually rather than with others.

As a result, I think it should be off by default.

Strongly disagree. There'd be much table flipping from this side if the "off-by-default" crew got their way.

There's no big deal enabling it.

Well, actually... GO TO LINE 1.

I hope this won't become a drama similar to returning boolean in rails :)

I hope also, but if the opposing team keeps coming up with such ridiculous arguments then I can see it heading that way for sure.

[maxim]

It's important to run specs randomly, yet I like my non-deterministic behaviors to be explicitly enabled by me.

+0 (whatever, folks)

[dnagir]

@radar, can't argue with you, simply expressing my thoughts on this, but...

working on product_spec.rb and then the next adjustment_spec.rb. Alphabetisation is not how normal people work.

That's not what I said (well, or at least meant).
For example, I often create contexts/specs that are logically ordered by the rule of "happy path", "avg path", "wrong path".

It would be surprising to see those in random order every single time.
It is not how human brain seems to work.

With the --seed option you will be able to run the specs in an identical order

Good point. Didn't think about it at first. I still find that I rarely have shared state issues in my specs though :)

slower TDD - because we have to deal with random output

How? The same specs will run, it'll only be the order that's changed. By changing the order of three operations, taking >> 1, 3 and 5 seconds long each, doesn't magically make it take any more/less than 9 seconds. Basic math, really.

Not what I said, I didn't talk about speed of specs, or runtime.

I was talking about the fact that I have to see the output and try to make sense of what is where and why it is not where I saw it last time.

This is what makes the TDD _process_ slower (not the runtime).

Making random a default will make them [TDD guys] less productive.

Time will be saved by people learning to write specs properly and not spent tracking down failures in their specs because this one time they chose to run that spec file individually rather than with others.

random won't teach anybody to write specs well. It may teach not to leak a state.

95% of the time I work against a single spec and I want output to be consistent so I don't have to waste my time even thinking about why order has changed.

So why do I have to bother with random output if CI will run it with random and full suite will probably run it with random too...

[JoshCheek]

I already said -1, but I'll say again for another reason: People who want it on are perfectly capable of turning it on. It is not a hurdle for someone with experience and familiarity to turn it on. Newbies who don't understand what is happening will be confounded by errors that only sometimes occur.

It is easier for experts to turn it on than it is for newbies to turn it off. Don't stack the deck against newbies, there are enough hurdles for them to deal with. Default config should not get in your way, even if in the end, most people agree that some option is better used than not.

If you're saying +1 because you want it, then you're being short-sighted, there is already a mechanism for you to set defaults, just echo '--order random' >> ~/.rspec This is a question about what you think everyone else should have, including the people who don't understand how to turn it off.

[beneggett]

+1 to --random being default. It's absurd to want tests to depend on other tests by design.

[dnagir]

@beneggett the way you phrase it changes the meaning of what random does :)

I don't think known order of output makes anybody _want_ specs to depend on any others.
Nobody wants that.

Neither known nor random order will change that.

It's the output that is a problem.
And really, it's only 1 simple command to enable it by default for yourself as @JoshCheek pointed out: echo '--order random' >> ~/.rspec

[thibaudgg]

+1 for randomness by default!

[mlakewood]

+1 and I dont even program in ruby.

I see it like this.

1. I want my tests to test my code.
2. If i've managed to put shared state in there (this could be as simple as not removing something from somewhere) then I want my tests to fail.
3. If my tests pass when there is shared state, then I dont think they are testing correctly.

In reply to steep learning curve for newbies, I think its far preferable to have a test fail than to have a bug come out of the woodwork in production. Reliable tests stop this happening. Randomised test runs make this happen in the medium to long term.

[dchelimsky]

I'm not surprised to see such an overwhelming bias toward including this as the default, but I haven't seen any arguments for doing so that have swayed my position. Echoing my earlier argument, and those who have made similar arguments, it's easier for all of you who prefer --random to add it than it is for someone who doesn't want it (or know what it is) to take it away. We can still make it the default in the generated spec/spec_helper.rb, so new projects will always get this as a default, but it's managed in a place that's obvious to end-users so it's easy to turn it off, and existing projects won't see a surprise change in this behavior. This strikes me as a good compromise, even if it means it will take longer for more users to adopt this feature.

There are also sufficiently different (and reasonable) opinions about when to use --random (dev/ci always/sometimes) that I think rspec should leave this to the user.

[rosenfeld]

Okay, after a good night of sleep and getting older - thus wiser ;) - since it was my birthday yesterday, I woke up with a different opinion about this matter.

I'd rather prefer to entirely remove the "--random" or "--sequential" option. This is too limiting in the sense that it will be applied to all specs.

I'd rather prefer to set this in a per describe/context statement because this is an important description of your specs suite.

For example, if you write:

describe "Some feature", order_dependent: true do
  ...
end

You're telling others reading your code that your code was intentionally designed to run those specs in a certain order.

More than that. If you try to run a single example of that suite, RSpec could warn you that the example may fail because it depends on previous specs, or even run all examples before that specific requested example.

This would be much more flexible.

For example, in some setups you'll want to enable the "js" option for all specs under "spec/requests" (or integration, acceptance, whatever). In the same sense we could enable the "order_dependent" option for some kind of specs as the default and then disable this option in some "describe"/"context" of your requests specs if they don't need to be ordered.

I see an application test suite as part of the app documentation and that's why I'd prefer to read the order option in spec_helper.rb or directly in the examples instead of an option of the rspec command.

That way, this option would mean more to those reading it. Consider someone reading it on spec_helper.rb:

config.order = :random

This means that the specs were specifically designed so that examples are independent from each other and if it is not the case, then it is a bug.

In the other hand, if the order is set to :sequential this will be read as the specs were specifically designed to be run in a given order and you should expect them to fail if you try to run a single example without running the previous ones.

Maybe RSpec could support an option like --run-previous-examples for specifying how single examples should be run when order_dependent is set.

[rosenfeld]

Then, I wouldn't mind if config.order = :random was present in the default generated spec_helper.rb.

[adzap]

-1 I think that @dnagir made himself clear and agree with all his points. And @probonogeek remarks about specs as documentation should not be underestimated.

As I develop I read through the output of my specs to review that my expression of the behaviour reads well, is correct, and that I have covered all the behaviour in each context. I can do this from the spec code, but the clarity of the output is important, stripped of the code.

Testing idealism as whole can be left to those who want to achieve it (such as myself). For RSpec as a BDD/TDD tool, should aim to lead people to high quality, well described tests first, then progress to higher realms of testing, such as randomisation for interdependence. High quality does mean avoiding interdependent tests, but the random approach can't be enshrined as deterministic practice to improvement

Test interdependence is about accidental(?) sharing of global state (database, constants, files on disk). We should be teaching people to assert on the state of the system before it is changed.

[myronmarston]

I originally opened this issue, and after seeing this play out and thinking about it some more, I'm with @dchelimsky on this one. I think I'll always use --random, and I think the majority of people should (unless you have a specific reason not to, but I'm not convinced by any of the reasons given in this thread, frankly)...but it's far easier for those of us who want the option to turn it on than for a new rspec user to know how to turn it off. So count me as +1 for having it added to any generated files.

@rosenfeld -- I opened #636 for your idea to allow the ordering to be overriden by individual example groups. Great idea!

[rosenfeld]

thanks, @myronmarston :)

[GeekOnCoffee]

A new rspec user shouldn't be presented with "tests running in the same order is the norm" because it shouldn't be. Running tests in random order should be the standard, except when there's a really good reason not to...

People don't discover flags unless they realize it's something they need, which is why you strive for ideal defaults. In this case I'm definitely for random by default

[nbryan]

:+1:

In my major work project I know we have order-dependent tests, but this might just prompt me or one of my co-workers to fix them.

[lsegal]

I am for the compromise solution that @dchelimsky proposed. Having it in a generated file will make new users aware of exactly what it is they are getting.

[leifbladt]

:+1: for the spec/spec_helper.rb approach

[gerrywastaken]

Furthermore, random by default punishes new users [...] Experienced users who care about independent tests can easily opt into --random, but new users will be seriously confused by tests that fail randomly.

New user here who stumbled onto this topic by chance. I think the notion that this would "confuse" or "punish" new users is a strawman. How do you think new users learn to use Rspec? They are going to watch or read a tutorial. Don't you think these tutorials will mention this default? Secondly, even if we only ever ran into outdated tutorials, it's going quickly become obvious when we see the random ordering of failing tests.

If anything new user's like myself are punished by not having best practice enabled by default. Not having it enabled will lead to the formation of bad habits and bad tests till one day two years down the track we happen upon the --random switch and instantly start cursing Rspec for not enabling it by default.

+1

[dchelimsky]

If anything new user's like myself are punished by not having best practice enabled by default.

--random is _not_ a best practice. It is a great tool in certain contexts. As you can see from this thread, there are diametrically opposed positions on when it should and should not be used and none of them are universally right or wrong.