Factory_bot: Reset factory_girl sequences to start with “1”

[PeterBloom]

I would like to be able to reset factory_girl sequences to start with “1” if needed.

I use Cucumber tests with lots of Scenario Examples. I might use factory_girl to seed a test db with 10 or so objects (with a numbered sequence, so object id => 1 might have attribute name => name1, etc.) I can run my first Cucumber scenario example with no problem (db has objects with attributes name1 to name10). When the second scenario example runs, the db factories, but this time the object attributes are name11 to name20. I would really like to be able to start the sequence at “1” for each scenario example.

Cucumber::Rails::World.use_transactional_fixtures = true is in the env.rb

There is a nice monkey patch that deals with this (and some other nice factory_girl sequence functionality), but I would sure like to see sequence reset functionality available in the official factory_girl gem without a patch.

The patch is here…
http://www.pmamediagroup.com/2009/05/smarter-sequencing-in-factory-girl/

[vinayvinay]

did anyone try that patch out with the 'factory_girl_rails', "1.0" on rails 3? i couldn't get it to work.

[jferris]

This seems like an antipattern to me. You shouldn't depend on the generated sequence values. If you need a particular value in your cucumber scenario, I'd specify that value explicitly. Can you explain why referencing the generated value is necessary?

[joshuaclayton]

Closing due to inactivity; I also agree with Joe that it seems like an antipattern.

[Adamantish]

@jferris The anti-pattern is having state leaking between test examples.
Test frameworks and add ons like database cleaner go to a lot of effort to ensure a fresh slate at the beginning of every test example but factory girl doesn't respect it.

It's reasonable to want to use sequence to, for instance, generate a daily history with dates going back an arbitrary number of days. In that case I will specify exactly how many dates I need and I expect them to have the same values every time.

I mean, does this look good to you?

trait :with_deposit_withdrawals do
      transactions do
        grrr = []
        grrr << create(:deposit_withdrawal, executed_at: 15.days.ago, amount: 12_000)
        grrr << create(:deposit_withdrawal, executed_at: 12.days.ago, amount: 7_000)
        grrr << create(:deposit_withdrawal, executed_at: 9.days.ago, amount: 2_000)
        grrr << create(:deposit_withdrawal, executed_at: 6.days.ago, amount: -3_000)
      end
end

[jferris]

Test frameworks and add ons like database cleaner go to a lot of effort to ensure a fresh slate at the beginning of every test example but factory girl doesn't respect it.

I agree that cleaning your database in between scenarios makes sense, but I think there's a limit to what you should assume about a fresh database. For example, it would be a mistake to depend on a record inserted in the previous scenario, but would you want to write a test that assumed that the first record you inserted always had an id of 1?

I could see having something in between tests that started off sequences at a random value, to make sure that tests didn't accidentally depend on a particular value.

I mean, does this look good to you?

I frequently create little helper methods for creating related bits of data. I agree that it's pretty heavy as written, but I would find it very confusing if executed_at and amount were assumed to be specific values because of a sequence declared outside the spec. For example, I would have no idea what was happening here:

5.times { create(:deposit_withdrawal) }
expect(DepositWithdrawal.minimum(:amount)).to eq(-3_000)

The purpose of the attributes declared in factories is to hide the parts you don't care about. Anything you actually rely on in the test makes more sense in the test itself. When I see create(:user), I think, "this is just a user, and any user will do." When I see create(:user, email: "[email protected]"), I think, "that particular email address will be important somehow."

[gtd]

@jferris 6 years later I want to present an opposing viewpoint based on what I just experienced.

The scenario I have is that I am screen scraping a site with numbered records. Those numbers don't matter to me except for a small condition in the codebase that will fail with a very quiet log message when a duplicate number pops up. This is deemed the correct logic for the real world.

In my tests I have a number of VCR cassettes recorded of real world data. Most of these have high numbers, so _usually_ they don't conflict with the factory-generated numbers. Maybe you can see where I'm going with this...

What happens in my tests is that they pass 99% of the time, but then every once in a while, the planets align, and the sequence number generated by a particular test conflicts with a recorded VCR cassette.

In order to code defensively against this I have to comb through the tests and make these numbers explicit for all tests that _might_ fail. It's non-trivial to determine which tests might fail, and it's definitely not a core concern for the vast majority of them, it's just a single nasty edge case waiting to happen.

My question is why should sequence state carry over between tests at all? Is there a good use case for that behavior? I understand the arguments for test order randomization, but I don't think the same reasoning applies to sequences since they are in a very real sense leaking state between tests. Perhaps you could at least consider providing sequence functionality that _does_ reset between tests.

[Adamantish]

@jferris makes some good points. The idea of factory girl is indeed that create(:user) is so brief that it shouldn't hide complex considerations of whether this is the first time you've called it or second. Unfortunately it has to matter a bit because of database uniqueness constraints.
I would argue along with @gtd that once you've had to break that ideal there need to be other concessions to impurity to support and limit the abstraction leakage. The first would be accepting that yes, it will sometimes make most sense to rely on deterministic behaviour relating to order of creation. When we write tests it's normal to assert something like expect(User.count).to eq 2. We all know perfectly well that 2 is just the arbitrary number resulting from the specific actions in that test. We know it's not a magic number that applies in the app which has many more users. In the same way if I test for expect(id).to eq 2 this is just a good shorthand for my understanding of how the real app works with much more than 2 records. It just saves me having to diff the before and after.

Behaviour will be deterministic in the real app so why not allow the option to make it so in FG? If only for the sake of consistency with every other testing tool in our Gemfiles. Principle of least surprise, etc..

[jferris]

Maybe an API for changing how values are providing to sequences would be acceptable? That would allow you to call a method in a global before(:each) hook the same way you do with Database Cleaner.

[gtd]

👍, would work for me.

[Adamantish]

@jferris Nice.

[gareth]

I just wanted to leave a note about our use case for deterministic sequences. We use a visual regression tool that highlights page changes between deployments. Since our test order is random – specifically to help identify fragile tests – that currently means we get failures because [email protected] becomes [email protected] when the same test was run at a different point in the test sequence.

I agree with all the comments about not writing your tests to expect particular sequence values to be generated, but that's not the only case where the order has an effect.

[obie]

Hi everyone,

Having a similar issue to @gtd with VCR, but in my case, what I need is deterministic control of IDs so that I can ensure that when my VCR cassettes are used, the data that they contain is able to match up with known records in my test database.

Trying to mock interactions with getstream.io -- if you look at what it does (activity streams in the cloud) then you can probably start to imagine the pain I'm currently in.

Cheers,
Obie

[odlp]

Rewinding sequences (https://github.com/thoughtbot/factory_bot/commit/50eeb67241ea78a6b138eea694a2a25413052f49) has been released in version 4.10.0.

Please feel free to reopen this issue if you need to.