Openfoodnetwork: Fix unreliable features test

Created on 17 Apr 2018  路  8Comments  路  Source: openfoodfoundation/openfoodnetwork

Description

As exposed in https://github.com/openfoodfoundation/openfoodnetwork/pull/2232 recent changes may have dropped the following tests reliability:
https://github.com/openfoodfoundation/openfoodnetwork/blob/ac37b0fc7e7203477ad59f61081b393c69fdfc26/spec/features/consumer/shopping/embedded_shopfronts_spec.rb#L50-L97

It fails way more than it passes in CI.

Expected Behavior

The test should pass either like this or refactored.

Actual Behavior

It fails most of the times on CI, not so sure about development environment.

Steps to Reproduce

Open a PR, no matter what you do, and check it out

Severity

Not having this test may have an impact on our test suite's coverage so this needs to be done sooner rather than later, so assigning a severity 3.

Possible Fix

We need to find out what happened to it, fix it and then enable it back. The fact of being a so long feature test greatly increases the chances of running into strange failures.

tech debt

All 8 comments

We don't normally use severity levels for these issues so please, @myriamboure let me know if this feels right to you.

Sounds ok to me...I guess we can use severity labels to know the emergency of a tech debt to be solved...? It's a bit hacky, but let's review that when we go. Maybe @daniellemoorhead will have ideas ;)

I guess we can use severity labels to know the emergency of a tech debt to be solved...?

That's my point. Let's what the authority, @daniellemoorhead , has to say.

Let's prioritize it @sauloperez we want to have some things in the pipe occasional devs can jump on. I propose we put this one in, so moving to delivery backlog.

Rob has made some great progress using headless Chrome as browser for testing: https://github.com/openfoodfoundation/openfoodnetwork/compare/master...oeoeaio:headless-chrome

Since he won't be working on this for another month, I'm willing to take over. This has been really annoying and it costs us a lot of time reviewing and staging pull requests.

Hey there, so I have removed the bug label...because it's not a bug in the _traditional_ sense of the word. However, I do agree that we need to understand whether tech debt items are a priority and need to be done, so that they are added to the delivery pipe.

I think in the same way we sort bugs we also need to manage a tech debt backlog, and ensure that things like this are prioritised alongside of bugs and alongside of features.

Thoughts @myriamboure @sauloperez?

We need to find out what happened to it, fix it and then enable it back. The fact of being a so long feature test greatly increases the chances of running into strange failures.

@mkllnk will the work Rob has done and that you're taking over fix the problem Pau has identified? Or does further work need to be done to find out what happened and fix it?

I think it will fix it. But maybe it's already fixed since we merged #2468. Let me test.

Was this page helpful?
0 / 5 - 0 ratings