Openfoodnetwork: Complete product range doesn't load on shop front for UK enterprise

[lbwright22]

Description

The complete product range for this shop front is not loading. Other shop fronts are unaffected.

Shop front: https://openfoodnetwork.org.uk/slow-food-birmingham/shop#/shop
Products get 'stuck' at 'Spring Onions' from Top Barn Harvest Shop

Removing Top Barn Harvest products from the OC resolves the problem- all products load.

Unable to load up Top Barn Harvest Shop products in admin:

Could be a repeat of issue #4537

Expected Behavior

-All products which are in stock should load on shop front
-All products should load in Admin

Actual Behaviour

• only the first products load on shop front. all those after Spring Onions won't load
  -None of Top barn harvest shop products load in admin

Steps to Reproduce

1.
2.
3.
4.

Animated Gif/Screenshot

Workaround

Severity

S1
User has customers unable to order.
Sunday evening.
User has contacted OFN UK team via WhatsApp.
Order Cycle closes tomorrow
Less than 2 weeks til Christmas.

Your Environment

• Version used:
• Browser name and version:
• Operating System and version (desktop or mobile):

Possible Fix

[lbwright22]

Putting back Top Barn Harvest products and just removing the Onions- Spring just means the shop front gets stuck loading on Parsnips.
Consistently only the first 16 Top Barn Harvest products will load if they are added to the order cycle

[lin-d-hop]

This seems to have been caused by one corrupt product:
POTATOES - 5KG WILJA POTATOES

By a process of elimination every other product (all 90+ of them) are back in their shopfront and the shopfront appears to be loading fine.

[lin-d-hop]

This product also doesn't load in the Back Office. Somehow it is very corrupted.
I'm going to stop now but its a pretty high priority to find out how this specific product became so corrupted it broke a shopfront.

POTATOES - 5KG WILJA POTATOES
Producer: Top Barn Harvest
ID: 202066

[Matt-Yorkley]

The variant's weight and unit_value are both: NaN...

[Matt-Yorkley]

It should be fixed now but we need to look at these recurring weight problems.

[Renatacumming]

HI! I have a shop that has just contacted me with the same problem. Happy Underground in Australia; only loading 10 of her 21 products. Based on what I have read here I am going to try removing one product at a time to find the corrupt product. Will update as I go.

[Matt-Yorkley]

Hey @Renatacumming I think it's Zucchini, mint and chilli pickles...

[Matt-Yorkley]

Should be fixed now.

[Renatacumming]

Amazing, thank you Matt! So the producer should now create a new product for the zucchini, mint and chilli pickles?
Edit- I can see they are active, just wondering if the bug has been fixed? Thanks so much!

[Matt-Yorkley]

There was an issue with that specific product which should be fixed now, but we haven't fixed the underlying cause yet, so it might crop up again in other places.

[Matt-Yorkley]

Unless this needs more immediate action for these two cases (please shout if it does) I'll drop it down to an S3 for now, but I think we should deal with this.

The bits we need to look at are:

• replicating these cases where totally invalid products/variants can be created via the UI
• improving some of the handling and validations around these cases so it doesn't continue to happen
• adding some regression tests for these cases

I think it's mostly centered around changing a product from weight to items and back again, and shouldn't be wildly difficult to fix.

[andrewpbrett]

When you noticed that the variant's weight and unit_value were "NaN", was that on the front end? They're stored as numbers (decimal and float) in Postgres, so I'm having trouble figuring how they could have gotten saved that way without a validation kicking in.

[Renatacumming]

I think I might have anther situation- Terrewah Farm (OFN UID 1839) where products are only loading to Oregano on their shop. Is this issue still happening? Is anyone able to help with this?

[andrewpbrett]

The issue with Terrewah Farm was the same issue as before.

I think this should be an S2 since we don't yet have a handle on how/why/when it might happen, and when it does, it can essentially down a shop, with no workaround except raising one of the core devs.

I'm growing suspicious that https://github.com/openfoodfoundation/openfoodnetwork/pull/6431 might have had some unintended side effects. As @Matt-Yorkley noted above, changing a product from weight to items and back again on a staging server works, but not without an error in the javascript console (the "Changes Saved" message doesn't display). And on localhost, I get "Saving failed. Saving failed with the following error(s): Unit value can't be blank"

This was the offending variant, which sort of answers my question about how we could be saving things in Postgres as NaN:

> v
=> #<Spree::Variant id: 44889, sku: "", weight: #<BigDecimal:561075e93690,'NaN',9(18)>, height: nil, width: nil, depth: nil, deleted_at: nil, is_master: false, product_id: 17548, cost_price: nil, position: 3, cost_currency: "AUD", unit_value: NaN, unit_description: "", display_name: "Panforte", display_as: "250g", import_date: nil>
> v.unit_value
=> NaN
> v.unit_value.class
=> Float
> v.weight
=> #<BigDecimal:561075e93690,'NaN',9(18)>

I'll have to set this down for now but can pick back up in the morning.

[Matt-Yorkley]

I had a quick crack at a possible solution for this this a while ago here: https://github.com/openfoodfoundation/openfoodnetwork/pull/6369/files but there were some failing specs. The main issue seems to be the variant's unit_value ending up as nil, and in that case we can just set it to 1 whilst saving (if the product's variant_unit has changed from weight to items).

The issue with validations is that the objects are essentially composite; that is: changing something on the product can leave the product valid but the variant invalid. They're co-dependent. So in theory when we change a product's variant_unit we would need a validation that also checks if the product's variants are still valid before allowing the product to be saved, which gets a bit messy...

[Renatacumming]

Hi, (I suspect) we are seeing this problem again; the shop is Prom Coast (UID 1965) and they can only load part of their product (only to around the Tone Made products). I don’t know how to check the code for the product but if someone can help me investigate and fix this problem we would be most grateful. They are a significant shop for us.

[filipefurtad0]

Thanks for reporting @Renatacumming ,

This is the request causing the error:

https://www.openfoodnetwork.org.au/api/order_cycles/7710/products?distributor=1965&page=22&per_page=10&q[name_or_meta_keywords_or_variants_display_as_or_variants_display_name_or_supplier_name_cont]=

Bugsnag:
https://app.bugsnag.com/yaycode/openfoodnetwork/errors/5fd4909d813f6300171bc752?filters%5Bevent.since%5D%5B%5D=30d&filters%5Berror.status%5D%5B%5D=open&event_id=5ffb891600659c16a9890000

[Renatacumming]

Thanks Filipe, is this the same issue as this bug? Can it be fixed? I'm sorry but I am not a developer or have any experience in this space so I do not know what I am looking at or what to do. Any help appreciated!

[andrewpbrett]

I believe this is the same issue, there was a variant with NaN as its weight and unit_value. I've fixed that product for now but we will likely keep seeing this crop up until we address the root cause. There's a good chance that #6369 will fix this once it's reviewed and merged.

[andrewpbrett]

This doesn't fit neatly into the bug categories; when it happens, it's an S1 though. I've upped it back to an S2 to try to move the fix along.

[filipefurtad0]

Issue #6369 seems to prevent new corrupt data to appear - this appears to relate to how data was created. So maybe it's ok to downgrade the severity to s3 on this one and move it back to All the things for now?

[andrewpbrett]

I would suggest that we:

• deploy #6369 (next Tuesday)
• query all managed instances for any variants where unit_value or weight is NaN
• either close this issue entirely, or as @filipefurtad0 suggests move it down to S3 and back to All the things.

I would lean toward closing it, since I think that once #6369 is deployed, it will be assumed that this is fixed unless we see another occurrence, at which point we can reopen it.

[filipefurtad0]

Hey @andrewpbrett ,
Thanks for the feedback.

query all managed instances for any variants where unit_value or weight is NaN

This is a great idea - sounds like a production test, right? Would be great to have an estimate of these occurrences _before and after_ the deploy of #6369, next Tuesday.

Let's close this one once we have the query data, ok? I'll downgrade it and add the prod-test tag, keeping it here in Test ready.

[filipefurtad0]

There is a new occurrence of this bug in UK:

https://openfoodnetwork.org.uk/api/order_cycles/207112/products?distributor=201203&page=2&per_page=10&q%5Bname_or_meta_keywords_or_variants_display_as_or_variants_display_name_or_supplier_name_cont%5D=

Taken from Bugsnag:
https://app.bugsnag.com/yaycode/openfoodnetwork-uk/errors/60000a13eb43d0001761847d?event_id=60000a130065a789482d0000&i=sk&m=nw&pivot_tab=event

[andrewpbrett]

Thanks for pointing out that new occurrence @filipefurtad0; I manually fixed that one.

Basically the query is Spree::Variant.all.select { |v| v.unit_value&.nan? }. We wouldn't expect the deployment of this PR to change the number of variants in the result of that query, but we would expect (hope?) that once it's deployed, we won't see that result be anything but zero variants ([]) again.

[RachL]

I'm closing this one and adding 2 things:

• A tasks in tomorrow's release issue
• A reminder on slack to run the query again in 2 weeks :)

[lbwright22]

I've found another occurrence of this bug.
Hub: Kent Food Hubs Ashford. It is one of the My Wild Jars products.

I think it is the Smoke + Spice Beets. I can't access the products to look. I'll update the issue but it was one I think which should be fixed by the release this week? Is it possible one of these products is the 'bug' which is causing problems with the UK deployment? Sorry for my ignorance.

[andrewpbrett]

The release didn't get deployed as usual, so this is (fortunately!) not evidence that the fix didn't work.

I've manually updated that product for Kent Food Hubs Ashford.

The bug causing problems with the deployment is a separate one. There's a fix on the way, we're testing it today and deployment should happen later tonight (US Pacific time zone, so in about 9 hours).

[andrewpbrett]

@lbwright22 when I fixed that variant earlier, I also looked for any other variants that were corrupted. There were 6, so I fixed them all.

I checked again after deploying this fix and there are currently 0 corrupt variants in the UK database. So if we see another one pop up, we'll know that this bug is still not fixed.

We have a reminder to run the query again in two weeks, so that we're not relying on people to notice the corrupt variants. I'd say that if we check all instances again in two weeks and there are no corrupt variants, we can consider this definitively closed.

After deploying v3.5.4 just now (which included this fix), I checked all managed instances for corrupt variants. The only instance that had one was FR (ping @RachL) - the supplier was id: 849, name: "La Bulle de chèvre". I fixed that variant so if it was present in a shopfront, it should load fine now.

[lbwright22]

Reopening this bug (Sorry).

Hub: Kent Food Hubs (Ashford and Folkestone)
Producer: My Wild Jars
Not sure which variant/product

Hub manager reports having edited a variant and then everything going wrong again.

[lin-d-hop]

@lbwright22 Can you ask for screenshots as to how the user edited the variant? Ask her to include the URL bar of the browser in the screenshot :pray:

[Matt-Yorkley]

Should be fixed now. I'll take another look at this.

[filipefurtad0]

We have another occurrence reported by @eugenichafer , I think this would correspond to this Bugnsnag error:
https://app.bugsnag.com/katuma/katuma/errors/600c8d9e294a3300185af079?event_id=600c8d9e006997abd2b10000&i=sk&m=nw

[eugenichafer]

Hello. I tried to narrow down the problem a bit more. In the back-end we also have the problem, and it seems that it is only with the products of a specific produce. But I don't see how I can edit these products if I can't access them.

[RachL]

@eugenichafer can you provide names of producer, name of hub and ID for all them? This is what will help developers to narrow down the products.

[eugenichafer]

Producer is 490 "Thymus Organic Soaps"
Hub is 352 "Mengem Garrigues".
Now, I have seen that I can access to these products if I don't select producer on https://app.katuma.org/admin/products

[eugenichafer]

It seems to be corrected now. Thank you !

[lbwright22]

Another instance of this in the UK
hub: Alcester Country market
producer: Potter's Preserves by Sally White

I don't know which product it is. Is some one able to find out and sort it? The user has no idea what they are editing and so it is difficult to pin point the page or place they managed to corrupt the product

[lin-d-hop]

I'm not following closely on this, but just sharing this mess. Users should not be able to save data like this:

[lbwright22]

Another occurrence is:
Sole of Discretion
Product : Wild Alaskan Pink Salmon

[filipefurtad0]

Thank you for reporting @lbwright22 .
I've opened issue #6737 to address validation when changing the variant unit of a product. But I'm not sure this is the only way to create corrupt items, so I'm raising the severity again on the present issue.

[Renatacumming]

I have another occurrence in Australia and the products page is also not loading. The producer is Gran Elly Orchard UID 2499.

in Australia

[JenSheridan]

The Aus one is coming up with these Console errors: https://openfoodnetwork.org.au/api/products/bulk_products.json?import_date=&page=1&per_page=15&q%5Bname_cont%5D=&q%5Bprimary_taxon_id_eq%5D=&q%5Bs%5D=&q%5Bsupplier_id_eq%5D=

https://openfoodnetwork.org.au/api/products/bulk_products.json?import_date=&page=1&per_page=15&q%5Bname_cont%5D=&q%5Bprimary_taxon_id_eq%5D=&q%5Bs%5D=&q%5Bsupplier_id_eq%5D=2499

[kirstenalarsen]

[lbwright22]

I have another in the UK:
Supplier: Delea Fermented Foods
Hub: Liskeard Hive

[lbwright22]

Occurred again for the same hub: Liskeard HIve
pretty sure the corrupt product is from Landmark Specialty Coffee

[lbwright22]

New occurrence:
Producer: RESTOCK Kent
Page 3 onwards of the product list won't load

[Renatacumming]

Similar error current:
The entire shopfront is not loading
Shopfront: https://openfoodnetwork.org.au/westies-dry-goods-buying-group/shop#/shop

Please note this order cycle is for registered customers only, if you are not registered you get a message to that effect, if you are you get the spinning wheel. Customers noticed this issue last week, we have only just been notified overnight.