Openfoodnetwork: Variant missing from order details but shows on invoice

[tschumilas]

Description

A hub reports that a variant is missing on the order details page. But the item has been purchased, is in the order total, and is listed on the invoice.

Expected Behavior

Items on the order details should match the invoice

Actual Behaviour

Item is on the invoice but not on the order details - but the item was indeed purchased and the cart total is correct

On OFN CAN - Baileys Local Foods - Hub # 80

Steps to Reproduce

Have not been able to reproduce

1.
2.
3.
4.

Animated Gif/Screenshot

Here is the order page - see that gooseberries are not there - but the total includes them

Here is the invoice - see the gooseberries

Workaround

None known

Severity

s2? total is correct, invoice is correct

Your Environment

• Version used:
• Browser name and version:
• Operating System and version (desktop or mobile):

Possible Fix

[RachL]

This is nuts! It's the reverse of https://github.com/openfoodfoundation/openfoodnetwork/issues/6680

S2 I would say yes. At least for investigating.

[andrewpbrett]

I looked into why this might be happening. The code that generates the order page looks at order.shipments:
https://github.com/openfoodfoundation/openfoodnetwork/blob/5d1af5620cc39172d708f35d88907d73c875f294/app/views/spree/admin/orders/_form.html.haml#L5

So in a console:

irb(main):029:0> order.shipments.count
=> 1
irb(main):030:0> order.shipments.first.line_items
=> [#<Spree::LineItem id: 115724, order_id: 188206, variant_id: 21930, quantity: 1, price: 0.288e1, created_at: "2020-07-22 18:15:14", updated_at: "2020-07-22 18:15:14", max_quantity: nil, currency: "CAD", distribution_fee: nil, final_weight_volume: 0.1e1, cost_price: nil, tax_category_id: 2>]
irb(main):031:0> order.shipments.first.line_items.first.variant.product.name
=> "Eggs"

No gooseberries.

Somehow there's only one shipment - normally there would be one for each line item. The invoice, on the other hand, just looks at the line items:

https://github.com/openfoodfoundation/openfoodnetwork/blob/5d1af5620cc39172d708f35d88907d73c875f294/app/views/spree/admin/orders/_invoice_table.html.haml#L13

which is why it appears on the invoice.

There are a few places we delete shipments on an order. The one that looks most suspicious is:
https://github.com/openfoodfoundation/openfoodnetwork/blob/5d1af5620cc39172d708f35d88907d73c875f294/app/models/spree/order.rb#L582-L592

My best guess is that something went awry when we tried to rebuild the shipments for this order, and the gooseberries were not included, possibly because they were out of stock.

I'm out of ideas at the moment but hopefully that will help if someone else wants to pick it up from here. Without more to go on or more reported occurrences, I think it might be safe to downgrade since I don't see anything that we've recently changed that might be causing this or that makes me think it's widespread.

[Matt-Yorkley]

normally there would be one for each line item

I'm pretty sure we have one shipment per order and each shipment can have multiple line items, right?

There's some complexity when modifying orders that have already been placed or already have shipments though. If we change line items without updating the shipment, there will be issues. I think the OrderContents class handles this (changing the line items and shipment correctly on a completed order).

[andrewpbrett]

we have one shipment per order and each shipment can have multiple line items, right?

Yep, that's right, I think it must have been late when I typed that :)

[RachL]

@jaycmb Have you already had time to take a look at it?

[jaycmb]

Yes, we had a conversation with @andrewpbrett.

As the information on the invoice is correct and the one on the order details is not, due to different "sources" of data (details page queries the shipmentswhile invoice the line_items?)
we discussed, if the query for the details page can be adjusted to also use line items for preventing this bug to come up again.
Andy wanted to have a look if there might be a good reason why it´s calculated differently in different places and or it would be "safe" to change it.

[tschumilas]

fyi - there might be another advantage to your suggestion @jaycmb - I noticed with a user today -- when a variable weight item is adjusted, the adjusted description shows up correctly on the invoice, but the original product description (as is in the product list and as shows int he shop) shows up on the order details. So - if my shop lists carrots - 1 kg for $10, customer purchases it, but the product weighs out at .8 kg. I adjust the weight in BOM, and the cost adjusts to $8. The adjusted cost shows correctly on the order details page, and on the invoice. The product description on the invoice says - .8 kg of carrots, but on the order details page (and the order confirmation if it is re-sent to the customer) says 1 kg. So - maybe your solution solves this too????

[Matt-Yorkley]

Alright, so the order has two line items but the order's shipment has only one item.

irb(main):016:0> order.line_items
=> #<ActiveRecord::Associations::CollectionProxy [
  #<Spree::LineItem id: 115724, order_id: 188206, variant_id: 21930, quantity: 1, price: 0.288e1, created_at: "2020-07-22 18:15:14", updated_at: "2020-07-22 18:15:14", max_quantity: nil, currency: "CAD", distribution_fee: nil, final_weight_volume: 0.1e1, tax_category_id: 2>, 
  #<Spree::LineItem id: 115725, order_id: 188206, variant_id: 23627, quantity: 2, price: 0.5e1, created_at: "2020-07-22 18:16:27", updated_at: "2020-07-22 18:16:27", max_quantity: nil, currency: "CAD", distribution_fee: nil, final_weight_volume: 0.2e1, tax_category_id: 2>
]>

irb(main):017:0> order.shipment.manifest
=> [
  #<OpenStruct variant=#<Spree::Variant id: 21930, sku: "Kitchen", weight: nil, height: nil, width: nil, depth: nil, deleted_at: nil, is_master: false, product_id: 7759, position: 2, cost_currency: "CAD", unit_value: 1.0, unit_description: "", display_name: "Extra Large - 1 doz", display_as: nil, import_date: "2020-08-01 18:32:25">, quantity=1, states={"on_hand"=>1}>
]

irb(main):018:0> order.shipment.line_items
=> [
  #<Spree::LineItem id: 115724, order_id: 188206, variant_id: 21930, quantity: 1, price: 0.288e1, created_at: "2020-07-22 18:15:14", updated_at: "2020-07-22 18:15:14", max_quantity: nil, currency: "CAD", distribution_fee: nil, final_weight_volume: 0.1e1, tax_category_id: 2>
]

irb(main):004:0> order.shipment.inventory_units
=> #<ActiveRecord::Associations::CollectionProxy [
  #<Spree::InventoryUnit id: 200911, state: "on_hand", variant_id: 21930, order_id: 188206, created_at: "2021-01-18 20:43:27", updated_at: "2021-01-18 20:43:27", shipment_id: 24251, return_authorization_id: nil, pending: false>
]>

Interestingly, Shipment has it's own implementation of the line_items method here:

https://github.com/openfoodfoundation/openfoodnetwork/blob/2c0e9d77b8424434b4ddaee85c6b5044a5d4aa0f/app/models/spree/shipment.rb#L183-L189

and it explicitly excludes line items that don't correspond to the shipment's inventory_units...

[Matt-Yorkley]

There's a simple way to put an order exactly into this state, which is to edit the line items in the admin order edit page and then _don't_ press the Update and Recalculate Fees button. Not sure if that's what happened here...

There's some pretty bonkers dates associated with this specific order. The order and both of it's line items were created on 22nd July 2020, but it was completed on 3rd February 2021 (six months later?!). The order's shipment and inventory units were created on 18th January 2021 (two weeks before it was completed?!).

[Matt-Yorkley]

So... two items were added to a cart, then it was left in cart state for 6 months, then the order was completed, but with the shipment creation and order completion inexplicably happening two weeks apart. There doesn't seem to be any Subscriptions involved.

That period around July 2020 (when the line items were created) sticks in my head from investigating other anomalies in the data... we were part-way though incorporating Spree, and there was a brief period where we had some weird class-loading issues whilst we had brought in Spree classes but were still loading the Spree gem, and it created a few minor data inconsistencies here and there.

[andrewpbrett]

There's a simple way to put an order exactly into this state, which is to edit the line items in the admin order edit page and then don't press the Update and Recalculate Fees button

Could we have the order do an Update and Recalculate Fees any time the line items are changed then? It might be expensive, but would be better than things getting out of sync; and it probably doesn't happen all that often, otherwise I assume we'd be hearing many more reports of this bug.

The dates thing is also pretty bizarre. Unless @tschumilas has some idea of why it would be a regular occurrence for the order to be completed 6 months after the cart was created, that alone might be enough to downgrade this IMO.

[tschumilas]

This wouldn't be a regular occurence - at least not that I can imagine. I can't explain how it would happen that an order is completed so long after the cart was created. I notice that their shop preferences are set to 'customer can change/cancel orders while the order cycle is open'. I don't think very many hubs select this - so maybe its part of the explanation here. But the order shouldn't be editable once the OC is closed. Unless - there's a bug there? I suspect this hub clones their OCs weekly because they have so many suppliers/products. I'm wondering if cloning an OC and having shop perferences results in an OC being read as open for cart editing? When I get a chance I'll test that out. Regardless - no reason not to downgrade this that I see.

[andrewpbrett]

Thanks for the information @tschumilas 👍 I'll make it an S3 but keep it in Dev Ready In Dev.

@Matt-Yorkley I do think it'd be good to consider whether we could do an Update and Recalculate any time an admin changes line item quantities, unless you think the performance hit would be too large.

[jaycmb]

might be enough to downgrade this IMO.

Agree on the that the dates thing seems to be rather rare.

But if I understood correctly the problem is not related to that, but could happen to any order where a user

edits the line items in the admin order edit page and then don't press the Update and Recalculate Fees button.

Which would actually surprise me that it does not happen more often?

Or is it in this specific case also linked to Spree that

then the order was completed, but with the shipment creation and order completion inexplicably happening two weeks apart.

[tschumilas]

It happens fairly often that I get calls from a user, confused about the order in some way, and the first thing I do is hit re-calculate. I admit to be confused about this -- is it that when an item is added or deleted from an order, some fees associated with the item(s) auto calculate but others don't. I don't have it clear which do and which don't. On my list to test this - but just to say, it does happen a lot that people are confused about the calculation on orders. But the date problem above does not happen often (I've never seen it before). ( I think the hub has to be set so that the buyer can adjust their order for this date problem.)

[andrewpbrett]

Which would actually surprise me that it does not happen more often?

Yeah that does seem surprising. My best guess (especially with what @tschumilas just posted) is that this is the equivalent of the "save" button - people have gotten used to the fact that they need to press it for things to look right, so they don't report it as a bug. I might have some time later today to write a PR to do it automatically if @Matt-Yorkley doesn't get to it first (or hasn't already started).

[andrewpbrett]

Closing via #7299