Openfoodnetwork: Build Enterprise Fee Summary Report

[kirstenalarsen]

https://docs.google.com/spreadsheets/u/1/d/1pnJwHgBo2sokdcwNoLquYlVh8ICUcx6X4J2ZAMnRyio/edit?usp=sharing
Sheet "THIS By Customer"

Fee Type | Enterprise Owner | Fee Name | Customer | Fee Placement | Fee calc on trans through | Tax Category | $$ SUM
-- | -- | -- | -- | -- | -- | -- | --
Admin | Coord 1 |   | Customer A | Coordinator | All |   | 2.3
Admin | Coord 1 |   | Customer B | Coordinator | All |   | 4.7
Admin | Hub 2 |   | Customer A | Outgoing | Hub 2 |   |  
Admin | Hub 2 |   | Customer B | Outgoing | Hub 2 |   |  
Mgmt | Coord 1 |   | Customer A | Incoming | Supplier 2 |   |  
Mgmt | Coord 1 |   | Customer B | Incoming | Supplier 2 |   |  
Packing | Supplier 1 |   | Customer A | Incoming | Supplier 1 |   |  
Packing | Supplier 1 |   | Customer B | Incoming | Supplier 1 |   |  
Transport | Coord 1 |   | Customer A | Incoming | Supplier 2 |   |  
Transport | Coord 1 |   | Customer B | Incoming | Supplier 2 |   |  
Transport | Hub 3 |   | Customer A | Outgoing | Hub 3 |   |  
Transport | Hub 3 |   | Customer B | Outgoing | Hub 3 |   |  
Transport | Supplier 1 |   | Customer A | Incoming | Supplier 1 |   | etc
Transport | Supplier 1 |   | Customer B | Incoming | Supplier 1 |   | etc

[kirstenalarsen]

@mkllnk @myriamboure - do we have a 'standard' view on whether reports pages should all load empty rather than slowly bringing up a zillion results?

and @mkllnk if you have any thoughts on performance / good building of reports based on the challenges you've seen in others feel free to put them here?!

[mkllnk]

Yes, all reports should display the search form without loading anything first.

[mkllnk]

For good performance it's important to filter as much as possible on database level and avoid queries in loops. But I think Kristina knows that very well. ;-)

It could be very helpful to create a diagram of where the data comes from. In this case it looks like we start with orders, filter them and fetch the remaining data for the report. But I haven't looked into it very much. Happy to discuss approaches with Kristina. :-)

[kirstenalarsen]

Yes I assume the information is all in orders, with all the fee information in 'adjustments'?

[sauloperez]

We all know that reports is the worst part of our codebase and we've been postponing any decisions as we didn't know what was its future.

If we're sticking with them adding more reports, we should at least do things differently. May I suggest following a slightly different architecture like the one we did in Katuma? See specially what we did with controllers and routes?

[daniellemoorhead]

Hey @sauloperez the decision was made to _not_ fix reports but to add this to what we know is a very sub-optimal reports feature that needs to be replaced, as that is what we approved during product curation.

You can read more about it here: https://community.openfoodnetwork.org/t/report-that-isolates-various-enterprise-fees/1361/17

But to highlight the bit we're talking about:

If what you're suggesting is an equivalent estimate of time then sure, let's do it. But if it gets into more complex areas and extends the time we spend on it then I say no deal 🙅

We know reports need fixing, but let's follow the process and get it written up as an icebox item and globally prioritised.

saynotoscopecreep #longconversationshappenedaboutthisalready #talkdirectlytokirstenifyouwanttoknowmore

[kirstenalarsen]

if @sauloperez and katuma have done some reports and done them better we should definitely follow their style lead if it makes sense for this report @kristinalim @mkllnk

[myriamboure]

@daniellemoorhead we said we would not redo report but do the new ones in a good way if we can, so if there is an existing solution already implemented and working I think. @sauloperez shared the code apparently so that might be useful :-) That will be one less report to redo later...

[kristinalim]

I checked the katuma_reports repository, and confirm that the changes suggested by @sauloperez were indeed not about reworking existing reports but would better organize new reports. Existing reports could still be reorganized, but no need to do that now.

(Except the part where the reports index already shows search filters for the Variants by Order report. @sauloperez Am I correct that this particular detail is not a recommendation but is only because there is only one report in the repository at the moment?)

So, from what I understand, your recommendations are:

1. Dedicate one controller for each report
2. Scope the report URL under their business domain (and probably put them under e.g. Reports::OrderManagement too). This would be in line with the concept of organizing code by domain as discussed here, but just requires scoping the URL and Ruby namespace at this point.

[sauloperez]

that's exactly it. You got it @kirstenalarsen :+1:

[myriamboure]

I guess you mean @kristinalim @sauloperez ;-)

[sauloperez]

lol yes :trollface:

[daniellemoorhead]

That is awesome news 🎉

[kirstenalarsen]

@kristinalim do you need me to add anything here (or elsewhere) to capture the slack discussion about including fees set on shipping and payment methods? or you already have that in?

[kirstenalarsen]

nb. Payment method fees have no tax and shipping method fees are taxed (or not) according to instance level configuration

[kristinalim]

Thanks, @kirstenalarsen! Adding this link to the Slack discussion too: https://openfoodnetwork.slack.com/messages/CCNR0BW1H/convo/CCNR0BW1H-1538016212.000100/

[kirstenalarsen]

so you've got it @kristinalim and don't need anything else explained to include that?

[kristinalim]

Nothing regarding that, but for authorization.

Could you confirm the following?

1. If role in an order is distributor and not coordinator, user should not see coordinator and supplier fees.
2. If role in an order is coordinator, user should see all fees except payment method and shipping fees.
3. If role in an order is supplier, user should only see supplier fees for them.

[kirstenalarsen]

ok, I would also like opinions from @sstead @myriamboure @tshumilas on this, but I reckon that in all three cases above we COULD just say they see all fees. The fees are transparent to the customer so why would we hide them here? and the report interface lets them select by enterprise if they need to, so they shouldn't be flooded with too many fees

Can we think of cases where it is important that

1. The Distributor does not see the Coordinator or Supplier fees? I think if someone is Distributor for an order they need to know exactly where the Customer's money is going?
2. Coordinator cannot see payment and shipping fees?
3. This seems more 'obvious' but again, if point of OFN is transparency, why shouldn't the Supplier see fees that have been added to the product for sale?

[tschumilas]

Interesting question indeed. I'd like to hear what others think too - but I agree with @kirstenalarsen - that generally these fees are all visible to buyers on the platform now, and that is in keeping with our transparency principle. BUT
Right now, a supplier doesn't know how much a peer supplier is getting. They can see the consumer price in the shop, and the pie chart - but none of that identifies which enterprises are getting what - ie they can't tell how much of that price goes to the other supplier. So - I'd say this applies to supplier fees too. Supplier fees get aggregated in the pie chart as usual - but in the back end, supplier A cannot see the fees set, or the price set, by supplier B.

Coordinators have to see all fees - because they are collecting the money - and they need to parcel it out to the appropriate suppliers, distributors.... So they need to know all.

I think a distributor is really a partner of the coordinator in the OC - the coordinator has already given them enterprise permissions for example. So I think its ok if they see the fees.

But - what happens when the players change? Is the visibility (or not) of these fees attached to being in the same OC, or is it attached to the enterprise permissions? Or to the login? So for example - maybe a distributor is dropped by a coordinator. When does their permission get dropped? - will seeing fees be linked to the other enterprise permissions? .

I just don't know how these permissions and roles are ended, and who ends them. (And partly I've been thinking about this because I'm theorizing that this who gives who and who ends whose various permissions and roles, is at the root of the bug on the Canadian server we are trying to solve (the can't save to inventory bug).

[kirstenalarsen]

OK, i think this is going to be very curly and we are going to need to pick something that is reasonably straight-forward and then trial and adapt it. Very keen to get a working version of this and have @kristinalim move on to mobile - we can't afford to spend weeks designing the perfect solution. @tschumilas as the 'user lead' for this issue I am going to need you to nominate where you think is the best place to start. @kristinalim also interested in your thoughts on what is simplest to deliver so that we can get it done and refine from there

Would you have a preference for e.g.
a) Everyone in the order cycle can see all fees in the order cycle
b) Coordinator can see all fees in the OC; Distributors can see Coordinator fees + all fees applied to products in their 'Outgoing' shops; Suppliers can see their fees, and any other fees applied to their products (but NOT other Supplier fees or fees applied to other Supplier's products)
c) Restricted as originally suggested by Kristina

I don't think we need to get too caught up in the permissions, as the OC and actual sale of the product supercedes them i.e. the permissions determine what can go into the order cycle and what fees can be added. Once that is done I think we map fee visibility more to what is actually sold in the OC . . unless it's much easier to build using permissions rather than the exchanges - which is a question for @kristinalim

If b) and c) are heavily / evily complex, I say we build a) and make it more complex if/when a problem actually arises . .

[tschumilas]

c is counter productive - the coordinator NEEDS to see all the fees in order to parcel out payment.

b would be a preference - but only marginally so. So I totally agree that if a is totally simple - lets go with it a. We just need to note that this is giving a supplier information that can help them figure out another supplier's prices - and we haven't done that until now. I think its in the spirt of transparency - but we might anticipate some reaction perhaps.

[kirstenalarsen]

I would imagine that use cases where suppliers are using fees to 'quietly' mark up their products are rare, because they would then be reliant on the coordinator pulling that out and paying them - which until now hasn't been possible! So if we consider that existing use of that case is very low - and most suppliers are just using their price with fees applied by distributor and coordinator, people have been able to work out supplier prices for a while now!

It might become a thing that people worry about in the future - I know people are prickly about others seeing their prices (and for good reason i.e. undercutting, which is rampant - https://www.redtomato.org/blog/) . . but that's a much bigger discussion and extremely challenging alongside our goal of transparency!

[kristinalim]

I am also worried about consequences of suppliers and distributors seeing the data of others :worried: so I don't like Option A very much, but I understand what you're saying about transparency. It is the simplest to implement.

I have structured the code in a way that would somewhat easily allow limiting entries based on permissions, so Options B and C are doable.

Regarding permissions:

1. Access is not bound to user login but to the enterprise. So if I created an OC in the past, but I'm no longer authorized for that coordinator, I would not be able to see the OC data.
2. Right now, access is given as a participant in the OC. So a newly authorized coordinator for a distributor would not be able to see data for past OCs for the distributor. Is this the desired behaviour? If not, I suspect this is something we can just change later.

[sstead]

I don't anticipate that option A will cause any issues for Australian users. We don't have much uptake of complex fees at this stage. But Theresa makes good points in scenarios where complex fees are used.

[WalkingBlackBear]

Option A wouldn't be an issue for us at all. Currently, every producer is charged, and charges, the exact same fees. So there is no issue with 'hidden' markups. Every fee is transparent to all parties.

I see where option B might be really good overall though. I think that would ultimately be the best option of the three.

[kirstenalarsen]

@kristinalim I think it's option a for now, especially if as you say it won't be that hard to retrofit option b. Let's do this thing :) thanks!

[sigmundpetersen]

Hey @kristinalim is this (and #2656, #2672) In Dev? Or are they done after 3115 was merged? Where do they belong? 🙂

[kristinalim]

@sigmundpetersen Thank you!

• #2616 (current) - I updated #3337, the last PR addressing an issue in report values, to close this.
• #2656 - Closed this. See below.
• #2672 - Closed this. See below.

We now also have a new issue #3406 for removing the feature flag for the report. In the testing instructions for this, I listed indirect concerns of #2656 and #2672 which should be tested again.