Easy-digital-downloads: 3.0 - Order item meta needs to be created for order item options

[cklosowski]

Bug Report

Expected behavior

When migrating over orders, the options array could contain information that should be migrated over as meta like license upgrades/renewals, etc.

Actual behavior

The options are simply dropped other than price_id and quantity

[cklosowski]

Just thought, we should probably make sure the purchase process also handles non-standard meta data as well.

[cklosowski]

For instance, here is what the item_number array looks like for an initial order for a recurring product:

      ["item_number"]=>
      array(3) {
        ["id"]=>
        int(1245717)
        ["quantity"]=>
        int(1)
        ["options"]=>
        array(3) {
          ["quantity"]=>
          int(1)
          ["price_id"]=>
          int(0)
          ["recurring"]=>
          array(4) {
            ["period"]=>
            string(4) "year"
            ["times"]=>
            int(0)
            ["signup_fee"]=>
            int(0)
            ["trial_period"]=>
            bool(false)
          }
        }

Here is an SL renewal, that also generated a subscription:

      'item_number' => 
      array (
        'id' => 284975,
        'quantity' => 1,
        'options' => 
        array (
          'quantity' => 1,
          'price_id' => '0',
          'is_renewal' => true,
          'license_id' => 57156,
          'license_key' => 'XXXXXXXXX',
          'recurring' => 
          array (
            'period' => 'year',
            'times' => 0,
            'signup_fee' => 0,
          ),
        ),
      ),

And here is an upgrade, that also included a recurring profile:

      'item_number' => 
      array (
        'id' => 1150319,
        'quantity' => 1,
        'options' => 
        array (
          'quantity' => 1,
          'price_id' => '',
          'is_upgrade' => true,
          'upgrade_id' => '1150319',
          'license_id' => 'XXXXXX',
          'cost' => 1,
          'recurring' => 
          array (
            'period' => 'year',
            'times' => 0,
            'signup_fee' => 0,
          ),
        ),
      ),

[cklosowski]

@easydigitaldownloads/developers

so this could use some discussion. Since we store elements like the above into the options array we need to move them to meta. I feel like we have 2 options:

Option 1:
We retain an _order_item_options meta element in the wp_edd_orderitem_meta table. This would be a serialized array of any non-standard options (price_id and quantity are standard). We could then use this to backfill the _edd_payment_meta array when it's used for backwards compatibility.

Option 2:
We convert each individual option (outside of core ones like price_id and quantity) to it's own row in wp_edd_orderitem_meta. For example, we'd have something like:
meta_name => _option_is_upgrade
meta_value => _option_upgrade_id
meta_value => _option_license_id
meta_value => _option_cost
meta_value => _option_recurring ( this would then have a meta value of a serialized array to keep the data format the same.

I'd love your thoughts on the idea here. It's likely to be an intense change either way. I feel like option 2 is the _right_ way to go, but that does pose some more issues with custom options added by 3rd party addons as we'll release update before hand to account for it in our own extensions.

[pippinsplugins]

I vote for option 2.

Serialized data that we might want to query on is brutal.

[mintplugins]

Just so I am understanding option 2 correctly, any serialized array that is currently in payment meta would be iterated out so that each array key becomes a new row?

[cklosowski]

@mintplugins, no this would _only_ apply to the options array key.

[cklosowski]

Also, if the entry is an array, I do not want to break it out, but just store it's own array so it's possible that some of this would be serialized, but only some of it would be.

An example of this is the recurring array.

[mintplugins]

We would have to make sure to update the code on any extensions expecting it as a serialized array, correct?

[JJJ]

Whichever we choose, would we want/need some kind of magic shim to identify any 'option' touch, and map it to the new location, get/set/isset, etc...

I'm finding lots of $item['item_number']['options']['price_id'] references, which worries me that add-ons or plugins are assuming the same array key safely exists, or that people might be expecting parity with those cart arrays.

[cklosowski]

@mintplugins There are two ways we could handle this.

I think both should happen.

1) We update code to look for the options instead of the array.
2) We backfill and recreate the payment meta array for at least 1 major version of EDD core. Being very vocal with developers about where this stuff will live.

We can do our part about that with the development blog post I'm working on now. Informing them that we're moving the order item options from that array, with a specific meta name format _option_<array key>, etc.

This is one of the situations where that open ended format of an array is really going to be a pain point, but one that we have to move past in order to get this data out of the serialized array.

[cklosowski]

I'm going to code this up for the migration and see if the data format seems to make sense, once it's realized and query-able.

I'll start off with a format of _option_<array key>.

[cklosowski]

First migration seemed to work fine, I'm going to try this on the EDD data set I have, and see how it holds up.

[cklosowski]

This is ready for some testing. The migration is set to move any non-standard options (price_id and quantity) over to a format of _option_<array key>. From there it is put into the order items meta table.

I've also fixed this up for the edd_build_order function, so that it to will move those options into the table.

To test I am using EDD SL and this branch.

[mintplugins]

@cklosowski Can you confirm whether these are the _minimum_ testing steps required here:

1. Create a site on EDD 2.9
2. Create variable product
3. Buy a variation of the product
4. Upgrade to EDD 3.0
5. Do the table migration
6. Confirm that the price_id is stored in the order_meta table with the key _option_price_id_0

[cklosowski]

Not quite. price_id and quantity are _not_ options anymore, they exist in the order item table.

You would need to buy a licensed product that has upgrades, and then upgrade the license. Then migrate. The options are an internal api we use for subscriptions, renewals, and upgrades.

[mintplugins]

Ok thanks. And what should I be looking for after migration?

[cklosowski]

@mintplugins If you update your EDD Dev Tools to be on the compat/3.0 branch, there is now a "View Item Meta" link in the order items actions while on the view order details page.

This will expose any of the meta that was added.

[mintplugins]

I just did the following, but I am not seeing anything that begins with _option_ in the dev tools for the order.

1. I created a brand new site on 2.9
2. I installed Software Licensing master
3. I created Product A and Product B, with Product B being an upgrade for Product A.
4. I purchased Product A.
5. I upgraded to Product B.
6. I switched EDD core to branch issue/7197 and pulled latest.
7. I rant he upgrade/migration for EDD 3.0.
8. I installed EDD dev tools on the EDD3 compat branch.
9. I viewed both of my orders.
10. Neither order shows anything that begins with _option_.

| ID | Key | Value |
| -- | -- | -- |
| 1 | payment_meta | Array ( [key] => 88c93949cefd8ce0125c76aebe867987 [email] => [email protected] [date] => 2019-05-23 13:36:41 [user_info] => Array ( [id] => 1 [first_name] => Phil [last_name] => Johnston [discount] => [email] => [email protected] ) [downloads] => Array ( [0] => Array ( [id] => 10 [quantity] => 1 [options] => Array ( [quantity] => 1 [price_id] => 0 ) ) ) [cart_details] => Array ( [0] => Array ( [name] => Product A [id] => 10 [item_number] => Array ( [id] => 10 [quantity] => 1 [options] => Array ( [quantity] => 1 [price_id] => 0 ) ) [item_price] => 5.000000000 [quantity] => 1 [discount] => 0.000000000 [subtotal] => 5.000000000 [tax] => 0.000000000 [fees] => Array ( ) [price] => 5.000000000 ) ) [fees] => Array ( ) [currency] => USD [tax] => 0.000000000 ) |
| 2 | _edd_sl_upgraded_to_payment_id | 16 |

[mintplugins]

Oh wait just found it. Does this look right?

[cklosowski]

Yep @mintplugins that's it!

[mintplugins]

_Takes a moment to understand the new itemmeta table_

[mintplugins]

This looks good I think. I'm trying to figure out if there's a real-world scenario where this (item meta) data is currently used for something logic-wise, so we can be sure that old logic continues to operate.

[cklosowski]

@mintplugins There is, and it's with SL upgrades. We use it, however I've opened an issue to update the logic in SL. I will be opening an issue for 3.0 to backfill this data into the old array format so that we don't have backward compatibility issues.

We will retain backfilling for ~1 major version (is the current thought).

I want a separate issue for backwards compat logic to make things easier to test and make it much easier to find in the issue history.

[cklosowski]

Going to merge this in, and create an issue for backwards compatibility.