Evalml: Integrate Woodwork DataTables into EvalML

[angela97lin]

This Epic tracks all issues related to integrating Woodwork DataTables into EvalML.

Notes here: https://alteryx.quip.com/XedaAs9HXc9f/EvalML-Woodwork-Table-Integration
Design doc here: https://alteryx.quip.com/Oa4aA6h0Rf2k/Woodwork-DataTable-Integration-Design-Document

Implementation Plan:

• Update AutoML: 1 week
  
  
  
  • Update AutoMLSearch to support WW DataTables. Maintain support for pandas input.
  
  
  • AutoML will still pass pandas DFs to the pipelines/components
  
  

• Add documentation about what types we expect categorical, numeric and text to be set to in order for automl to treat features correctly, as well warn users what may happen if users pass in pandas DataFrames / numpy arrays. 2 days

• Update pipelines and all components to accept WW DataTables: 1 week

  • Can (and should) be done in groups
• Update AutoML to pass datatables to every pipeline/component instead of pandas DF. 1 day
• Update components from selecting using pandas dtypes to selecting using DataTable logical types instead. 3 days
• Update documentation to use DataTables in all examples. 2 days
• Update all graph methods, util methods which accept data, to support DataTables. 1 week

Total: 3 weeks 3 days, then 1 more week to do graphing/utils.

Key Dates
Oct release is Tues October 27th.

Goal
Complete all by Fri Nov 6th (4 weeks). Ship in Nov release.

Stretch Goal
Everything except graphing/utils done by the October release (Oct 27th).

[dsherry]

@angela97lin : I think we should drop new datatable types which we won't support initially. Then we can add support for those on a case-by-case basis. So:

dt = ... # woodwork datatable as input to automl search, for example
numeric_features = dt.select('numeric')
categorical_features = dt.select('categorical')
natural_language_features = dt.select('natural_language')
# then, in order to drop unsupported types, from here on out, don't use
# the other features which may still be held in the original datatable

A notable example: the fraud dataset uses lat/long. Currently, it passes that in as two independent floats. Short-term, we should just drop. I suppose we could encode to two independent floats, but I doubt that'll perform well. Long-term, we'll want to use featuretools' latlong-compatible primitives to build features for that type.

That make sense?

[dsherry]

@freddyaboulton brought up a good point today: because we ship evalml on conda, we need woodwork to be on conda before we can expect users to have it installed. That leaves us with the following options for how to proceed:

1. Find out when woodwork will be added to conda (and/or help expedite that). Hold any woodwork merges until then. Because we need that done in order to release code which depends on woodwork.
2. In the short-term, make woodwork an "optional" dependency, meaning if the import fails we simply don't support woodwork tables as an arg to automl search. In the long-term, it feels klunky to do this for all pipeline/component fit/predict methods, especially if we're gonna add woodwork to conda anyways, so we should avoid this.
3. Merge woodwork support. Any conda install of evalml will have import errors. Live with this until woodwork is on conda.
4. Merge woodwork support. Don't release to conda until woodwork is on conda.

I'm a fan of option 1, i.e. getting woodwork onto conda, because I believe that's what we'll want to do long-term.

I see there's already an issue filed for adding woodwork to conda. I'll follow up.

[angela97lin]

@dsherry Thanks for this! @gsheni mentioned here that they'll likely have an update near the end of the week. Should we hold off on merging for the October release then?

[dsherry]

@angela97lin yep, given that the release is in a few days, and that you'll be out for a bit as well, it shouldn't be too much trouble to wait to merge until afterwards.

[dsherry]

Was just chatting with @angela97lin and @freddyaboulton about the woodwork upgrade. Here's what we listed as currently outstanding / in-flight:

• Update the data checks (in progress)
• Update all components to use woodwork types where applicable (#1290)
• Update model understanding methods
• Return woodwork instead of pandas from pipeline/component transform/predict, and anywhere we return a copy or extension of the user-inputted data.
• Update our old dtype lists and all usages to use woodwork types

@angela97lin did I miss anything?

[angela97lin]

@dsherry That looks about right!

• Update documentation to use Woodwork (in progress, #1466)
• Update the data checks (in progress, #1481)
• Update model understanding methods + graphing not covered in documentation
• Return woodwork instead of pandas from pipeline/component transform/predict, and anywhere we return a copy or extension of the user-inputted data (#1406)
• Update our old dtype lists and all usages to use woodwork types #1290

[dsherry]

@angela97lin awesome, thanks!

[angela97lin]

@chukarsten @dsherry All of the issues associated with this Epic have been closed! Is it safe to close this Epic and just track and WW issues that come up in isolation? :)