Handson-ml: Chapter 2: ModuleNotFoundError: No module named 'sklearn.compose'

[chinsat11]

My python version is 3.6.2

While executing the below piece of code, I am getting ModuleNotFoundError: No module named 'sklearn.compose' error. I jalso tried importing the ColumnTransformer from future_encoders but facing the same issue, Please help.

from sklearn.compose import ColumnTransformer

num_attribs = list(housing_num)
cat_attribs = ["ocean_proximity"]

full_pipeline = ColumnTransformer([
    ("num", num_pipeline, num_attribs)
    ("cat", OneHotEncoder, cat_attribs)

])

housing_prepared = full_pipeline.fit_transform(housing)

[daniel-s-ingram]

Hi @chinsat11,

Yes, you should be importing ColumnTransformer from future_encoders.py for now. Can you post the error message you get when you try importing it from that?

[ageron]

Hi @chinsat11 ,

Thanks for your question. You should make sure the future_encoders.py file is located in the same directory as the notebook you are running, and make sure you import the ColumnTransformer like this:

from future_encoders import ColumnTransformer

Make sure to comment out the from sklearn.compose ... line.

Hope this helps,
Aurélien

ps: thanks again @daniel-s-ingram :)

[chinsat11]

Thanks Aurélien, problem is now resolved,

[jliub]

I got an error saying 'No module named 'future_encoders''

[ageron]

Hi jliub,

Importing from future_encoders was only needed before Sklearn 0.20. If you have an earlier version of Scikit-Learn, you should upgrade it:

pip3 install -U scikit-learn

If you have upgraded it, and it still doesn't work, make sure you restart the Jupyter kernel. If you have done that and it still doesn't work, then make sure you are using the latest version of the notebook.
Hope this helps,
Aurélien

[gmichaelson]

Even the code in the documentation doesn't run...

If you try to copy/paste the code here (https://scikit-learn.org/stable/auto_examples/compose/plot_column_transformer_mixed_types.html) with some slight changes:

from __future__ import print_function

import pandas as pd
import numpy as np

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import Imputer
from sklearn.preprocessing import StandardScaler
from sklearn.preprocessing import OneHotEncoder
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split, GridSearchCV

np.random.seed(0)

# Read data from Titanic dataset.
titanic_url = ('https://raw.githubusercontent.com/amueller/'
               'scipy-2017-sklearn/091d371/notebooks/datasets/titanic3.csv')
data = pd.read_csv(titanic_url)

# We will train our classifier with the following features:
# Numeric Features:
# - age: float.
# - fare: float.
# Categorical Features:
# - embarked: categories encoded as strings {'C', 'S', 'Q'}.
# - sex: categories encoded as strings {'female', 'male'}.
# - pclass: ordinal integers {1, 2, 3}.

# We create the preprocessing pipelines for both numeric and categorical data.
numeric_features = ['age', 'fare']
numeric_transformer = Pipeline(steps=[
    ('imputer', Imputer(strategy='median')),
    ('scaler', StandardScaler())])

categorical_features = ['embarked', 'sex', 'pclass']
categorical_transformer = Pipeline(steps=[('onehot', OneHotEncoder(handle_unknown='ignore'))])

preprocessor = ColumnTransformer(
    transformers=[
        ('num', numeric_transformer, numeric_features),
        ('cat', categorical_transformer, categorical_features)])

# Append classifier to preprocessing pipeline.
# Now we have a full prediction pipeline.
clf = Pipeline(steps=[('preprocessor', preprocessor),
                      ('rf', RandomForestClassifier())])

X = data.drop('survived', axis=1)
y = data['survived']

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

clf.fit(X_train, y_train)
print("model score: %.3f" % clf.score(X_test, y_test))

You end up with:

ValueError                                Traceback (most recent call last)
<ipython-input-39-0a6e58318d2d> in <module>()
     51 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
     52 
---> 53 clf.fit(X_train, y_train)
     54 print("model score: %.3f" % clf.score(X_test, y_test))

/anaconda3/lib/python3.7/site-packages/sklearn/pipeline.py in fit(self, X, y, **fit_params)
    263             This estimator
    264         """
--> 265         Xt, fit_params = self._fit(X, y, **fit_params)
    266         if self._final_estimator is not None:
    267             self._final_estimator.fit(Xt, y, **fit_params)

/anaconda3/lib/python3.7/site-packages/sklearn/pipeline.py in _fit(self, X, y, **fit_params)
    228                 Xt, fitted_transformer = fit_transform_one_cached(
    229                     cloned_transformer, Xt, y, None,
--> 230                     **fit_params_steps[name])
    231                 # Replace the transformer of the step with the fitted
    232                 # transformer. This is necessary when loading the transformer

/anaconda3/lib/python3.7/site-packages/sklearn/externals/joblib/memory.py in __call__(self, *args, **kwargs)
    360     each time it is called.
    361 
--> 362     Methods are provided to inspect the cache or clean it.
    363 
    364     Attributes

/anaconda3/lib/python3.7/site-packages/sklearn/pipeline.py in _fit_transform_one(transformer, X, y, weight, **fit_params)
    612 def _fit_transform_one(transformer, X, y, weight, **fit_params):
    613     if hasattr(transformer, 'fit_transform'):
--> 614         res = transformer.fit_transform(X, y, **fit_params)
    615     else:
    616         res = transformer.fit(X, y, **fit_params).transform(X)

/anaconda3/lib/python3.7/site-packages/sklearn/compose/_column_transformer.py in fit_transform(self, X, y)
    447         self._validate_remainder(X)
    448 
--> 449         result = self._fit_transform(X, y, _fit_transform_one)
    450 
    451         if not result:

/anaconda3/lib/python3.7/site-packages/sklearn/compose/_column_transformer.py in _fit_transform(self, X, y, func, fitted)
    391                               _get_column(X, column), y, weight)
    392                 for _, trans, column, weight in self._iter(
--> 393                     fitted=fitted, replace_strings=True))
    394         except ValueError as e:
    395             if "Expected 2D array, got 1D array instead" in str(e):

/anaconda3/lib/python3.7/site-packages/sklearn/externals/joblib/parallel.py in __call__(self, iterable)
    777            of time, controlled by self.verbose.
    778         """
--> 779         if not self.verbose:
    780             return
    781         elapsed_time = time.time() - self._start_time

/anaconda3/lib/python3.7/site-packages/sklearn/externals/joblib/parallel.py in dispatch_one_batch(self, iterator)
    623             pass
    624         elif hasattr(backend, 'Pool') and hasattr(backend, 'Lock'):
--> 625             # Make it possible to pass a custom multiprocessing context as
    626             # backend to change the start method to forkserver or spawn or
    627             # preload modules on the forkserver helper process.

/anaconda3/lib/python3.7/site-packages/sklearn/externals/joblib/parallel.py in _dispatch(self, batch)
    586                  temp_folder=None, max_nbytes='1M', mmap_mode='r',
    587                  prefer=None, require=None):
--> 588         active_backend, context_n_jobs = get_active_backend(
    589             prefer=prefer, require=require, verbose=verbose)
    590         if backend is None and n_jobs is None:

/anaconda3/lib/python3.7/site-packages/sklearn/externals/joblib/_parallel_backends.py in apply_async(self, func, callback)
    109         are expected to be submitted to this backend.
    110 
--> 111         Setting ensure_ready to False is an optimization that can be leveraged
    112         when aborting tasks via killing processes from a local process pool
    113         managed by the backend it-self: if we expect no new tasks, there is no

/anaconda3/lib/python3.7/site-packages/sklearn/externals/joblib/_parallel_backends.py in __init__(self, batch)
    330 class ThreadingBackend(PoolManagerMixin, ParallelBackendBase):
    331     """A ParallelBackend which will use a thread pool to execute batches in.
--> 332 
    333     This is a low-overhead backend but it suffers from the Python Global
    334     Interpreter Lock if the called function relies a lot on Python objects.

/anaconda3/lib/python3.7/site-packages/sklearn/externals/joblib/parallel.py in __call__(self)
    129 
    130     - 'loky': single-host, process-based parallelism (used by default),
--> 131     - 'threading': single-host, thread-based parallelism,
    132     - 'multiprocessing': legacy single-host, process-based parallelism.
    133 

/anaconda3/lib/python3.7/site-packages/sklearn/externals/joblib/parallel.py in <listcomp>(.0)
    129 
    130     - 'loky': single-host, process-based parallelism (used by default),
--> 131     - 'threading': single-host, thread-based parallelism,
    132     - 'multiprocessing': legacy single-host, process-based parallelism.
    133 

/anaconda3/lib/python3.7/site-packages/sklearn/pipeline.py in _fit_transform_one(transformer, X, y, weight, **fit_params)
    612 def _fit_transform_one(transformer, X, y, weight, **fit_params):
    613     if hasattr(transformer, 'fit_transform'):
--> 614         res = transformer.fit_transform(X, y, **fit_params)
    615     else:
    616         res = transformer.fit(X, y, **fit_params).transform(X)

/anaconda3/lib/python3.7/site-packages/sklearn/pipeline.py in fit_transform(self, X, y, **fit_params)
    298         Xt, fit_params = self._fit(X, y, **fit_params)
    299         if hasattr(last_step, 'fit_transform'):
--> 300             return last_step.fit_transform(Xt, y, **fit_params)
    301         elif last_step is None:
    302             return Xt

/anaconda3/lib/python3.7/site-packages/sklearn/preprocessing/data.py in fit_transform(self, X, y)
   2017 
   2018     Read more in the :ref:`User Guide <preprocessing_transformer>`.
-> 2019 
   2020     Parameters
   2021     ----------

/anaconda3/lib/python3.7/site-packages/sklearn/preprocessing/data.py in _transform_selected(X, transform, selected, copy)
   1807 
   1808     def __init__(self, threshold=0.0, copy=True):
-> 1809         self.threshold = threshold
   1810         self.copy = copy
   1811 

/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)
    431 
    432     ensure_min_samples : int (default=1)
--> 433         Make sure that the array has a minimum number of samples in its first
    434         axis (rows for a 2D array). Setting to 0 disables this check.
    435 

ValueError: could not convert string to float: 'female'

Seems like something is definitely wonked...

[gmichaelson]

Sorry... always got to remember to upgrade with conda... not pip. Sorry. This is my error.

[suganyaplds]

Hi jliub,

Importing from future_encoders was only needed before Sklearn 0.20. If you have an earlier version of Scikit-Learn, you should upgrade it:

pip3 install -U scikit-learn

If you have upgraded it, and it still doesn't work, make sure you restart the Jupyter kernel. If you have done that and it still doesn't work, then make sure you are using the latest version of the notebook.
Hope this helps,
Aurélien

After upgrading scikit-learn, restarting the kernel etc., still I'm getting the below error...
'No module named 'future_encoders'
Any help ?

[ageron]

Hi @suganyaplds ,

Thanks for your feedback. That's really weird. Could you please run this code to double check your Scikit-Learn version:

import sklearn
print(sklearn.__version__)

If the version is 0.19 or earlier, then you still need to upgrade Scikit-Learn to 0.20 or above. If it's already upgraded, then make sure you use these imports instead of importing from future_encoders:

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OrdinalEncoder
from sklearn.preprocessing import OneHotEncoder