Handson-ml: Chapter 2 :Transformation Pipelines - ValueError: could not convert string to float: 'NEAR BAY'

[Tikam02]

ValueError: could not convert string to float: 'NEAR BAY'

[ageron]

Hi @Tikam02 , thanks for your feedback.

I am guessing the cat_pipeline is missing the one-hot encoder? Or perhaps you are using Scikit-Learn 0.20 dev version? Could you please confirm that you are using the latest version of the notebook (do a git pull)?

Oh, this does not seem to the notebook from chapter 2 (the name should be 02_end_to_end_machine_learning_project.ipynb). Could you please check that the notebook 02_end_to_end_machine_learning_project.ipynb works fine? If so, then please check where the notebooks differ.
Hope this helps.

[uragirii]

Having same problem, I checked my sklearn version is 0.19.1

[ageron]

Hi @NotCherub ,
Thanks for your feedback. This is odd... I just tried, using 0.19.1 as well, with the latest notebooks on github.com/ageron/handson-ml, and everything ran smoothly. Are you running the latest version of the notebook from this repo? I'm not sure why the name of your notebook is not 02_end_to_end_machine_learning_project?

Could you please confirm that your pipeline is built like this:

num_attribs = list(housing_num)
cat_attribs = ["ocean_proximity"]

num_pipeline = Pipeline([
        ('selector', DataFrameSelector(num_attribs)),
        ('imputer', Imputer(strategy="median")),
        ('attribs_adder', CombinedAttributesAdder()),
        ('std_scaler', StandardScaler()),
    ])

cat_pipeline = Pipeline([
        ('selector', DataFrameSelector(cat_attribs)),
        ('cat_encoder', OneHotEncoder(sparse=False)),
    ])

from sklearn.pipeline import FeatureUnion

full_pipeline = FeatureUnion(transformer_list=[
        ("num_pipeline", num_pipeline),
        ("cat_pipeline", cat_pipeline),
    ])

housing_prepared = full_pipeline.fit_transform(housing)
housing_prepared

[ageron]

Ohhh, I may have an idea: perhaps you are using the OneHotEncoder class provided by Scikit-Learn 0.19.1 rather than the one provided in the future_encoders.py module?

If you have Scikit-Learn < 0.20, then make sure you have the file future_encoders.py (available in this project) and you are importing from it like this:

from future_encoders import OneHotEncoder # requires future_encoders.py

If you have Scikit-Learn ≥ 0.20 (not yet released at the time of this writing), then you can use the OneHotEncoder class it provides:

from sklearn.preprocessing import OneHotEncoder # ONLY FOR SCIKIT-LEARN >= 0.20

[uragirii]

Yes, i was using sklearn.preprocessing instead of future_encoders, as couldn't import the file

[ageron]

I see, so that's definitely the problem. The solution is to download future_encoders.py and place it in the project root directory (and once Scikit-Learn 0.20 is released, you can get rid of it and use sklearn.preprocessing.OneHotEncoder instead).

For the record, the notebook does have this warning... :)

Warning: earlier versions of the book used the LabelBinarizer or CategoricalEncoder classes to convert each categorical value to a one-hot vector. It is now preferable to use the OneHotEncoder class. Right now it can only handle integer categorical inputs, but in Scikit-Learn 0.20 it will also handle string categorical inputs (see PR #10521). So for now we import it from future_encoders.py, but when Scikit-Learn 0.20 is released, you can import it from sklearn.preprocessing instead:

[cmcgrath1982]

So I this same problem after I tried dealing with the error CategoricalEncoder was causing. So I'm using Scikit-Learn 0.20 because its where i ended up chasing the first error, now I'm having the 'Near Bay' issue. I'm not great with system mapping so can I get a hand here. How do I download and install future_encoders.py? I copied the code and saved it as a .py file using IDLE, how do i find where to save it? After that, do I do a pip install or will it just run as a import future_encoders line?

[ageron]

Hi @cmcgrath1982 ,
Thanks for your question.
I haven't tested the notebooks with Scikit-Learn 0.20 yet, because it's not released yet (you installed the dev version, I suppose). It would be best if you used the stable version, i.e., Scikit-Learn 0.19.1 for now, if you known how to downgrade (pip install sklearn==0.19.1). If not, it may be okay, but I can't guarantee it.
Regarding the future_encoders.py file, you should place it in the same directory as the notebooks. Next time you want to download a file from github, you can first click on that file to visualize it in github, then right-click on the "Raw" button located just above the code, and select Save Link As... (if you are using Chrome, or else this option may be named slightly differently), then select the directory you want to save the file in (in this case, you want the same directory as the notebooks).
Hope this helps.

[kalnist]

Hi @ageron : I have the same issue described by @cmcgrath1982. I did put the file future_encoders.py in the same directory as the .ipynb notebook and I also tried installing sklearn 0.19.1 using conda and got the following error.

PackagesNotFoundError: The following packages are not available from current channels:

• sklearn==0.19.1

[kalnist]

ValueError: could not convert string to float: 'NEAR BAY'
pls help.

[rohith28]

@Tikam02 I had similar issue. I think below code will help you.

class CustomLabelBinarizer(BaseEstimator, TransformerMixin):
    def __init__(self, sparse_output=False):
        self.sparse_output = sparse_output
    def fit(self, X, y=None):
        return self
    def transform(self, X, y=None):
        enc = LabelBinarizer(sparse_output=self.sparse_output)
        return enc.fit_transform(X)

from sklearn.base import BaseEstimator, TransformerMixin
class DataFrameSelector(BaseEstimator, TransformerMixin):
    def __init__(self, attribute_names):
        self.attribute_names = attribute_names
    def fit(self, X, y=None):
        return self
    def transform(self, X):
        print(X[self.attribute_names].values)
        return X[self.attribute_names].values

num_attribs = list(housing_num)
cat_attribs = ["ocean_proximity"]

num_pipeline = Pipeline([
        ('selector', DataFrameSelector(num_attribs)),
        ('imputer', Imputer(strategy="median")),
        ('attribs_adder', CombineAttributesAdder()),
        ('std_scaler', StandardScaler()),
    ])

cat_pipeline = Pipeline([
        ('selector', DataFrameSelector(cat_attribs)),
        ('label_binarizer',CustomLabelBinarizer()),
    ])

from sklearn.pipeline import FeatureUnion
full_pipeline = FeatureUnion(transformer_list=[
        ("num_pipeline", num_pipeline),
        ("cat_pipeline", cat_pipeline),
    ])
housing_prepared = full_pipeline.fit_transform(housing)


housing_prepared.shape 
(16512,16)

Let me know if it is not helpful

[kalnist]

Thanks @rohith28. For me I tried all the steps below from @ageron but in an conda env and downgraded to the 0.19.1 and it worked.

" haven't tested the notebooks with Scikit-Learn 0.20 yet, because it's not released yet (you installed the dev version, I suppose). It would be best if you used the stable version, i.e., Scikit-Learn 0.19.1 for now, if you known how to downgrade (pip install sklearn==0.19.1). If not, it may be okay, but I can't guarantee it."