Handson-ml: Chapter 2: use of Ordinal Encoder for ordinal attributes

[GMalaguti]

@ageron extremely useful book to get into machine learning problems!

After having implemented Chapter 2 following your notebook, I wanted to apply what I learned on a different dataset (Kaggle's House Prices competition).

I want to apply the Ordinal Encoder to a set of attributes, but now giving the mappings between categories and numerical values (e.g. 'good', 'average', 'bad' mapped into 2, 1, 0), but I don't understand how to do that from this page:
https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OrdinalEncoder.html
Even looking at the User Guide did not help...

I have also found this other page:
http://contrib.scikit-learn.org/categorical-encoding/ordinal.html
where it becomes clear to me how to pass the mapping to the encoder, but I think this is not the same encoder as the previous link, since it is in a different class. I am a bit confused...

Should I use OneHOt Encoder even if I can interpret the categories as ordered?

[ageron]

Hi @GMalaguti ,
Thanks for your kind words! :)
Indeed, if the categories have a natural order (which is the case here as good > average > bad), then you should generally use an ordinal encoding.

You can use it like this:

>>> from sklearn.preprocessing import OrdinalEncoder
>>> import numpy as np
>>> X = np.array([
...     ["good", "london"],
...     ["good", "tokyo"],
...     ["bad", "paris"],
...     ["average", "so so"],
...     ["good", "tokyo"]])
...
>>> encoder = OrdinalEncoder()
>>> encoder.fit_transform(X)
array([[2., 0.],
       [2., 3.],
       [1., 1.],
       [0., 2.],
       [2., 3.]])
>>> encoder.categories_
[array(['average', 'bad', 'good'], dtype='<U7'),
 array(['london', 'paris', 'so so', 'tokyo'], dtype='<U7')]

Notice that:

• the columns get converted independently: there is one set of categories for happiness, and another for city.
• the categories are automatically sorted by alphabetical order. This is fine for the cities, since we don't care about the order. However, it's a problem for the happiness column. :( I have tried forcing another order by setting the categories argument when constructing the OrdinalEncoder, but it does not work: it requires it to be sorted. However, the error message says numerical features should be sorted, so I think this is a bug. It should be possible to pass text categories in arbitrary order. You can work around this issue by changing the labels so that they have a prefix like "0 bad", "1 average", "2 good", but this defeats the purpose of the OrdinalEncoder, which is supposed to make your life easier.
• the encoder works on the full array you give it, so you should probably use it in conjunction with the ColumnTransformer (see the notebook for chapter 2) so that you apply it only to the columns you want to encode.

Hope this helps,
Aurélien

[ageron]

It seems that the bug is only for NumPy arrays. This works:

>>> encoder = OrdinalEncoder(categories=[["bad", "average", "good"]])
>>> encoder.fit_transform([["good"], ["bad"], ["bad"], ["average"]])
array([[2.],
       [0.],
       [0.],
       [1.]])

But this does not:

```pycon

encoder.fit_transform(np.array([["good"], ["bad"], ["bad"], ["average"]]))
...
ValueError: Unsorted categories are not supported for numerical categories

[GMalaguti]

Hi @ageron ,
thank you for the hints and the fast reply!

I tried your example without a NumPy array, but also this does not work for me, maybe for my Scikit version (0.19):

>>> encoder = OrdinalEncoder(categories=[["bad", "average", "good"]])
>>> encoder.fit_transform([["good"], ["bad"], ["bad"], ["average"]])
...
ValueError: Unsorted categories are not yet supported

I also found out that there is the replace function. Would that be an alternative option? I can pass the mapping and the attributes involved. The most tricky part should be to create a transformer to include that into the pipeline.

[ageron]

Hi @GMalaguti ,
No problem! :)
I highly recommend you upgrade to Scikit-Learn 0.20. It introduces the ColumnTransformer which greatly simplifies pipelines when different features need different proprocessing operations, and it improves encoding categories (and of course, it adds much more).
Cheers,
Aurélien

[GMalaguti]

Hi @ageron ,
with the new version of Scikit-Learn it works properly!
I am still struggling to include the OrdinalEncoder in conjunction with ColumnTrnasformer into my pipeline... but this is another problem.
Thanks for the hints!

[siddley1001]

I was wondering if it was possible to provide a list of dictionaries to establish orders for specific columns. I was thinking this would be useful in a column transformer pipeline