koalas appears to ignore None values when converting the type of a DataFrame or Series using .astype() instead of converting them which is the pandas behaviour. For instance when converting to str, in pandas None becomes 'None' but this does not happen in koalas. Example below shown by using sorted() which can't handle None:
import databricks.koalas as ks
import pandas as pd
data = pd.Series(['a', 'b', 'c', None])
sorted(data.astype(str).tolist())
# => ['None', 'a', 'b', 'c']
data = ks.Series(['a', 'b', 'c', None])
sorted(data.astype(str).tolist())
# ---------------------------------------------------------------------------
# TypeError Traceback (most recent call last)
# <ipython-input-11-493f99e0fb6f> in <module>
# 1 data = ks.Series(['a', 'b', 'c', None])
# ----> 2 sorted(data.astype(str).tolist())
#
# TypeError: '<' not supported between instances of 'NoneType' and 'str'
Is there a reason for this or can we bring this in line with pandas? Thanks
Ubuntu 18.04
python 3.7.6
koalas==1.2.0
pandas==1.0.5
@itholic can you take a look when you find some time?
Sure, let me take a look.
Thanks for the report, @amin-nejad !