Machinelearning: NA Handling

[satish860]

System information

• OS version/distro: Windows
• .NET Version (eg., dotnet --info): 2.1.1

Issue

• What did you do?
  I was trying to work on Titanic kaggle problem and trying to train and I am looking for an example of how to handle Null values.
  I have seen handling option but not able to find how to use it in Pipeline.

Between one difference I have seen between Python interfaces and .net is the ability to experiment easily and I am finding it bit harder to do here. If you have any practices it will be helpful if we can discuss here.

[jwood803]

I may be mistaken, but I _think_ you should be able to use the NADrop and/or the NAReplace transforms.

https://github.com/dotnet/machinelearning/blob/c023727b76970ab913ec1ce38276508835c17bcf/src/Microsoft.ML.Transforms/NADropTransform.cs#L27

https://github.com/dotnet/machinelearning/blob/52cc8740aa33ce59dd9bf84922bb57a3816dc234/src/Microsoft.ML.Transforms/NAReplaceTransform.cs#L37

[satish860]

Unfortunately, both of them has not implemented the ILearningPipeLineItem interface. So I was not able to use the same.

[zeahmed]

The name is different in C# API. it is MissingValueHandler. We have a work item #371 where we are going to remove higher level abstraction and use the components directly as mentioned by @jwood803. But for now MissingValueHandler will do your job.

Python and C# are totally two different environments. Since you are accustom to Python, you feel bit uneasy in C#/ML.Net. Once you get familiar with ML.Net environment I believe you will find it more convenient than Python.

Let me know where you are facing difficulty I will be able to help...:)

[CESARDELATORRE]

Additionally, for the ability to experiment easily we're researching and developing additional approaches/UIs for experiments, data transforms and trainings, so it'll be a lot easier than just by using the APIs.
That will be coming in future releases, we still cannot comment any further until we make further announcements in previews.
Our goal is to have "the ability to experiment easily" as best in class. Stay tuned! 👍

[sfilipi]

This will be called complete after adding a sample for MissingValueHandling/Indicator/Replacing transforms.