Machinelearning: Will there be ML.NET support for ARM applications?

[tauheedul]

Build 0.7 introduced support for x86 applications.
Will ARM support also be included?
Examples of usages are Always Connected Windows 10/Linux laptops running ARM processors

.NET Core supports ARM so the route to enable it should be possible

[TomFinley]

Hi @tauheedul , the plan last I heard was that we would offer ARM support (we need this for other reasons) but not as part of the targeted scenarios for v1.0. So eventually, but because we have something of a deadline I think it was scoped out. @asthana86 or @shauheen would know better than I about these plans though, and would have a more up to date understanding I think, than just a poor lowly dev like me. 😄

There was one thing I was curious about though, there are two things we might want to use ML.NET for -- training machine learning models, and predicting using those already trained models. Would it suffice for whatever you had in mind if predicting worked on ARM, but training had to happen on an actual x86/x64? Or would training on ARM also be a scenario?

The reason I like to know these things is that, while we have native code dependencies for training (which is what makes running on ARM hard) since those are operations that heavily benefit from native code, predicting benefits less in most situations, and involves far less complex operations and mathematics. So doing prediction/scoring only would be far easier to bring to ARM somewhat quickly. But of course I'd like to understand whether that would even be beneficial to people.

[CESARDELATORRE]

@TomFinley - I'd support the second choice as the first step. For most scenarios in IoT and Mobile devices (based on ARM) just scoring/running a model is mostly needed, not needed to train a model on those small devices, usually.

@tauheedul - What's your feedback about it?

[tauheedul]

Hi @TomFinley @CESARDELATORRE . Thank you for the background information it is insightful.

I understand ARM processors don't have the kind of performance we get standard x64 processors, what makes it ideal is they can generally last a lot longer in terms of battery life, so could deliver similar results (although a little slower)

If there are technical constraints that don't align with the planned road map, may I suggest the following...

At the very minimum ML.NET running on ARM should be able to perform Prediction operations as this is a quick win from an end user perspective. The user will not be concerned where the model was trained. It would show case ML.NET capabilities and attract further interest from the development community.

In the interim developers should be able train their models externally on the cloud or on a standard x86/x64 computer and target the output in their ML.NET ARM supported application

In the future Training capabilities would be advantageous because modern ARM processors are making considerable gains and there are processors being developed that are optimised for AI tasks and are designed for Laptops (Windows 10 on ARM devices: https://www.techradar.com/amp/news/windows-10-on-arm-laptops-set-to-be-more-powerful-thanks-to-eight-core-snapdragon-8180).

[shmoradims]

@CESARDELATORRE @GalOshri @shauheen could you please discuss if we want to put "Scoring on ARM" on the roadmap?

[CESARDELATORRE]

I think it is very important especially/initially for scoring a model on ARM devices.

Just for scoring might be a smaller effort from our side that’s trying to have support for ARM for the whole ML.NET.

We need ARM for scoring models on IoT devices and for mobile devices running Xamarin apps on iOS or Android. All those workloads are important for .NET.

But training on ARM might not be so important, for now..

Cesar

---

From: Shahab Moradi [e-mail]
Sent: Thursday, November 29, 2018 11:25 AM
To: dotnet/machinelearning
Cc: Cesar De la Torre Llorente; Mention
Subject: Re: [dotnet/machinelearning] Will there be ML.NET support for ARM applications? (#1662)

@CESARDELATORREhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCESARDELATORRE&data=02%7C01%7Ccesardl%40microsoft.com%7Cd128a113a2ef48d4f80608d656306436%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636791163172531812&sdata=lE%2Bx8fbvai7%2FMauRIYLVDPkJBTw0StHnV32OPArVlN0%3D&reserved=0 @GalOshrihttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FGalOshri&data=02%7C01%7Ccesardl%40microsoft.com%7Cd128a113a2ef48d4f80608d656306436%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636791163172541817&sdata=jFZfHHpJDlq8QsfBjjFpcGbKAQ0N3MYjMuwHh0Elz4E%3D&reserved=0 @shauheenhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fshauheen&data=02%7C01%7Ccesardl%40microsoft.com%7Cd128a113a2ef48d4f80608d656306436%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636791163172541817&sdata=GCe6WTo1kFwugYMCJNRO6GNF%2FEDDkyYMiSAltYBTAd8%3D&reserved=0 could you please discuss if we want to put "Scoring on ARM" on the roadmap?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdotnet%2Fmachinelearning%2Fissues%2F1662%23issuecomment-442959653&data=02%7C01%7Ccesardl%40microsoft.com%7Cd128a113a2ef48d4f80608d656306436%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636791163172551822&sdata=Fe1ThVYGN1m4A5EKxvSnN8XUbq3F9A%2BzKU3%2Br7uiZNs%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABoh-9DpJviW9xbmlHkMHYx6qNNPscbHks5u0DSbgaJpZM4YnZ2R&data=02%7C01%7Ccesardl%40microsoft.com%7Cd128a113a2ef48d4f80608d656306436%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636791163172551822&sdata=sl3ffy9HR8g99myi9DruKmQANalPEU1Afe5M9vRsfhE%3D&reserved=0.

[shmoradims]

Added concrete work item #1790.

[rdzidziguri]

After attending Build 2019 online I can understand this is planned staff in ML.NET so just wait for guys! eventually, MS will end up supporting ml.net on ARM and hope no less than ARM 7