Cntk: .NET Support

[StevenGann]

Issue #817 was closed with this response:

We fell a little behind. The Python bindings are done in SWIG. I think that can be quickly repurposed into .Net bindings, once done. So it shouldn't be too far out.

That was September 6th. It's now late October and there's been neither an announcement regarding .NET, nor an update to the original issue.

I'm eager to finally work with deep learning on Windows with C# instead of on Linux with Python. I'm sure Python bindings were helpful to some devs, but frankly every other deep learning framework already supports Python and C++. I'm surprised .NET hasn't been a higher priority, since it's something no other system has been able to provide. Now's the chance to become the de facto deep learning library for OOP development!

[wolfma61]

Yes Steven, we hear you and we are sorry for the delay.
This is now one of the highest priority items for us, no definite 'landing day yet', but we should be able to give you at least a date shortly, and you will some work on this topic materializing in branches ...

thx
Wolfgang

[StevenGann]

That's fantastic, and thank you for the quick reply. I'm happy to wait, I just wanted some signal that it was still in progress.

[andrewfry]

Great news! I'm very excited for this.

On Oct 27, 2016 9:35 AM, "Steven Gann" [email protected] wrote:

That's fantastic, and thank you for the quick reply. I'm happy to wait, I
just wanted some signal that it was still in progress.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/CNTK/issues/960#issuecomment-256642116, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHgBXmpKyUsZKcuUE0HFMcicK-AL3UZQks5q4KicgaJpZM4KiTg-
.

[ghost]

+1 on C/C++ and C#

[gvoysey]

Will support target .NET, .NET Core, or Mono? I am eager to do cross-platform learning in C#.

[ghost]

python/R is great for research and prototyping - the general "lab settings". this is what alphabet is doing. CNTK extends further as platform and has to support the footprint of why it is being chosen in the first place.

small to medium scale commercial projects would benefit from the complete integrated API. For many, C# replaces VB and likes. this is different ML/AI sector - from phone apps to websites and small corporate projects.

C# leverages over 15 years of the entire DCOM/J++/COM+/DNA2000 etc effort around integrated windows platform development and is used from distributed applications development, transaction processing to desktop and phone applications.

[StevenGann]

@zergxyz I'd like to see some numbers showing that more commercial production code uses R than .NET. I'd like to see some data showing more end-user applications and games using Python than .NET.

Can R even be compiled? I know it's basically an open source Matlab but to my knowledge it's still a scripting language. Python can technically be compiled, but compatibility and platform support is poor. Heaven help you if you need something more exotic than the stock Python 2.7 environment.

Academics may favor scripting languages like R and Python, but actual applications tend to be written in actual programming languages. Furthermore, the indie game community is largely dominated by C#, at least more so than Python or R, which means

the main stream in machine learning and deep learning community should be c++/python/R based

Am I addressing the Godking of Deep Learning? Is there an Asimov-style governing body determining what research can be done and how?

You have Python support. I'm sorry if you think R support is more important than C# but the numbers don't lie. C# is more popular than R by every metric. Besides, a quick Google search suggests R already has it's own RNN system, plus it already has support from Tensorflow. Go try Tensorflow instead.

[bklooste]

As these products become more mature they will have to use common languages/platforms , im sick of converting flakey lab code to something that customers can actually use. Large c# programs are everywhere outside of linux (which is used by only a small amount of desktop users).

[n17s]

We have started actual work on C# bindings.
A few questions to the folks that watch this issue:

• for Python we relied heavily on NumPy as the way to pass tensors in and out. What is the most natural way of doing this in C# ? Especially if you need some simple preprocessing operations like transposing a 640x480x3 tensor to 3x480x640 ?
• the issue says ".net" but everyone here has not mentioned anything other than C#. Is it safe to assume that C# should be the next language to cover?

[ghost]

The best would be to clean-up and merge what had been developed for BrainScript with one of the OS libraries or branch it into separate .net development.

Math.NET Numerics : http://numerics.mathdotnet.com/ (http://www.mathdotnet.com/) could be a good starting point. It also supports using native library ie. MKL etc. There had been some effort within MSFT to use/expose it as well https://msdn.microsoft.com/en-us/library/hh304363(v=vs.100).aspx

There are some smaller tensor library specific porting projects like https://libraries.io/github/alex-weaver/TensorSharp .

Not sure I understand your second question. .NET is a framework and leaves the choice of language free. What is the CNTK strategy over the long term ?

Is to provide a class library targeting all CLI compliant languages? If yes, then you need to choose the actual framework - for instance .NET CORE which is cross-platform and open source. The language is then only relevant for documentation and QA.

Alternatives:

• provide dll/com libraries targeting C++/CLI
• standalone C++ "unmanaged code" libraries with interfaces for both .net (System.Runtime.InteropServices) and unmanaged code.

This is TensorFlow discussion about C# API - who face similar strategy choice:
https://github.com/tensorflow/tensorflow/issues/18

[n17s]

Thanks for the pointer to the TF issue and the pointers to the libraries.

User convenience is the main reason we are doing the bindings. The library is in C++ so if you are determined enough you can program like this, and call it from C# with unsafe code. But that would be ugly, error prone, and hardly deserving to be called "support for .net".

[ghost]

You are facing some strategic choices for the platform you are building. A lot of CNTK code is redundant or/and dependent on some specific non-easily portable (platform/language) libraries. Looks like you want to re-package the added value over a couple of different directions.

If you target the .net framework (for example .NET CORE) you will have python support free with it on par with other CLI languages (https://github.com/IronLanguages). Not targeting .net means exactly what you had said - poor penetration of potential CNTK users who face hurdles of complexity of interfacing with unmanaged code from their .net environment.

My point of view is product/platform and the necessary resources. It is clear that anything is feasible and usable right now with a given effort. The core of the platform works well - distributed training using GPU/CPU etc.

I would of thought that for model training BrainScript should be enough (with parallel bindings in python). For training via .net applications you could integrate BrainScript into .net or provide a .net library that interprets BrainScript files. You dont have to re-invenent the function by function brainscript binding for model training!

The key area is for the model evaluation. Python users would not notice any difference, while people targeting commercial applications (providing a trained model and an application built on top of the model) would benefit from robust model loading and evaluation via managed code support. There is also less of a performance impact - and smaller percentage of use cases requiring HPC like latencies in evaluation.

I will have a lot more feedback over the next few weeks as we check various CNTK areas - including real-time usage of CNTK Evaluation (C unmanaged code interfacing with native win C++ DLL that uses CNTK Evaluation)

[ghost]

http://venturebeat.com/2016/11/16/google-joins-microsofts-net-foundation/

[bklooste]

@n17s I would wrap the C++ tensor. C# has unsafe pointers for exactly this reason and you don't want large amounts of memory in C# when the lib doing the work is C++ , you especially dont want to move the memory from to C# and then copy it into C++ thats a killer often seen in network stacks. So id provide access to the C++ tensor , pin it to modify . A lot of C# network apis do that as well they hold the packet in native and allow the reading into C# only when desired. Probably consider to just write the wrapper in C++CLI as it fits better with the project however C++ CLI will not fit with core ( and core on Linux) .

Agree on brain script only for training , it makes C# a much lighter easier to keep up to date wrapper. Agree on good support here to load and save brain script files. Some methods to construct a brain script file , maybe via expression tree. ( though an extern c call in brainscript would be nice) .

[n17s]

Sure, the C++ stuff will be wrapped but I believe that's not sufficient. To make this discussion more concrete let's look at this wiki page. An image is loaded, transposed (width x height x channels to channels x height x width), and 128 is subtracted all in one line. This has nothing to do with the CNTK TensorView class, this is all preprocessing to put the input into the right format and numpy is super helpful here. If there's a good library like this in C# we should be leveraging it.

[ghost]

If this is the immediate orientation... whats wrong with using R.NET? https://rdotnet.codeplex.com/ or looking at the msft R roadmap to see if it fits?

[StevenGann]

@avader906 To me, the problem with using R is that if I'm going to use an interpreted scripting language inside my native application, I might as well use Python since it's a lot more developed and supported than R.

[bklooste]

@n17s Its data in C# and there are a number of image processing libraries , ImageProcessor nuget may do - I dont think this should be part of CNTK project at best a community extension to existing image processing . That said my personal project is for the network to do this .

[deviousasti]

Um, I'm only asking since it's been a while - rather than a date, could you confirm if .NET support is indeed planned?

[andrewfry]

Any news or updates on this one? I saw a new release was out, but nothing noted regarding .NET/C# support.

[asdfgasdfsafgsdfa]

2017 now... no love for .net yet? :(

[zhouwangzw]

We have just released the beta version of CNTK Library .NET/C# Eval API. https://github.com/Microsoft/CNTK/releases/tag/v2.0.beta8.0.

[aiunderstand]

Thanks for this important update @zhouwangzw, we are looking forward to see .NET/C# as a first class citizen with both eval and training API's.

[andrewfry]

Definitely a good start.

[lefig]

What is the current status of the .NET interface support? Is it live now..? Thanks

[andrewfry]

There is no API to create networks/train - not that I am aware. They have a beta version of the C# evaluated after you've already trained your network. Personally I have given up hope for .NET support and have been doing all my ML development with TF & Python. The only selling point for me with CNTK was the fact they were talking about support C#, which I love versus Python. Since that does seem to even be a consideration I have moved on. Hopefully they will reconsider ... one day.

[lefig]

Thanks for that. I see. Well I guess its worth trying to use a pythonnet interface between python and .net. I bet the performance would be a dog though.

[SuperDaveOsbourne]

I have just downloaded the RC 2 from April 21st and made a .net c# project with it. Not much there. Is this conformation that c# will be a red headed step child of Python support?

[ghost]

@SuperDaveOsbourne There is everything needed in C# API to evaluate models trained in Brainscript or Python. However, training models vs evaluating models requires a different ecosphere of support libraries and infrastructure.

The core of CNTK is C++ codebase - it is possible to extend the managed API further. From the number of commits it does not look that wider community is participating much in that direction. One has to accept the fact that the product originated from MSFT research, and without much of an external drive, it would be driven by the commercial priorities of the company funding that research.

What might impact those priorities?

• Python is the default "glue" choice for DNN across all open source projects. A lot had been developed in terms of libraries and specific support required for scientific computing. Python is overtaking R in terms of statistics and data warehouse / big data applications too. Arguably, C# does not have the same facilities and the required workload to offer them might not be commercially viable without community participation.

• Python is taught widely - providing steady pool of junior people to join the industry. This could not be said for C# - which remains specialized to middleware and windows applications. It would be logical for C# to focus on model evaluation - easy to integrate with existing systems to run pre-trained models.

You can read between the lines from the recent MSFT announcements regarding the future of SQL Server platform and its satellite products (R Server, PolyBase, PDW/APS) as well as Azure. Python will be brought in on par with R and ML libraries will be broaden. Under that view, Python related commits can be interpreted as priority.

[normanhh3]

@sethjuarez maybe you can go join this team and give them a leg up!? NuML has some great ideas!

[Sam7]

What does the road map for training in c# look like? @zhouwangzw

[zhouwangzw]

Currently we focus on supporting model evaluation/inference with C#, but do not have a plan yet to add training capability to C# in short term. Of cousrse, this can be changed if there are a lot of demands on it. And we are also very appreciated for any community contribution.

[SuperDaveOsbourne]

Seems like Microsoft will miss machine learning train as much as they did smart phones. With 17 years of .net C# it pains me to see that investment is now worthless for ML. If the future is python then the future is not Microsoft. Maybe they have something hidden, but they need it now with NVidia, Google and others making dedicated hardware to address this market. Intel's huge push late this summer for processors with ML shows that if they miss this wave , they will be relegated to a cloud container company not much different than any other.

[aiunderstand]

@Zhouwangzw thanks to CNTK I could evaluate speech, object, gesture recognition data (with multimodal fusion) in a single C# WPF app (for MSc thesis). I trained these networks in python fine, but it did broke the workflow since all other steps (eg. data aquisition with Kinect, analysis, tagging and performance eval) could be done without leaving the app. I'd happily help dev training networks in C# and integrate this in Unity 2017 (with upcoming .net standard 2.0 support) and am sure many others think likewise. So please provide some guidance on where we can contribute and how you see this feature happening. Very much appreciated!

[bklooste]

@SuperDaveOsbourne I dont think Python or C# are good languages for Neural Networks of the future except as thin wrappers ( Tensorflow is C++) ,. I like BrainScript /C++ , why put a thin language on top .

"If the future is python then the future is not Microsoft" This is insightful and correct

[whitmark]

". . . but do not have a plan yet to add training capability to C# in short term"

Disappointing given early expectations set last year and for those looking to deploy the entire train/eval workflow in .NET/C#.

[Sam7]

Thanks @zhouwangzw is there a uservoice setup for CNTK where we can vote for this feature?

As well: how could we go about contributing to a managed training library? Do you mind giving some guidance, I'm having trouble even generating the SwigProxyClasses.

[vermorel]

I have posted a long answer on CNTK should adopt a .NET first approach. I hope it would be of interest to the CNTK team. It feels to me that a major opportunity is being wasted for CNTK.

[mathias-brandewinder]

I find it interesting that the discussion assumes "either Python or C#", and "scripting vs. production code". I have been on .NET for a while, and for everything machine learning related I use F#, which has all the benefits C# has in terms of performance, is a fantastic scripting language, and can be put in production to interop with C# code without issues. For that matter, every time I look at BrainScript, I see a language that is really, really close to an F# DSL. I'd love to see a Brainscript equivalent in F#.

[vermorel]

@mathias-brandewinder Agreed, F# is fantastic. That's why my post was titled a a .NET-first approach and not a C#-first approach. C# was mentioned rather than F# because C# is dominant and I did not want the discussion to be sidetracked in a F# vs C# discussion. The friction to move back and forth from F# and C# is negligible compared to the friction between .NET and Python.

[ghost]

@mathias-brandewinder @vermorel CNTK toolkit has 2 parts - model learning/training and evaluation/forward-only pass. Native C++ API runs both.

In addition, toolkit has wraps to expose the full 2.0 API under Python and 1.0 API + some under brainscript - both for training and evaluation. Separately, some proxy code is provided to run evaluation in Java and C# that wraps around full 2.0 or the evaluation-only C++ API.

The latest seems to be the question of whether to drop brainscript. This would leave C++ API or Python (on top of C++ API) as two CNTK environments supported for model training.

There is some confusion with what "production code" means. Generally - the workflow would be for a data scientist to use python / R / scala etc to compose a model and train/test it. That would be model development. This is what you would typically see in "become ML PhD in 5 minutes" youtube videos.

Production would be to run either the model evaluation alone on specific data stream or the full data acquisition / transformation / model training and evaluation cycle - for the specific business use.

Arguable, one environment focuses on features and ease of use while the other on the stability/maintainability and speed.

Those two environments roughly correspond to what Python and brainscript provide (for now) - and apparent effort is to converge further platform developments and merge both into Python.

There are other uses of "production code" - it could be that Python is used in "production" (example: azure spark hadoop ) or that .NET is the production environment and the need exists to run the model training under that ecosphere too.

.NET is a complicated subject in itself - should the framework be re-developed in managed code or providing DLL wraps is sufficient? A managed version would provide convergence with all .NET languages but what data structures should be used and what is the performance penalty - given that all the (low level nvidia / mkl / mpi) computational stuff will remain in the native code?

Please help the team to make your use cases clear - #1962

[JimSEOW]

@vermorel there is a slightly similar discussion here to your article

[JimSEOW]

To ALL,

CNTK has a small team. We need to be _more organize to help them_. We also need to help them to justify .NET support strategy.

[1] From what I have gathered, .NET support makes sense for production at cloud backend and more importantly for

[2] Front end like e.g.
[a] [mobile platforms ( Xamarin Forms for iOS, Android, UWP, and WPF (WinOnArm)](https://github.com/Microsoft/CNTK/issues/1947),
[b] game engines using .NET (e.g. Unity)
[c] ASP.net

[3] What tools we need to improve .NET support?
[a] Brainscript extension for Visual Studio (Code)
[b] Using this tool to visually create Brainscript for input into .NET CNTK

[c] Xamarin Workbook for .NET (in this case, tensorflowSharp is shown here to port python tensorflow codes to C#)

[4] [.NET as First class citizen ](https://github.com/Microsoft/CNTK/pull/1459)
[a] Comparable tutorials [issues 1967 & 1755] in .NET shared through jupyter or Xamarin Workbook

These notebooks need to be ported to .NET using Jupyter or Xamarin Workbook based on (close to first class .NET managed wrapper DLL)

• CNTK_101_LogisticRegression.ipynb
• CNTK_102_FeedForward.ipynb
• CNTK_103A_MNIST_DataLoader.ipynb
• CNTK_103B_MNIST_LogisticRegression.ipynb
• CNTK_103C_MNIST_MultiLayerPerceptron.ipynb
• CNTK_103D_MNIST_ConvolutionalNeuralNetwork.ipynb
• CNTK_104_Finance_Timeseries_Basic_with_Pandas_Numpy.ipynb
• CNTK_105_Basic_Autoencoder_for_Dimensionality_Reduction.ipynb
• CNTK_106A_LSTM_Timeseries_with_Simulated_Data.ipynb
• CNTK_106B_LSTM_Timeseries_with_IOT_Data.ipynb
• CNTK_200_GuidedTour.ipynb
• CNTK_201A_CIFAR-10_DataLoader.ipynb
• CNTK_201B_CIFAR-10_ImageHandsOn.ipynb
• CNTK_202_Language_Understanding.ipynb
• CNTK_203_Reinforcement_Learning_Basics.ipynb
• CNTK_204_Sequence_To_Sequence.ipynb
• CNTK_205_Artistic_Style_Transfer.ipynb
• CNTK_206A_Basic_GAN.ipynb
• CNTK_206B_DCGAN.ipynb
• CNTK_207_Training_with_Sampled_Softmax.ipynb
• CNTK_301_Image_Recognition_with_Deep_Transfer_Learning.ipynb
• CNTK_599A_Sequence_To_Sequence.ipynb

[5] Here is the list of the .NET CNTK examples
[coming]

• WPF
  for CNTK\Examples\Image\GettingStarted01_OneHidden.cntk
  

• Unity
  

Immediate actions:
[a] Please help to close DUPLICATED issues.
[b] Please work together to laser focus our request
[c] Please anticipate the challenges the CNTK team will face
[d] It is clear chasing the python community will not impact the race against TF any time sooner. What we are doing here is to gather feedback/suggestions to help CNTK go round the frontal race against TF but end up highly differentiated in some interesting future/existing use cases that are still untouched by Tensorflow.

[aiunderstand]

Awesome!!! Thanks Jim for this start of a coordinated effort. To add some discourse to your 5 bullets:

1) Full .NET support (with training) is indeed key for production. We need the ability to choose if datasets are trained client side or server side. Not all datasets are large and not all training takes hours. Sometimes a 80% fit is good for the purpose at hand. With the rise of dedicated ML hardware, client side training will become more and more viable, especially for often smaller, "your data" datasets. Finally, another reason for training support is the ability to update your model in real time (again either client or server side). It's a strong selling point that your app gets better with more data automagically (although i know that ML doesnt work like that, see chatbot Tay). But for some controlled domains this actually works.

2) Absolutely, front side is what will sell CNTK, it makes things pretty and is what solves a real problem. With Unity I can currently build CNTK apps in WebGL and webassembly with footprints of less than 10 mb. I havent tried this on mobile, but am actively discussing options with the devs on the Unity WebGL team. The state of the web is also heading for proper threading support using SAB soon (see this link with Unity supporting it soon after. Then mobile support of CNTK will be a blast -- well client side computing web will be blast actually :)

3) Currently we can't integrate CNTK as a binary (.dll) in browsers since the death of ActiveX and NPAPI (loading native dll's is prohibited). This one is actually really really tricky. For example we could use the approach of the legendary @migueldeicaza, a MSFT employee that ported TF to C#. Line by line. With emscripten we can then transpile all C++ code to Javascript to run it on the web. But this means all dependencies must be c++ code too and maybe some systems calls are not supported by emscripten yet (i'm not an expert on this)

In short: I'm not sure what approach is best here but ideally the tools make it possible to build a web client with CNTK to make it future proof.

4) I'm not aware of any similar jupyter notebook technology, maybe some one can chip in? To foster a community of .net ML developers we definitely need tools to easily teach the workings of CNTK in .NET as best as possible. As mentioned before the web client in Unity is able interact with CNTK in real time and output results, but the training part is lacking.

5) As soon as training support becomes available, exact ports of the tutorials can be made. Until then we can at least build partial tutorials, where we train models with python and eval in .net. My efforts will be captured here with the Image recognition task 201A already done. I'll clean up my code up a bit and update my repo this weekend. Hopefull more efforts are done to kickstart a CNTK.NET ML movement ;)

[JimSEOW]

@aiunderstand Like you, the challenge is complex, no one could easily see the whole picture. However, like you, I join you to bring people here. Let us keep the discussion going.

Ideally, I would like to see the porting of the python notebook to .NET using the PR. Have you tried Xamarin Workbook before? They are interested for ideas how to improve it to support .NET community.

[vermorel]

@JimSEOW I have posted a rather long winded discussion about my more detail perspective on a .NET-first roadmap for CNTK http://blog.vermorel.com/journal/2017/6/6/details-on-the-net-first-strategy-for-cntk.html

[JimSEOW]

@vermorel Could you look into Xamarin Notebook as a replacement for Jupyter. The team at Xamarin is very interested to position it for sharing .NET codes for all platform and OSs. I have the impression that they will want to implement "Very interesting" features not found in Jupyter but VERY .NET centric.

[vermorel]

@JimSEOW Xamarin Notebook is very cool, but is it really a priority for CNTK? If the CNTK team concentrates on _backend bindings_ for .NET (i.e. network description + advance data I/O), then, Xamarin Notebooks are basically a given.

[JimSEOW]

@vermorel There are CNTK team (dealing with, I guess backend issues of interest to you) , perhaps Xamarin team very interested to promote Workbook for .NET. Both are from Microsoft. The more Microsoft teams involved in pushing AI [both backend (Azure) and front end (Cross platform AI) , the better, isn't it?

=> I am currently investigating putting more classes into .NET wrapper.

[MgSam]

Just to add my 2 cents...

There are millions of C# developers. Developers that use the Microsoft stack are overwhelmingly using C#. First class C#/.NET support (including training) is critical to spur adoption of this toolkit among Microsoft developers.

I don't want to have to learn BrainScript to train a model or set up a parallel Python environment in production to train models. Arguments in this thread saying how these other languages are a better fit for ML are silly- the package should offer support where the users are, not vice versa. I think the Python/BrainScript support is great, but it's frankly insufficient.

If the staffing of the CNTK team is not enough to develop this support properly then it needs to be increased. Microsoft just spent half of the Build conference talking about ML- it seems the company strategy is clear- the dollars and .NET support need to follow.

[gamemachine]

I'll cast my vote for .NET on this. But even more then that what is needed is a developer first approach, not a data scientist first approach. As someone using ML in real world stuff, it's been a huge pain working with ML because most of the people involved, are not developers by trade. Our needs are just very different.

I don't need the 'perfect' ML language. .NET languages are just fine, as are most languages not python.

I could care less about Notebook. In fact notebook gets in the way. I'm not a student! I want to see good abstractions that make the library easy to use. A few examples, and decent documentation.

I'm frustrated that ML is yet one other place where .NET is a ghetto. And I'm at a loss for why a MS project doesn't get that this is a bad thing.

[mathias-brandewinder]

@JimSEOW Jupyter notebooks already support .NET, via F# and the IFSharp project. This is, for instance, one of the supported languages on Azure Notebooks.

[SuperDaveOsbourne]

My thoughts are that Microsoft is treating ML like Apple treated Maps. Google had 6000 people working on maps and Apple 200. The Apple CEO was mystified on why Apple maps was a joke. No amount of open source shoveling by humble users of CNTK will matter. Without massive support of Microsoft, this effort is doomed. Python reminds of all the dll hell of the early 90's. I have yet to find a ml demo that works without finding out which version of python works with CUDA, Which version of Tensorflow with run with what version of Python. The latest version of Tensorflow doesn't work with python 3.6 on windows and never will.

So even if you continue with windows and python, development is on a second class track.

Hopefully someone at MSFT corporate reads these blogs and will wake up from there coma.

[JimSEOW]

@mathias-brandewinder Yes, I know Juypter supports c# etc.
Xamarin Workbook is built using .NET with SDK etc. Perhaps there is a chance of doing something very different from jupyter.

[JimSEOW]

Feedback from the CNTK team

[aiunderstand]

Thanks Jim, all. This wonderful response of the core team and the vivid responses even outside github is already an achievement to raise .net ml awareness. Training with cntk must become a priority. And, we as a community need to prove we can reward that effort by submitting ideas, use cases, tutorials, lots of code and responses to questions on SO and here. We need more gh-stars 😁 and who's going to write the first cntk .net book 😉 I'm looking forward to become a part of this community and share my real-time AI passion and experience in the geo domain.

[dsyme]

I would be interested in helping trial F# support. (cc @mathias-brandewinder). Either that or I would use work on TensorFlow + F# :)

[mathias-brandewinder]

This is maybe a bit off topic, but I thought I would drop a thought here. When I started learning CNTK, I immediately gravitated towards BrainScript, because it looked like a great lightweight language to specify a computation graph. As it happens, BrainScript syntax is also pretty close to F#.

My understanding is that the CNTK team is not sold on developing/supporting BrainScript further, which I understand - maintaining a dedicated language is resource consuming.

In that frame, what I'd like to explore is, would it make sense to write a BrainScript-like DSL using F#? I would personally love something like that, because I would get both a nice, lightweight, VS/Vs Code ready language for my network specification, with all the benefits of static typing. And leveraging a language like F# would make it much easier to get community contributions to maintain and expand.

I would be very interested in taking a stab at it, but would need some guidance on where to look at for how the BrainScript-to-C++ part works; any comment on whether this is a good / bad idea, or what problems can be expected, would be awesome as well.

If someone on the team reads this, I can be reached at mathias dot brandewinder at gmail.

[JimSEOW]

=> Let us be Crystal CLEAR: We know exactly how CNTK stands against Tensorflow. CNTK is severely behind except a few fronts. e.g. C# [Unity, WPF, Azure] and F#.

One more time. I ask the followers here to be patient. CNTK team has limited resources. If there is slightly more effort put on C# and later F#, please continue your support. It is not C# versus F# here. It is all about .NET Deep Learning (CNTK and TensorflowSharp)!

[JimSEOW]

@mathias-brandewinder CNTK team has to fight many fronts. It is great that you would like to push BrainScript. Please organize the followers here who ever also like to contribute to BrainScript. The more we work together, the more CNTK team will take our views seriously.

[tobyclh]

Hello, I just come to this issue thanks to @aiunderstand, and in this thread all I have to say has almost been said so maybe just a few words.
I appreciated the .NET binding for what it is, lightweight, cross platform evaluation framework for what it is, and I still do, Thanks CNTK team if you are reading this! However, there are few cases it really hurts not to be able train your network in the clients machine.

From a HCI researcher perspective studying the individual user's behaviour and adjust your application, be it MR/AR apps or plain smartphone app is a wonderful way to provide a smoother UX.

As a game creator, I am creating this binding for Unity users to use CNTK as if they will use it like an in-engine feature. It works very nicely now. Yet, when I am trying to implement Style Transfer Tutorial and some others recent state of the art papers that are obviously highly applicable in content creation like Reflectance Modeling by Neural Texture Synthesis the training feature become so desirable. Last but not least, +1 for more tutorials, which is severely lacking right now.

[mathias-brandewinder]

@JimSEOW totally understand the fight on many fronts problem :) Just for clarification, what I am proposing here is not to contribute to BrainScript per se, but to (re)write something very similar to BrainScript, using F# instead. That being said, to do that, the first step will be to dig into the inner workings of BrainScript.

[JimSEOW]

@tobyclh IT will be severe Short Sightedness on the part of Microsoft (not addressing the CNTK team specifically) to focus ONLY on the [backend] Azure Cognitive Use Case. With Microsoft Mobile Reboot, with "Always On" connected cellular PC in the near horizon Q4 2017, with the need to tailor Learning at the Front End, be it game (Unity), be it adaptive UI (e.g. Xamarin Forms - one GUI for all mobile platforms) : I see FrontEND - the NEXT new big opportunities AND for that : WE NEED .NET DEEP LEARNING !!!!!!!!!! - We need Microsoft, to allocate MORE resource to .NET Deep Learning. Be it .NET for CNTK or TensorflowSharp.

[skynode]

Considering that CNTK was initiated by the research arm of MSFT, I think it's not abnormal to see their hesitation and current research-focused stance on the roadmap for CNTK. The .NET businessmen like yours truly and others might just be asking a dressmaker for a haircut here.

On the other hand, it's not impossible for the community to drive a CNTK pivot to a production-grade machine learning platform. We may require some architectural/core C++ library details from the MSFT Research team but that's about it really.

We have great antecedents in Redux (by Dan Abramov) and the Mono/Xamarin Project (by Miguel de Icaza). So maybe the time for talk is over.

[skynode]

Furthermore, I don't think it's appropriate to approximate the request for CNTK .NET bindings/support to some machine learning arms race against the TensorFlow community. Businesses​ will pivot to whatever platform that meets their unique requirements. I love the .NET platform but I care less who is winning the apparent ML arms race.

[JimSEOW]

@skynode It is not about Red Ocean arm race. It is about .NET Deep Learning (it could be .NET for CNTK or TenshorflowSharp or BOTH). It is about Blue Ocean New Business Opportunities presented by having .NET Deep Library. It is about the Microsoft vision to democratize AI to .NET community. We WISH that the vision IS NOT restricting the community ONLY to python and c++.

[denfromufa]

Pythonnet core developer here. Recently ML expert from Intel used pythonnet to call into TF from C# (F# should be fine as well):

https://github.com/pythonnet/pythonnet/issues/473

I'm open to suggestions about Keras & CNTK tutorials using pythonnet, since I'm going to present at PyData in Seattle at Microsoft.

Contact me at my username at gmail dot com or mailing list for pythonnet.
I used pythonnet to call into most of popular numerical python libs without issues. This is not hard, especially when using REPL such as scriptcs, csi, icsharp, and clrmagic.

Managed or native .NET API is great, but could be significant effort to develop and maintain, especially for 3 different CLR runtimes.

[JimSEOW]

@denfromufa Great you are here to LEND us a big land 👍

More to come on how we could pick your brain. Feel free to share your VISION on how we could clearly layout the different options for the .NET communities to integrate deep learning.

We are face with different use cases. For Unity, it is not clear if pythonnet can be relevant here.

However, I could EASILY think of many different roles how pythonnet will be pivotal in our effort to port python codes to .NET. (both CNTK and TensorflowSharp).

More will come :-) WELCOME AGAIN!

[denfromufa]

Here is prime example of using Python.NET from C# to call into Keras, scipy, sklearn, theano, tensorflow, etc, see also comments' section:

https://www.quantconnect.com/forum/discussion/1918/update-python-library-support

[skynode]

@denfromufa thanks for your really valuable work on this. I intend to play with it this weekend. Just wondering, have you had any performance issues or reports from current users?

[denfromufa]

I'm not aware of perf. bottlenecks or at least there are workarounds. Feel
free to report back if you find issues, there is also "python.net" tag on
SO.

On Fri, Jun 9, 2017, 8:08 AM James Valkyrie [e-mail]
wrote:

@denfromufa https://github.com/denfromufa thanks for your really
valuable work on this. I intend to play with it this weekend. Just
wondering, have you had any performance issues or reports from current
users?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/CNTK/issues/960#issuecomment-307383611, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHgZ5QIhC_G-G2Liq7OO6yVZ5LeOtKjqks5sCUOsgaJpZM4KiTg-
.

[JimSEOW]

@denfromufa Could you please look into this comment by @migueldeicaza

Google has built a number of convenience libraries in Python that are not available to the C API (and by extension to the C#, Go, Java, Rust bindings), so we would need to port some of that code.

I wonder if pythonnet could contribute to above? As we gain more experience how to do that completely in pure C# or F# codes, especially for production deployment (e.g. Unity), independent of python environment.

[vermorel]

@JimSEOW We have just released an super early draft of CNTKSharp. This is bringing the evaluation capabilities of CNTK to .NET. We would be thrilled to engage a more direct discussion about this project with the CNTK team. My email address joannes.[email protected].

[JimSEOW]

New updated CNTK .Net example

Preliminary

[JimSEOW]

TensorSharp

[JimSEOW]

TensorflowSharp Workbooks by LarcAI Technology

[aiunderstand]

Hi CNTK.NET community, just a small update on my effort to code a open source Unity-CNTK integration for cross-platform. realtime AI. Look forward to play with it very soon on initially the Windows and WebGL (with ASP.NET backend) platforms.

Currently features:

• 2 tutorials:
  evaluate random CIFAR 10 image or upload your own image
  evaluate random MNIST image or draw your own digit (integration of this project by Rodrigo Fernandez Diaz)
• Material Design UI system (integration of this project by InvexGames
• Unity Postprocessing effects (this project by Unity)
• StandaloneFileBrowser (this project by gkngkc)
• 3D data of Amsterdam in minecraft style (Geocraft by Geodan)

[JimSEOW]

@aiunderstand Impressive! Do keep us update regularly. The feedback I received puts Unity an important Deep Learning .NET use case opportunity.

[aiunderstand]

I will Jim! The windows build is as-good-as finished, including a msi installer script for easy deployment (needed for the vc++ 2015 runtime dependency). Currently im refactoring code to let the community create new tutorials by dragdroping a single prefab.

I'll update my repo the moment I finish the webgl client with ASP.net server code with (bs and python APIs), making demonstrations easy.

On a side note: Early test show that the windows client evaluates in real time ( 30+ fps) while cloud based is only a fraction (1fps). It will be a fun challenge to improve that. However the main challenge is support for .net training support. Especially automated model update routines (in c#) when new data becomes available would be worth much.

[JimSEOW]

@cesarsouza who pioneers Accord.NET is allocating some time in helping tensorflowSharp. Great!

[JimSEOW]

TensorflowSharp for .NET Core 1.1

[JimSEOW]

Please come and join us at Microsoft/CNTK at Gitter

[JimSEOW]

TensorflowSharp coming to Xamarin Android native

[JimSEOW]

New tensorflowSharp Inception test

[JimSEOW]

Not related to Tensorflow and CNTK for .NET but interesting
SharpLearning

[danielsc]

As we are planning for a broader .NET support, we would like to talk to some folks in the community who are interested in using CNTK with .NET for training/tuning scenarios.
So, if you have such a scenario and are willing to talk to us about it, please contact me at my github name (at) microsoft.com

[JimSEOW]

@danielsc Thanks for reaching out to us!
I will definitely try to summarize the feedback I have gathered for you. For the users here, feel free to feedback what you need, either sharing some of your ideas here or contact Daniel directly.

When I started this .NET Deep Learning initiative, there were close to 800 open CNTK issues. As of today, this number has almost reduced to half. Great job CNTK team!

Great job for addressing our interests in .NET Deep Learning.

[aiunderstand]

Wonderful @danielsc ! I've been busy with side projects, at the same time i'm very close to releasing this project (see screenshot). I will test it with the Hololens next, but it works at 90fps in VR with the HTC Vive and GTX 980 PC (as you can see in the image, there is a rough benchmark function to test this).

[JimSEOW]

@aiunderstand Thx for updating your progress. Very Impressive!

[SuperDaveOsbourne]

On Tech Crunch a few minutes ago:

Microsoft has created a new research lab with a focus on developing general-purpose artificial intelligence technology, the company revealed today. The lab will be located at Microsoft’s Redmond HQ, and will include a team of more than 100 scientists working on AI, from areas including natural language processing, learning and perception systems.

[JimSEOW]

Keras Sharp coming to .NET deep Learning

The .NET Deep Learning community has answered. WE WANT A BROAD .NET Deep Learning strategy where C# is treated as FIRST CLASS CITIZEN - so as python.

It is timely that MSFT provides the leadership.

=> WE are MOTIVATED by many NEW features coming to Windows 10. We would like to integrate AI to all these new FEATURES. WE want to democratize AI pervasively in all aspects of Windows 10 that empower Business and Connecting people together.

We see this future CLEARLY through .NET managed languages (C#, F# etc. ) _instead of c++ and python._

[danielsc]

To be a bit more precise on what I am looking for: we need to prioritize which scenarios we are should enable first and therefore we need to understand what is more important to you. So we want to understand better what problem you want to solve and what algorithms you want to use in .NET.

Here some concrete questions that I would really like to get some answers to -- either directly here, or if you prefer via email to my github name (at) microsoft.com

1. Could you describe the scenario you would like to use the .NET API for? Which business problem would you like to solve (For instance: Predict whether a customer in the sales funnel is likely to buy; Detection of faulty products based on photos; Detection of anomalies in data)?
2. Which type of network architecture are you thinking of using (for instance: LSTM, CNN, Fast-R-CNN)?
   Are you planning on building on a pre-trained model or do you expect to build up you own network from building blocks? Are you using a reinforcement learning scenario?

[pedronahum]

For Reinforcement learning, I would be very interested in training a deep network in C#.

I believe that Azure Service Fabric or Microsoft Orleans could help to train an Evolutionary Strategies or A3C approach, for example.

The challenge in RL is how to scale to thousands of workers and how to smooth the communication among them. Microsoft has already the cluster tools, we just need to be able to use CNTK natively in those tools.

Great work with CNTK btw.

[veikkoeeva]

@pedronahum See at https://gitter.im/Microsoft/CNTK?at=5954c563bf7e6af22c863c39.

[veikkoeeva]

A note about improvements to .NET numerics, such as tensors at https://gitter.im/Microsoft/CNTK?at=59824118210ac269204b229c and it might make sense to try to pull the various people together to see if there's a chance to synchronize minds and see if it could in the future show in a roadmap somehow.

[danielsc]

We just posted the iteration plan for the August iteration: https://github.com/Microsoft/CNTK/issues/2194
In this first version of a .NET training API. We aim to enable building and training of simple models like MNIST.

[JimSEOW]

CNTK for ARM

• W10M Windows 10 Mobile [UWP ARM 32-bit] ;
• W10 on ARM [W10A UWP ARM 64-bit] and
• W10 IoT Core on ARM [UWP ARM 32-bit: e.g. Raspberry Pi,· Qualcomm Snapdragon 410]
• W10 IoT Core Pro on ARM [UWP ARM 64.bit]

Issues involved

• How to Build CNTK for ARM64?
• ARM support?
• UWP ARM

[vermorel]

For the reference, I would like to point the .NET deep learning project: Alea Tk. Their C# integration is impressive. They achieve C++ GPU speed with C# and .NET. I am personally impressed by the work done by QuantAlea.

[nmfisher]

+1 for C# support.

[fwaris]

.Net integration is very desirable but I think that F# should get special treatment because of it modeling capabilities. I have laid out the case for F# in another thread.

(thanks @veikkoeeva for cross referencing )

[n17s]

CNTK 2.2 has a C# API. We welcome feedback and contributions for enhancements.

[SuperDaveOsbourne]

That's one small step for C#, one giant leap for .net mankind.

A hello-world example to train and evaluate a logistic regression model using C#/API: https://github.com/Microsoft/CNTK/tree/master/Examples/TrainingCSharp/Common/LogisticRegression.cs

[SuperDaveOsbourne]

I have created a slack channel for CNTK for dot net. This is a public link.

https://join.slack.com/t/cntkdotnet/shared_invite/enQtMjQyMzE0MTA3MzYzLWEwNjZlYWFmMTI2NzZhZGE3YmQzOTNkY2VkZGM3NDFjODMxNTUwY2M5ZGU4NDQxZDUxZDk5MTdmYWQ4MjY4ZWM

Workspace URL: cntkdotnet.slack.com

[veikkoeeva]

@SuperDaveOsbourne Why not use the CNTK Gitter channel?

[danielsc]

As Nikos pointed out, we now have a C# API that allows for both training and inference. It covers a reasonably broad set of scenarios (including RNNs and Transfer Learning), but it will require further improvements to reach parity with the Python API at a good performance.

Also, we have built this API as a pretty direct mapping to the C++ API using SWIG. This allowed us to move pretty fast, but does not make for a pretty API that follows well established C# design conventions. Our intention here is to build a high-level API on top of this low-level one that would offer easy composition of layers, like the python layers API does.

On both aspects (improve the low-level API, design the high-level API) we are looking to the community to participate and help. Please open issues for improvements to the newly published APIs, and, if you see a way for you to fix them, do let us now, so we can coordinate.
For the high-level API, we would like have some design discussions in the coming weeks -- we will open a new issue for it and will post the link here. At that point, I would also close this issue as resolved -- please let me know if you disagree.

[vermorel]

@danielsc The .NET/C# lib draft is indeed a very good news, and it's very cool to see the CNTK team moving in the right direction for the .NET ecosystem. However, at this time, issues like Microsoft/CNTK/issues/2374 (a design flaw severely impacting performance) prevents most production uses of the library (except for small datasets). This design problem cannot be fixed by layers on top.

[deepakkumar1984]

Could be useful project: https://github.com/deepakkumar1984/SiaNet
A wrapper on CNTK for easy building of network and train them

[laygr]

Skip C# and prioritize F#, it can be easily advertised as a better Python. Honestly, what is good about C# for ML? If you want to do ML and .Net, lookout for F#.

[mikhail-barg]

@laygr I think it's a matter of language preference and have nothing to do with ML. I'd easily say "what's good about F#, do C#!" but there's no point in starting a functional vs. imperative holywar.
As for us we've choosen CNTK mostly because it had native support for .Net (meaning C#) so we could use it seamelessly in production code.

[deepakkumar1984]

@laygr I agree with mikhail-barg. F#, C#, VB all under .NET hood. You can load CNTK in F#. I think the .NET community need to build more libraries similar to python for machine learning.

[laygr]

@mikhail-barg, @deepakkumar1984. I should have given an example to be more convincing; let's look at the case of Solver Foundation. Microsoft wrote not too long ago a truly great optimization framework called Solver Foundation. It supported .Net, but its design was centered around C# and OOP. However, they also wrote a simple DSL for F# which allowed to define the models in a very succinct way. Lets look at how a model is defined in C# and how it's defined in F# via the DSL. (More on the evident superiority of F# at modeling for Solver Foundation: Units of Measure demo blog post, DSL Demo, check out how constraints are defined.
Clearly, the F# idiom is superior, really, no subjectivity going on, except that by not centering the design of Solver Foundation around F#, the DSL was lacking on functionality, I guess it was not a priority; so if you wanted to use the full power of Solver Foundation, you couldn't use the DSL and you had to write code in the clumsy OO way.
Today you can't use Solver Foundation at all because it is a dead project. Will history repeat?

@deepakkumar1984 I agree that we need more libraries to remain competitive against Python. For a quick fix, we can use IronPython, which allows us tu use Python's libraries in .Net.

[mikhail-barg]

@laygr Do you really mean that this Solver Foundation project is dead because MS designed it more for C# not for F#?

For me personally it would be a huge downside if CNTK would switch it's focus from C# to F# just because I live more on imperative side of programming universe, and I would have to learn another language just to be able to configure CNTK models (learning another language is not bad by itself, but it looks like an overkill for a modelling task) and it would probably be another DSL built upon F#, right? So why bother, there's already a DSL for these models — BrainScript (and we invested time in learning it already).
So I rather then learn more of Python (which I kinda hate) and it would be a better investment of time because Python is a lingua franca in ML society (sadly).

@deepakkumar1984 I aslo agree on the lack of ML libraries for .Net — I've spent some time just to find that there's no real sound processing library for .Net that is capable to reproduce scipy.signal.spectrogram out of the box.

@laygr As far as I know IronPython is rather dead as well. And I'm not sure on the performance side of the C#-IronPyhon-Python-Cython stack (and most of ML Python libraries are built upon native Cython code).

[vermorel]

Referencing here the Tensor<T> type coming to .NET which would be a great addition for CNTK.

[veikkoeeva]

Cross-referecing discussion at https://github.com/Microsoft/CNTK/issues/2352 since it's about .NET Standard 2.0.

[MgSam]

I tried going through the C# CTNKLibraryCSharpTrainingExamples today. I think its a great start!

I have some feedback, please keep in mind while reading this that it's all from the point-of-view from a complete ML novice who doesn't understand much about what goes on under the hood, only about the general ML workflow of training, validating, and applying a model.

Feedback:

• I think there should be a higher-level API than what currently seems to be available (at least as far as I can tell). For example, the Azure Machine-Learning Studio provides a Training Model step that requires literally only three inputs- a model, a dataset (which can be of hybrid types like strings and floats), and what the output variable(s) are. Everything else is set using what I assume are sensible defaults. I'd imagine that for the majority of users (who are not statisticians or ML experts), this is how they want to get started. Tutorial I used
• Why is the logistic regression network linear model defined explicitly in the code (create LinearModel method) rather than part of the library? According to the CNTK tutorial it has been a staple for the past decade. Under what circumstances would a non-statistician be knowledgeable enough to define a different model?
• I understand the convenience of generating random data for testing out the library, but its not a good way to help people learn. I think you should put a CSV with real-world data (ideally with mixed strings, dates, and floats) in the example solution, and in the example Program.cs, load it into memory and pass it into the TrainAndEvaluate methods. Seeing the code work on actual data instead of random numbers would be a lot more helpful in helping to understand how its working.

Thanks for all your hard work on the C# API!

[fwaris]

Bravely or foolishly taking matters into own hand see issue #2863

(functional nature of F# better suited for math modeling - at the end of the day the DL toolkits are Math languages)

[veikkoeeva]

Some F# and CNTK at http://brandewinder.com/2018/01/14/CNTK-etudes-sequential-model/.

[ivannp]

Another proof-of-concept Keras-like approach - NativeKeras.

[sgf]

CNTK developers have no belief.
Actually, Microsoft, many developers alike.
They are no longer capable of creating great works like Windows, Make history.
And now they are just doing the same like others.
Just Follow the footsteps of others.

[veikkoeeva]

I'll add here for the benefit of passers-by, the .NET kid on the block seems to MachineLearning.Net. It should be able to use CNTK binaries also.

[Jiuyong]

多好的 Windows Mobile ，非要东施效颦，
把 WP 弄成这个死样子还不够？
机器学习还要这样搞？
18年 .Net/C# ，非要这样自寻死路？
人家公司坑用户，
M$现在不吭用户，只坑粉丝。

[Gregor959]

F# is still .net. if F# support is given and works fully ( e.g. it's not patchy) then it would be possible
to write a F# library of function that use CNTK. This F# library that can be called from c#. It does not even need to be compiled, since it can be used directly in your C# project. The functions and classes can be called from C#.
So as long as F# is functioning, we have .net support .

I have not coded F# for a few years and only tried it for two week, but it certainly did not hurt my coding to read and write a few functions in F#. ( My C# and coding in general actually improved a lot by doing that) So if you are a C# coder don't be afraid to read a few F# function or classes , it will not kill you. You don't even need to learn a new IDE or a new set of libraries .

So this is what I am hoping for - that the F# will be supported as well as the Python language is supported.

[GuntaButya]

C# and CNTK are currently very poorly Wrapped!

The CNTK Core I really want to use, but I am not a fan of Python, nor F#, I like the C Style, Java Style. After all, this is the original Programming Language structure, how C++ and C are constructed, like the C++ CNTK Core...

So WHY such a terrible job of wrapping the CNTK Core with C#?

I am frustrated, C# is an absolutely ideal Graphing Language, CNTK is based of Numerical Graphing, we have an Ideal Match!

Sparse Tensors, or Vectors are not working, Value Class or the Binding to the Core is messing with Values.

I just think a better job should have been done before releasing this! I am appreciative, don't get me wrong! But what's the point of releasing something when it has so many Buggs!

[GuntaButya]

C# Programmers unite!

Show MSFT we are in need of more! Mean while I have started a website: www.cntking.com for a little push.

[zhongkaifu]

I developed some C# projects for deep learning and they are widely used in different projects and areas. Currently, I mainly focus on this project: https://github.com/zhongkaifu/Seq2SeqSharp It can be used for sequence to sequence task. It's tensor based, supports multi-GPUs, automatic differentiation, built-in Transformer, LSTM network and many other features. You can try it and let me know your feedback if you like. :)