Embed AI in Windows Applications

Much recently, Microsoft released Windows Machine Learning stack – WinML. It enabled developers to use pre-trained machine learning models in Windows Applications. To me, this is second big enabler after giving ability to IoT developers to run ML jobs in Edge devices – Azure IoT Edge.
Earlier version of my IoT Gateway project were severely limited by the Cloud connectivity and speed issues. Azure IoT edge allowed me to run some intelligence in the edge (gateway) device. This enabled some face recognition use-cases. I am really excited about power and possibilities after trying out WinML recently. It can surely enable new use-cases in IoT and Security.
BTW, this is made possible by ONNX – Open Neural Network Exchange Format. It enabled a model to be built in one Framework and exported to be used in another. WinML uses models exported in onnx format.
Here’s a quick commentary on how to get started.


My Development setup is Windows 10 Version 1803, build 17134, corresponding SDK, Visual Studio 2017 with Visual Studio Tools for AI.
We want to use a Neural Network model in our app, hence, to start with, we can download from a few available models from Azure AI Gallery.
Azure AI Gallery
For this experiment, I picked an Emotion recognition model from below link.
You can download a model as *.onnx file. When you add this onnx file, a corresponding c# file is generated. Add that file to your project if it is not automatically added.



Emotions.cs has three sealed classes

  1. CNTKGraphModelInput: Initializes input type for the model, which is a VideoFrame
  2. CNTKGraphModelOutput: Initializes output type from the mode, which is a list of floats
  3. CNTKGraphModel: Initialized model and binds input and output for async evaluation.

Load Model

StorageFile modelFile = await StorageFile.GetFileFromApplicationUriAsync(
                                new Uri($"ms-appx:///Assets/Emotion.onnx"));
CNTKGraphModel model = await CNTKGraphModel.CreateCNTKGraphModel(modelFile);


Create a VideoFrame Input

VideoFrame vf = VideoFrame.CreateWithSoftwareBitmap(softwareBitmap);
CNTKGraphModelInput input = new CNTKGraphModelInput();
input.Input338 = vf;

Evaluate with Model

CNTKGraphModelOutput output = await model.EvaluateAsync(input);
foreach(var item in output.Plus692_Output_0)


Neutral is 3.2, Happiness is 0.9, Surprise is 0.8 …
Happiness: 12.4 🙂 Me too
WinML is a very powerful tool and a great progress in direction of making AI omni-present and always accessible. This will allow edge devices to be more intelligent. Models running local should address concerns around data privacy, hence a big leap. On DirectX12 capable devices, WinML can use GPUs for evaluation. This is really exciting time!


Source code published on Github

I am a curious mind, always building and experimenting, trying things to decode their inner design. Kind of a small scale inventor ;-) I like to help people realise their passion.

Leave a Reply

Your email address will not be published.