SharePoint and Machine Learning

So, by now you should have all heard about machine learning and what it can do for us. If you haven’t head of the Wikipedia page below and read about it:

https://en.wikipedia.org/wiki/Machine_learning

Of course, it talks about artificial intelligence etc. all the things that we are not quite sure about. However, what it does say the following: 

“Machine learning is closely related to (and often overlaps with) computational statistics, which also focuses in prediction-making through the use of computers. It has strong ties to mathematical optimization, which delivers methods, theory and application domains to the field. Machine learning is sometimes conflated with data mining, where the latter subfield focuses more on exploratory data analysis and is known as unsupervised learning. Machine learning can also be unsupervised and be used to learn and establish baseline behavioral profiles for various entities and then used to find meaningful anomalies.

Within the field of data analytics, machine learning is a method used to devise complex models and algorithms that lend themselves to prediction; in commercial use, this is known as predictive analytics. These analytical models allow researchers, data scientists, engineers, and analysts to "produce reliable, repeatable decisions and results" and uncover "hidden insights" through learning from historical relationships and trends in the data”

This means something great and can help us with the work that we all do each day. One of the hardest tasks is often finding relationships between tasks, content or even teams of people. Machine learning as it is based on math, has a greater ability to do this and at great speed. 

Microsoft along with other vendors is spending lots of those research dollars on this very topic. You can see what they are doing from this page.

So, how does this fit into the Office 365 and SharePoint space?
Office 365 has the Office Graph which is using machine learning components to create the relationships between people, objects and content. 

Undercover though the core Azure Stack has a whole suite of what’s called “Cognitive” services that can be harnessed through development. To use these services, you first have to register with the Azure Management Portal. To do this, simply log in at https://portal.azure.com and then search for “Cognitive Services”.

Once found click to add a new service after which you are presented with some settings for your instance of this service.

For the “API Type” we are going to select the “Text Analytics API”.

And of course, choose the “FREE” option to test.

Once this is deployed into your Azure Tenant, you should then see your named instance become available. Clicking on this will take you to the properties and administration screen.

Take note here of the “Endpoint” and then from the “Keys” tab, the first “Key” listed as you will need both of these for using the service.

Now we have this setup we can create a sample C# console application with hardly any code. For this we will create a standard windows console application and create two string constants, one to contain the API URL and the other the Key we need to access the service.

Now we need to add a method for actually calling the service which requires us to use the HttpClient object passing in specific headers such as the subscription key we have from the Azure site. Importantly we need to also pass in the actual endpoint we need to call, so for the “Text Analytics” we use: "text/analytics/v2.0/keyPhrases"

Within the code, you will also see the “byteData” needs to be populated with our text, or document content that we wish to send to the service. For this sample, I used the paragraphs found here wrapped into a JSON format.

Now for the console application we need to escape everything to work, so mine looks like this:

Last but not least is a method that actually makes the call out to the web service.

Now we have that done, what does it do?
Running the console application will post the content from the JSON file to the web service, perform text analysis on it using machine learning and because we are using the “Key Phrases” endpoint it will iterate the entire content and return what it thinks the key phrases in the content are.

The text version (more readable) of the returned values are:

Now what does this mean?
Well it means that the Machine Learning code is smart enough to pull out important phrases that are within our content. What a powerful took that is now available to us. Now I only used it within a console application but now imagine we tie this together with SharePoint, a large document repository.

Let’s imagine the cycle that we could implement to better tag and classify our content, making it easier to search.

As you can see this process would be fantastic for a SharePoint site, facilitating auto classification potentially, but at least enriching the properties of the content. If we were to include the other endpoints in there too, we could then get language and sentiment of the content too. Also, remember the options that are available for other cognitive services such Translation, Recommendation, Emotion, Moderation and even Facial recognition. Combining these services together and making them available within SharePoint would only benefit those of us that collaborate and create content. 

Let’s make sure that these new types of services are on our roadmap for the solutions we build.