OMS Data Collector API and Twitter Sentiment Demo (Part 1)


With OMS Data Collector API you can now input any data into your OMS workspace, and not just computer data. The API allows programmatically insertion of json documents into OMS Log Analytics, so you can search, correlate, visualize and analyse it. In this tutorial, you’ll learn how I did my twitter demo for Ignite NZ 2016. Where I collected all tweets with #MSIgniteNZ, and filtered by session code to build a dashboard in OMS.
capture20161028120923000
There are 2 parts to this blog, this being the first part:
Part 1 – Create OMS Data Collector Azure App API
Part 2 – Create Azure Logic App
What you’ll need:
• Visual Studio (I used Visual Studio 2015 with Update 1) and ;
• Azure SDK (I used Azure SDK 2.8.2, which you can install with the “Web Platform Installer”)
• A twitter account
• An Azure Subscription
• OMS workspace ID and key;
• Cognitive Services Text Analytics subscription. You can sign up for a free one of up to 5000 transactions per month here: https://www.microsoft.com/cognitive-services/en-us/text-analytics-api
https://azure.microsoft.com/en-us/documentation/articles/cognitive-services-text-analytics-quick-start/
Part 1: Build your API
Create a new Project in Visual Studio, and select “ASP.NET Web Application”, (bonus points for also creating an Application Insights to monitor your API)
capture20161027090000222
Choose  ‘Azure API App’, as your template:
capture20161027090001105
I enabled Swagger (http://swagger.io/) so I could test my API before publishing it. In the project go to App_Start, SwaggerConfig class and enable the UI by uncommenting this:

})
.EnableSwaggerUi(c =>
{

The new project will have ValuesController under Controllers folder, rename the controller to OMSAPIController, make sure you change the controller name and the file name too to OMSAPIController.cs.Here is the code for my controller, you’ll need to update 2 values: customerId and sharedKey

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Web.Http;
using System.Security.Cryptography;
using System.Web.Script.Serialization;
using System.Runtime.Serialization;
using System.Text;
namespace OMSAPI.Controllers
{
public class OMSAPIController : ApiController
{
static string customerId = "";


//For shared key use either primary or secondary Connected Sources client authentication key
static string sharedKey = "";

//LogName is name of the event type that is being submitted to Operational Insights
static string LogName = "TwitterLogs";

//Optional field used to specify time stamp from the data. If time field not specified, assumes message ingestion time
static string TimeStampField = "";

public class tweet
{
public string TwitterUser { get; set; }
public string TwitterText { get; set; }
public Sentiment Sentiment { get; set; }
}


public class Sentiment
{
public Document[] documents { get; set; }
public object[] errors { get; set; }
}


public class Document
{
public float score { get; set; }
public string id { get; set; }
}

// POST api/values
public void Post([FromBody]tweet jsonInput)
{

//call function to determine Sentiment based on score
string SentimentName = GetSentiment(jsonInput.Sentiment.documents[0].score);

//build json out of the data for injection in OMS
string json= "{\"TwitterUser\":\"" + jsonInput.TwitterUser + "\",\"TwitterText\": \"" + jsonInput.TwitterText + "\",\"Sentiment\": \"" + SentimentName + "\"}";


var datestring = DateTime.UtcNow.ToString("r");
string stringToHash = "POST\n" + json.Length + "\napplication/json\n" + "x-ms-date:" + datestring + "\n/api/logs";

//build signture for encryption
string hashedString = BuildSignature(stringToHash, sharedKey);
string signature = "SharedKey " + customerId + ":" + hashedString;

PostOMSData(signature, datestring, json);
}

//function to build signature for encryption
public static string BuildSignature(string message, string secret)

{
var encoding = new System.Text.ASCIIEncoding();
byte[] keyByte = Convert.FromBase64String(secret);
byte[] messageBytes = encoding.GetBytes(message);
using (var hmacsha256 = new HMACSHA256(keyByte))
{
byte[] hash = hmacsha256.ComputeHash(messageBytes);
return Convert.ToBase64String(hash);
}
}

//determining sentiment as a string based on float score
public static string GetSentiment(float SentimentScore)
{
string SentimentString;


if (SentimentScore < 0.35) { SentimentString = "Negative"; } else if (SentimentScore > 0.65)
{
SentimentString = "Positive";
}

else
{
SentimentString = "Neutral";
}

return SentimentString;

}

//posting json into OMS using OMS ingestion API
public static void PostOMSData(string signature, string date, string json)
{
string url = "https://" + customerId + ".ods.opinsights.azure.com/api/logs?api-version=2016-04-01";
using (var client = new WebClient())
{
client.Headers.Add(HttpRequestHeader.ContentType, "application/json");
client.Headers.Add("Log-Type", LogName);
client.Headers.Add("Authorization", signature);
client.Headers.Add("x-ms-date", date);
client.Headers.Add("time-generated-field", TimeStampField);
client.UploadString(new Uri(url), "POST", json);
}
}
}

}

Now test your application, click on Run, that will launch your internet browser with your application: Http://localhost:[port]. Add “/swagger” to your URL, it will look like something like this: http://localhost:1486/swagger/
capture20161027111417534
Click on “Expand Operations” and you’ll see your application like this:
capture20161027111443279
The model on the side explains the json format expected, you can copy and paste the text below to test your API, click on “Try it out!”
{
"Sentiment": {
"documents": [
{
"score": 0.99,
"id": "1"
}
],
"errors": []
},
"TwitterText": "Test twitter text",
"TwitterUser": "test user"
}div>

You should get a 204 response code. Also, check in your OMS workspace, that the data arrived.

Once you’re happy with your API, you can publish it to Azure.
Right-click on your solution and publish it!

Next blog post we will put it all together using a Logic App.


Comments (2)

  1. EvMoreno says:

    Great post Rafaela - thanks for sharing

  2. subramani says:

    Great post.. better than keeping templates in .PS1 files!!

Skip to main content