Using #PolyBase in #SQLServer2016

It’s been a few weeks since the numerous Build and Ignite announcements ushered in the latest and greatest, SQL Server 2016. After having some time to soak it up (aka I’ve been too busy to blog) we will dive into some of features and capabilities I find most interesting. Poly-what? While there are many new…

3

Geospatial Queries Using Hive

During one recent engagement, I was helping my customer align ETL activities that were originally developed using SQL Server and T-SQL with the capabilities that were available using Hadoop and Hive. Most of the translations where straight-forward but when asked about geospatial support in Hive I was admittedly stumped. After full disclosure to the customer…


Automating Update of Azure-Powershell

Just a quick post to share a useful script. The PowerShell script below will download and update the Azure-PowerShell command-lets to the latest and greatest version. It even does a slick little version compare. I’ll put the disclaimer out there, that I an not the original author of this script and unfortunately I’ve lost the…

1

Building an Azure ML SSIS Task

In several previous blog posts (HERE and HERE), I’ve introduced and discussed the Azure Machine Learning service, its features, benefits and general capabilities. Since that time I have been toying with the idea of a building a custom SSIS Task to integrate Azure ML into SSIS. My vision of the project is pretty simple and…

2

Ooooh I’m Telling: Doing Swear Word Analysis with Storm on HDInsight

As promised, this is the first of three (maybe more) posts that will present an end-to-end example to showcase the distributed streaming capabilities of the Apache Storm project. This first post will provide an introduction to the project and an overview of all the moving pieces. Please note that I will not be getting into…

3

Introduction to Apache Storm

The Apache Storm project delivers a platform for real-time distributed (complex event) processing across extremely large volume, high velocity data sets. By providing a simple, easy-to-use abstraction, Storm enables real-time analytics, online machine learning and operational/ETL scenarios that have previously been non-trivial to implement. In this post we will familiarize ourselves with the Storm platform, its…

4

Something’s Brewing with Azure Data Factory – Part 3

In the first two parts of this blog series (HERE and HERE), we used Azure Data Factory to load Beer review data from an Azure SQL Database to an Azure Blob Storage account. We then processed that data using HDInsight and the Mahout Machine Learning Library to generate user-based recommendations. In this final post, we…

3

Something’s Brewing with Azure Data Factory Part 2

In my last post (HERE), I started hacking my way through the new Azure Data Factory service to automate my beer recommendation demo. The first post was all about setting up the necessary scaffolding and then building that first pipeline to move data from the Azure SQL Database into Azure Blob Storage. In this post,…


Something’s Brewing with Azure Data Factory

Awhile back I put together a presentation to show off HDInsight using Mahout and most everyone’s favorite Beer. The concept was simple. A spartan website allowed users to create a website and rate beers. The ratings were then fed into Mahout to generate recommendations using Collaborative filtering. Looking back, the process I used was good…


Hello…..Azure Data Factory!

Overview The boundaries between on-premise and cloud-born data continue to blur with more and more organization moving to hybrid data landscapes. The blurring of these lines introduces a number of complexities from data locality, to scale and even the diversity of the data which we must consume and processed. To meet these challenges head-on, Microsoft…