Using #PolyBase in #SQLServer2016

It’s been a few weeks since the numerous Build and Ignite announcements ushered in the latest and greatest, SQL Server 2016. After having some time to soak it up (aka I’ve been too busy to blog) we will dive into some of features and capabilities I find most interesting. Poly-what? While there are many new…

3

Geospatial Queries Using Hive

During one recent engagement, I was helping my customer align ETL activities that were originally developed using SQL Server and T-SQL with the capabilities that were available using Hadoop and Hive. Most of the translations where straight-forward but when asked about geospatial support in Hive I was admittedly stumped. After full disclosure to the customer…

0

Automating Update of Azure-Powershell

Just a quick post to share a useful script. The PowerShell script below will download and update the Azure-PowerShell command-lets to the latest and greatest version. It even does a slick little version compare. I’ll put the disclaimer out there, that I an not the original author of this script and unfortunately I’ve lost the…

1

Building an Azure ML SSIS Task

In several previous blog posts (HERE and HERE), I’ve introduced and discussed the Azure Machine Learning service, its features, benefits and general capabilities. Since that time I have been toying with the idea of a building a custom SSIS Task to integrate Azure ML into SSIS. My vision of the project is pretty simple and…

2

Introduction to Apache Storm

The Apache Storm project delivers a platform for real-time distributed (complex event) processing across extremely large volume, high velocity data sets. By providing a simple, easy-to-use abstraction, Storm enables real-time analytics, online machine learning and operational/ETL scenarios that have previously been non-trivial to implement. In this post we will familiarize ourselves with the Storm platform, its…

4

Something’s Brewing with Azure Data Factory – Part 3

In the first two parts of this blog series (HERE and HERE), we used Azure Data Factory to load Beer review data from an Azure SQL Database to an Azure Blob Storage account. We then processed that data using HDInsight and the Mahout Machine Learning Library to generate user-based recommendations. In this final post, we…

3

Something’s Brewing with Azure Data Factory

Awhile back I put together a presentation to show off HDInsight using Mahout and most everyone’s favorite Beer. The concept was simple. A spartan website allowed users to create a website and rate beers. The ratings were then fed into Mahout to generate recommendations using Collaborative filtering. Looking back, the process I used was good…

0

Hello…..Azure Data Factory!

Overview The boundaries between on-premise and cloud-born data continue to blur with more and more organization moving to hybrid data landscapes. The blurring of these lines introduces a number of complexities from data locality, to scale and even the diversity of the data which we must consume and processed. To meet these challenges head-on, Microsoft…

0

Building Custom SSIS Azure Service Bus Components

Over the last couple of weeks, I have been experimenting and working on a set of custom SSIS Service Bus components that will ultimately be the basis of the ETL framework described in my previous post (HERE). While these components are being developed as a proof of concept to test out my theories on they…

0