Architecting a Big Data Project in Azure

The aim of this post is to give a very brief overview of the technologies in Azure that can comprise a Big Data architecture and to help the reader understand the options out there. With the new data explosion over the last few years new methods and technologies have been developed to deal with data that…


SQL Checkpoint Scaling Observations

A recap on what a checkpoint does Essentially a checkpoint operation writes dirty pages to disk. This speeds up any crash recovery. If a checkpoint did not happen then all transactions since the last persisted state of the database would need to be applied to pages in memory. With a checkpoint a much smaller portion of the…


An Introduction to U-SQL in Azure Data Lake

Azure Data Lake Analytics is a Big Data analytics service. It differs from HDInsight principally in that you do not need to spin up a cluster to start submitting jobs. Essentially it runs as a service that is ready and waiting to process jobs. While in HDInsight you might use Pig or Hive to query or transform…