LINQ, collections and null reference exceptions

This short post describes an idiosyncrasy of LINQ that, for someone with several years of C# experience I should probably have known about, but didn’t. During development of an app the test team reported that a null reference exception was intermittently occurring. For a seasoned developer like myself I didn’t think it would be too…


Adding Trace to Azure Web Sites and Web Jobs

Using standard .NET trace in Azure Web Sites and Web Jobs is an extremely simple affair, with trace output being directed to either table, blob or file storage without the need for any configuration file or code changes. This is great and a very welcome simplification compared to what you had with web and worker…


Azure: Calling the Azure Management Rest API

The Azure Management REST API exposes a series of endpoints that allow you to perform various operations against your Azure subscription. This is documented at http://msdn.microsoft.com/en-us/library/windowsazure/ee460799.aspx. You can do many things with the REST API – the majority of Azure management operations can be initiated via this (Note: the Azure PowerShell commandlets wrap calls to…


Execution Time Based Heuristic Custom Task Scheduler

If you follow the samples for Parallel Programming with the .Net Framework, you may have come across the ParallelExtensionsExtras and the Additional TaskSchedulers. Although these samples cover a broad set of requirements I recently came across another that could be satisfied with the creation of a new custom task scheduler. In the current samples there…


Co-occurrence Approach to an Item Based Recommender

For a while I thought I would tackle the problem of creating an item-based recommender. Firstly I will start with a local variant before moving onto a MapReduce version. The current version of the code can be found at: http://code.msdn.microsoft.com/Co-occurrence-Approach-to-57027db7 The approach taken for the item-based recommender will be to define a co-occurrence matrix based…


.Net Implementation of a Priority Queue (aka Heap)

I thought I would take a break for a while from Hadoop and put together an F# .Net implementation of a Priority Queue; implemented using a heap data structure. Conceptually we can think of a heap as a balanced binary tree. The tree will have a root, and each node can have up to two…


Generic based Framework for .Net Hadoop MapReduce Job Submission

Over the past month I have been working on a framework to allow composition and submission of MapReduce jobs using .Net. I have put together two previous blog posts on this, so rather than put together a third on the latest change I thought I would create a final composite post. To understand why lets…


Framework for Composing and Submitting .Net Hadoop MapReduce Jobs

An updated version of this post can be found at: http://blogs.msdn.com/b/carlnol/archive/2012/04/29/generic-based-framework-for-net-hadoop-mapreduce-job-submission.aspx If you have been following my blog you will see that I have been putting together samples for writing .Net Hadoop MapReduce jobs; using Hadoop Streaming. However one thing that became apparent is that the samples could be reconstructed in a composable framework to…


Stream join aggregates using SQL Server Stream Insight

Lately I have been working with StreamInsight and thought it was about time to share some of my observations and key points. This is loosely based upon a requirement to aggregate Financial risk vectors and deliver changes to clients in an event based manner. When working through this I was primarily interested in the following…


Refactor boilerplate code with Actions and Funcs

I have seen quite a few projects where the code starts with the best of intentions in terms of readability, but soon setup and boilerplate code creep in and it becomes hard to focus on where the setup and clean up ends and the functionality begins.  The other downside being repetition of this boilerplate code…