Performance improvements in Infer.NET 2.4

This blog has been migrated from community.research.microsoft.com

Original date of blog: November 8, 2010

Original blog author: John Winn

Hello Infernauts!!

Now that version 2.4 of Infer.NET is released, we're planning a series of blog posts to describe the new features and capabilities that we've added over the last 12 months.    The main focus of this version is performance, so I'm going to start off the series by showing how we've made improvements in the speed of running inference for a range of models.  For those of you already using Infer.NET, the nice aspect of these changes is that your existing models will just run faster and use less memory without you having to do a thing!

Significant speed ups

I've picked two common models to compare on:

• Mixture of Gaussians - thisis themultivariate mixture of Gaussians model from this tutorial, with the number of points increased to 30,000. The inference algorithm is Variational Message Passing which is run for 50 iterations.
• Bayes Point Machine - a customised Bayes Point Machine model, similar to the one in this tutorial but with a large feature vector and several thousand data points, which I'm using for one of my own projects. The inference algorithm is Expectation Propagation, run for 25 iterations. To really maximise performance, I'm using a faster 'division' form of one of the EP operators, which can be switched on using the newly-introduced ability to override the operator search path - just add engine.Compiler.GivePriorityTo(typeof(Replicate2BufferOp)) . Note that this operator doesn't work in certain special cases, which is why it's not set as the default for EP. If you plan on using it, make sure it gives the same results with and without this line, for your model.

The results are shown in the table below. For both versions I am using the Release DLLs, so as to get the best performance out of each version of the framework. They show the total time for inference (not compilation) as given by the ShowTimings option on the inference engine.

As you can see, the changes we have made in version 2.4 provide significant speed ups for several useful models when running VMP or EP. These speed ups are mainly due to improvements in the compiler which means that the generated code is much more efficient. For example, we now check to see if more than one message in the message passing algorithm is being computed in the same way and, if so, remove the redundant computation and the redundant message. This optimisation means that, as well as an increase in speed, there are also significant reductions in memory usage.

Performance of Gibbs Sampling

Let's now look at Gibbs Sampling, which we have supported as an inference algorithm since Infer.NET 2.3.  As a model, I'll use the BUGS Rats model - the 'Rats' normal hierarchical model from the examples that come with the WinBUGS  Gibbs Sampling software. I ran inference on this model in both versions of Infer.NET and in WinBUGS - using 20,000 iterations of Gibbs Sampling in each case.  For a further comparison, I also ran exactly the same model using 50 iterations of VMP.

These results show that there's only a small improvement in speed in version 2.4. However, the solution is still more than twice as fast as WinBUGS! What's even more interesting is that, by using Infer.NET's ability to switch inference algorithms on the same model, we can use VMP and run in about 70 milliseconds i.e. about 80 times faster. 

Other performance improvements

In addition to improvements to the generated code, we have added support for sparse vectors which allow much more memory-efficient inference in models with large discrete or Dirichlet distributions as described in this user guide page. We'll be looking at using this new sparse support in a blog post later in this series.

Also on the topic of performance, we have upgraded the support for parallel inference described in this blog post to use the .NET 4.0 Parallel class.  Parallel inference is still an experimental feature but you may well get improved performance for your model if you are running on a multi-core machine.

So that's it - the same model code now runs faster.  Enjoy the new, more streamlined, Infer.NET!

John W. and the Infer.NET team