Data Science in a Box using IPython: Installing IPython notebook (2/4)

In the previous blog, we demonstrated how to create a Windows Azure Linux VM in detail. We will continue the installation process for the IPython notebook and related packages. Python 2.7 or 3.3 One of the discussions that happened at the Python in Finance conference is which version of Python you should use?  My personally…

1

Enter the Big Data Matrix: analyzing meanings and relations of everything (2/2)

Running the Python example step by step: We explained the basic idea behind LSA or latent semantic analysis in the first part of this blog. We built a matrix by word counting for each document.  The set of document vectors are then sorted by words they appear in. Then we applied SVD (single value decomposition)…

0

Let there be Windows Azure HDInsight

Windows Azure HDInsight Service, formerly known as Hadoop on Windows Azure,  is now available inside the Windows Azure Preview portal.  Hadoop-based big data tools are what I call the WMD(P), or Weapons of Mass Data Processing. (You heard it here first!)  This is a very exciting development, and I would like to take a moment to recognize the great work our HDInsight…

0

Running Weather Research Forecast as a Service on Windows Azure

        About 9 months ago, I deployed a Weather forecast demo at an internal Microsoft event, Techfest. The demo uses real data from NOAA and predicts high resolution weather forecast up to the next 3 days running a HPC modeling code called WRF.  Since then, I’ve received a great deal of interest from…

0

Hadoop On WindowsAzure Updated

HadoopOnAzure allows a user to run Hadoop on Microsoft Windows Azure as a service.  It is currently in private CTP with limited capacity, and by invitation only.  We did add more capacity today, you may attempt to sign up this free service at https://connect.microsoft.com/SQLServer/Survey/Survey.aspx?SurveyID=13697  Please allow one week for us to approve your invitation. To get…

0

Explicit and Implicit

This is the final post of the simple HPC Math series I am composing. The previous two posts were on linear, and nonlinear solvers.  Just like nonlinear solvers depend on linear solvers, time stepper solvers will depend on the previous two building blocks.  Explicit and Implicit methods are used in numerical simulations to solve a…

2

5 min Intro to nonlinear solvers

Nonlinear solvers are essentially solving a system of nonlinear equations.  X^2 + x -1 =0 is a nonlinear equation.  As we recall from Calculus 101, to solve these equations, one has to use some variants of Newton’s method because it has fast convergence to the solution property.  For systems of nonlinear equations, Newton’s method requires…

2

Math under the hood

We recently had a company called Simulia visiting us at a deep dive lab.  Their product, Abaqus, a commercial software package for finite element analysis prompted me to quickly brush up on my numerical computation knowledge.  After all, you can’t compute without numerical methods under the hood!  While Abaqus specializes in nonlinear solvers, I figured…

2

Problogue

Super computing is crucial for science in the 21st century.  For a long time people have been theorists or experimentalists, those have worked in synergy to develop scientific theory and explain natural phenomenon.  The incredible development of super computer in recent years has enabled theory validation, and new kinds of experiments that were not possible…

3