Data Science in a Box using IPython: Installing IPython notebook (2/4)

In the previous blog, we demonstrated how to create a Windows Azure Linux VM in detail. We will continue the installation process for the IPython notebook and related packages. Python 2.7 or 3.3 One of the discussions that happened at the Python in Finance conference is which version of Python you should use?  My personally…

1

Enter the Big Data Matrix: analyzing meanings and relations of everything (2/2)

Running the Python example step by step: We explained the basic idea behind LSA or latent semantic analysis in the first part of this blog. We built a matrix by word counting for each document.  The set of document vectors are then sorted by words they appear in. Then we applied SVD (single value decomposition)…

0

Enter the Big Data Matrix: analyzing meanings and relations of everything (1/2)

Data Science is compute and labor intensive In the previous blogs, we showed you how to find a dataset, clean it and run simple mapReduce, sort on the dataset.  It was meant to give you a flavor of what data science is all about, and I also wanted to expose Big Data’s rather labor intensive…

0

New Breakthrough in Big Data Technologies: the NullSQL Paradigm shift

  Mammoth the NullSQL tool Most of us by now understand the properties of big data.  Many of us are already working with big data tools, or NoSQL tools such as Hadoop.  I’ve spent a bit of my spare time in the last 2 months working on prototypes of a new set of tools that can help the…

1

Running Weather Research Forecast as a Service on Windows Azure

        About 9 months ago, I deployed a Weather forecast demo at an internal Microsoft event, Techfest. The demo uses real data from NOAA and predicts high resolution weather forecast up to the next 3 days running a HPC modeling code called WRF.  Since then, I’ve received a great deal of interest from…

0