How I became the “Grid guy”


Hi,

In one of my last posts, I quickly explained I worked in the
SharePoint Engineering team for few months. The experience was amazing, though
pretty challenging. I found out a lot of things on a lot of topics, but I loved
meeting smart, dedicated and great people.

Some people wonder what I was doing there.

 

Here are some clues in
this post…

 

You may did read this post few months ago from Jeff Teper
(SharePoint Corp VP) http://blogs.msdn.com/b/sharepoint/archive/2009/10/11/engineering-sharepoint.aspx
. It was a pre-SharePoint conference post where some of the engineering
organization details where explained.

In this post, you may notice this section:

“Grid
Team – In the last couple of posts I mentioned that SharePoint Online is big
part of our development cycle. […]. This release, we created a dedicated team
called “The Grid” team to analyze and optimize in great detail the total costs,
time, reliability and other factors of a SharePoint deployment we hope to reach
10s and eventually 100s of millions of users. Almost all of what we learn there
helps make the product more reliable for customers and partners hosting their
own servers from a small company to a 250,000 employee enterprise. There is a
bunch of great learning from things like virtualization with Hyper-V to best
practice analysis to operations monitoring we will share in the coming months.
The Grid team will be at the SharePoint Conference.”

 

For those of you who attended the SharePoint Conference, you
didn’t see a “Grid” session or a “Grid team” at asks the experts. In fact they
were here, but with their “classic” SharePoint titles.

In the excerpt, you read “we will share in the coming months”…..

 

Well, here you go: that’s partly what I was requested to do.

  • Why me?
    • Because of my work on SharePoint Automation
      (White Paper, TechReady internal sessions, automation tools and scripts, blog
      post series on SharePoint developer workstation automation).
  • Why
    SharePoint Automation is important?

    • Because it’s the key to reliable SharePoint
      platforms at any scale.
  • What does
    it mean?

    • It means SharePoint Engineering is taking the
      effort to develop SharePoint Online (Standard). In fact that’s the directions
      from “we’re all in” for Microsoft: Products need to be built for On-premises
      AND Online deployments…. And in between (mixed, hybrid, whatever you want to
      name it).
  • What was
    done?

    • With one of my colleague, Senior PFE from MSFT
      US, we interned the Grid team (I mean, being part of the team for months) to do
      various things:

      • Understand how Grid team plan to do SharePoint
        Online (I say plan because the General Availability is beginning of 2011 now –
        so it’s not here yet)
      • Abstract the concepts and tools used to fuel
        SharePoint Online 2010
      • Start internal evangelism on these concepts, in
        conjunction with the Dynamic Data Center team
      • Work on mapping these concepts and tools in
        Microsoft SKUs
      • Build awareness in SharePoint internal community
        on this “way” to fuel an Online Service based on SharePoint

So, step by step, that’s how I
became internally the “Grid guy”. With my PFE colleague we led few webcasts and
technical internal sessions/events to start the messaging about SharePoint
Grid. Being passionate about those topics of Automation and Orchestration,
people came to me on these aspects.

We also presented these concepts
to various internal stakeholders, and few customers interested in hosting their
own “Private Clouds” with SharePoint being part of it.

To be clear, “Grid” is a
SharePoint Engineering set of concepts, tools and processes. They own this
thing, and are quite amazing on how they do that. But out of the Building 16, I
became the “Grid guy” – its “public face” like the Grid PM lead said once – of
this work. Just because I was asked to understand, and explain outside of the
building how it’s done.

To test my understanding, I
reproduced SharePoint Grid on servers in France. That was pretty good
experience to do that. I needed to check what it was and how it worked. That’s
the way I am, I like to know what I talk about, up to be able to do it myself
if needed J.

  • What was
    delivered?

    • Internally, I delivered a lot of documentation,
      workflows, mapping, Technical References content plus discussions with different
      Product Groups. It’s all available -internally. With the rest of the involved
      team, we also developed an end-to-end approach to think workload first, then
      sourcing options depending on the use cases.
    • The team received the “CTO collaboration award”
      for this.
    • With one of the initiative I worked on (called
      “Image Factory” – to come in another post), I received the “Technical
      Excellence recognition” from the IW community (kind of internal MVP thing).

 

  • Now what?
    • Grid team has been reshuffled with Office 15
      reorganization. The small and highly motivated team that was the Grid one became
      SharePoint Core Data Center team. They now have much more resources to ship the
      product Online. Things change. What MSFT learned from this Internship is that
      the Learning curve really starts after the GA, when customers onboard…. So,
      that was too early to “share in the
      coming months
      ” how SharePoint Online is done. We’ll probably do that after
      the GA.

 

  • What SharePoint
    Grid means?

    • GRID itself means nothing. It’s not the acronym
      of something.
    • The project needed a name and the SharePoint
      Cloud for MSFT is SharePoint Online. , so another name was needed for its
      underlying infrastructure.
    • But I can share something pretty simple and
      clear on what SharePoint Grid is:

      • Imagine numerous Hyper-V hosts, with the exact
        same hardware. We call it a “compute resource pool”.
      • Then implement Virtual Machines which are “Roles
        based”: in the case of SharePoint, we have 3 obvious roles: SQL server,
        Application Server, WFE servers.
      • Of course, you can add more roles and diversity,
        like dedicating servers for Sandboxed solutions or Office Web Applications.
      • What do you end up with?

A
“SharePoint Grid” you can represent like this:

 

Host 1

Host 2

Host 3

Host 4

Host 5

Host 6

=> etc. =>

VM1

WFE

APP

APP

WFE

SQL

WFE

VM2

WFE

APP

APP

WFE

SQL

APP

VM3

SQL

APP

WFE

WFE

APP

SQL

VM4

SQL

WFE

SQL

SQL

WFE

WFE

VM5

SQL

SQL

SQL

SQL

APP

APP

VM6

SQL

WFE

WFE

APP

WFE

SQL

 

Nice, isn’t it? And that’s just for 6 Hyper-V hosts, hosting
each 6 VMs….. But it’s already 48 Windows Servers to operate and manage.

How can you do that? ….. Automation with new concepts is the
only way J
and that’s my passion!

 

I’ll share few useful insights and background in the weeks
to come. First one to start:

  • The Building 16 lab used initially to develop
    Grid, had to manage and pilot more than 900 servers (physical and virtual)…..
    1/3 of them where rebuilt automatically every week!

< Emmanuel />

Comments (0)