We're Back

Apologies', I've been somewhat away from the blogsphere for some time, and been meaning to get back into it. Something like 'writer's block' has prevented that, so I am hoping to will that away presently. This first one 'back' has been very hard to frame, since there has been much to talk about. (I have the willingness, just not the capacity to capture it all).

In the last few months there have been several things to be made sense of in this space. It's been a period of time where we have had to broaden the view of where we are going with factories and how exactly we are going to get there. Factories has captured many differing points of views in the industry, some are in support or empathetic from similar initiatives, and some are criticism from the confusion and overloading of the term factories have stirred up recently. However almost all looked poised to see what becomes of it from Microsoft.

What's New?

The most significant event recently from a Microsoft perspective, is that Jack Greenfield has now moved out of the darkness of the product unit and into the light of the Architecture Strategy Group. Now more visible and more closely connected with our customers, the plan is to formalize and publish software factories methodology in a way that makes them more accessible to the general development marketplace, so our customers can standardize on the methodology process and practices. Jack outlines that charter in the inaugural post of the new team blog 'Software Factories 2.0'

The other big news here is that we have just created a new team blog for the Software Factories Initiative, where the team will be posting information and materials about the initiative as we move forward.

We are also undergoing at Microsoft what many large companies struggle to do themselves with internal factory adoption by the business. By no means an easy feat as you can imagine with ever tighter budgets, and the paradigm change that factories call for. Fortunately though, like many large product companies, most of the businesses internally are seeking ways of improving product adoption and improving productivity with our products and solutions. So at the very least, the various businesses recognize the need for a paradigm shift towards reuse, modeling and automation. But getting the message through about factories has been tough. Hey it's Microsoft!

We are also starting to see some very significant investments in software factories methodology and process by our customers. Several leaders in various software markets are recognizing shrinking marketplaces for their hand crafted products using current manual methods. With so much knowledge and expertise in their respective business domains, it seems that for the market leaders the pain of the software crisis is too acute nowadays to ignore anymore. And for those creating repeated solutions, they are turning to software factories methodology to provide a warranted competitive advantage for them.

Where are we at today?

Time for some retrospection. The latest release of the WSSF Modeling Edition (a few months back) initially exploded. Downloads of it are now well over 40,000 or so, as reported by a few in the team. It seems to be the de facto software factory and domain example everyone refers to when evangelizing, demonstrating customization of existing factories. Being p&p's first and flagship factory, WSSF addresses probably the easiest, most topical domain today that illustrates the value of factories for a common technology domain. With the boom of SOA and the S+S (software plus services) strategy emerging from the company, ironically the success of the WSSF has quite rightly been somewhat overshadowed by new business opportunities for services, and as such it is on the verge of being taken for granted as the de-facto designer tool for implementing services in this space. Which is huge news for all of us, and testament to its value and ability to lower the bar significantly for those quickly building compliant/secure web services to the point its ubiquitous for building services. This has been a good initial start for demonstrating at least one of the values of factories.

Nonetheless, the current generation of Microsoft software factories (from p&p), like it or not, have not been really compelling enough yet to drive factory building adoption as fast as many of us would have liked to see. The end-game we've been moving towards has always been to build and use factories that target more specific (vertical) application domains, but beyond the few horizontal technology factories we can point to today, what is there to compel an organization to adopt factory methodology and tools and start building these vertical factories themselves?

The development savvy practitioners among you would probably say "...give us the Tools, product Support, Process and Methodology! ". Which is kind of cart before the horse in this case, but clearly we first need a well defined methodology to adopt, followed up by industry acceptance, and then supported by a toolset integrated with the current development processes and tools. But as developers consuming this stuff, it means very little to our day-to-day lives until we have reached the point where we can build and consume factories in our supported development environments as part of our agile processes. What we really need to see for broad developer adoption is: File -> New -> Software Factory, and have our tools supporting a standardized factories methodology.

As Microsoft (outward facing), at present, we don't seem to have much else right now to point to (apart from the four p&p factories) to compel our customers towards this goal, that demonstrate better capabilities and more compelling value of factories automation beyond DSL's, GAX Recipes and basic IDE extensibility.

And I have to say that is, and has been for a long time, very frustrating. Particularly for those customers following this closely who do see the larger potential of factories. Yet for them, they still have the tough organizational issues of conveying that to their businesses by proving viable return on investment, without being able to demonstrate most of that potential value up-front. What software business easily swallows the development investment pill: "It's going to cost us more time and money to build this factory, and then we expect to sell its products much later but much cheaper"?

There's more compelling capabilities than what we see in factories today. For starters, a factory schema to relate product architecture to separate views or aspects (with accompanying configurations of tools in the IDE). A meta- model of the product architecture you can work with in multiple views. Guidance automation that for each part of the architecture tells you what to do to complete it. What tasks you can and can't do right now and why, and what you have done, where tasks are linked to the right tools to perform the actions the guidance prescribes. Support for team development, so that multiple developers can use the same factory at the same time and build the product collaboratively, whilst metrics can be reported on the progress of the product using the factory.

So, even with what we have today, why is the value we see in factories today, whilst a great step forward, still not quite enough to tip organizations over to wide adoption of software factories as a business strategy moving forward?

I'd say the reason is because they (the businesses) haven't witnessed enough success with factories that are relevant to their businesses yet. What do I mean by this?

Verticalization is where it is at!

If you look at the organizations that have embraced software factories based upon the factories released by p&p.

[I am referring to the ones who take the factories and build products for their customers or internal projects. Not the ones who use it on a single one-off project and move on].

Then you will see that in almost all cases, they'll want to modify the factory in some way and tweak it to build multiple solutions that are slightly different in some but unique way for their business, but similar for each of their customers. In short, they want to customize the factory towards a slightly narrower (vertical) domain than the broad horizontal domains the factories were built for. Called 'Verticalization' of a factory, you would do this in the cases where you want to deliver solutions for a more specific business need than the one addressed by an existing factory.

Say for example, you wanted to build financial web services for your customers. You would customize the WSSF to abstract the details of the standardized data contracts, operations and security profiles prescribed in the financial data exchange world. Build higher level models that simplify the selection and configuration of those standardized choices. You would have a very specific factory for that vertical domain.

The other option you have is of course is just to build your own factory (models and processes and automation) and perhaps borrow the reusable assets (e.g. libraries, frameworks, code templates etc) from existing horizontal domain factories combined and composed into yours. In either case though, the bar to doing this with the current factories technologies, uncertain processes and rudimentary authoring tools is too high for most organizations to invest it. Indeed there are some that try and some that do succeed here. However, I am certain that these wins are not visible to most of the potential factory building market at present.

The fact that building factories to vertical domains is very hard technically today, and perceived to be 'expensive' to a business right now is what is holding broad adoption back. But you can't have more benefit and value for less money.

If we can make building your own vertical factory easier to do technically, with the development processes you use today, then its pretty hard for a business to ignore the revenue opportunity of selling the products created from them at a fraction of the cost of hand crafting them today. And furthermore ignoring the new business opportunities that would arise from this new delivery capability.

But where do you see examples of this today that would give you the confidence to adopt this approach? I'd say you don't for a couple of good reasons right now:

• It's hard to deliver this capability (to my earlier point about technical difficulty, paradigm change etc.).
• It's even harder to convince your organization to invest in factories as a a strategy, because it requires profound appreciation of the looming software crisis, understanding of the gains to be made with this methodology, and a firm financial and cultural commitment to make a change in development paradigm.
• An organization that has adopted this approach, and executed it successfully, is not going to advertise it to their competitors.

...there is a reason why its a business strategy.

So clearly at this stage in the game of adoption we (Microsoft) need to enable customizing and building factories to be much easier. So more people can do it, and demonstrate more wins publicly. We need to explain and demonstrate how organizations can take actionable steps towards adopting this as a strategy. And to the last point raised by competitive advantage, probably in these early stages won't change much until the factories approach is adopted across all major players in specific solution marketplaces. Only then might the competitors in that marketplace come out and say, "its no big secret, everyone in this space is doing it using factories anyway now, so what else's new?"

A Methodology, Processes and Tools

The Microsoft Software Factories Initiative recognizes these challenges, and we are on point to deliver these capabilities. First we are planning to deliver a methodology and specification for defining and executing software factories processes. That should enable other market leaders to adopt the approach and build their factories. We are also working with a few early adopting customers and community to implement that strategy.

Once the methodology is more broadly adopted and accepted, I could see us Microsoft (and the development communities) producing the development tools and environments to the spec to author factories, and have the necessary runtimes for them to execute in. I can't say we have all those pieces completely figured out yet, considering that no doubt as we go along other new techniques and applications of factories will evolve.

So what will the factories look like that come from that initiative? How will they differ from what we see today? We touched on that above, but the answer is: dramatically different.

I promised to post a bit more now, so I am leaving that discussion for the next post.