MS Development Process

I just did a reboot of my blog and unpublished some posts over the past month, although links are still valid. I have been following a thread too long, annoying readers, and didn’t particularly have much to say.

Last night, I went to a development talk given by Scott Guthrie, founder and product unit manager of the ASP.NET and VS Web Tool teams at Microsoft for the DotNet user group. Unfortunately, I was a bit late and didn’t have pen and paper in hand, so much of this is from memory.

He provided a behind the scenes walkthrough of how software is built at Microsoft, show the real schedule, design docs, code, test plan and cases which were used and how they were built.

The group is currently in MQ, which is a milestone after shipping, in which the group makes long desired but somewhat disruptive process improvements. Two such changes are:

• Replacing current build scripts with MSBuild.
• Moving to Visual Studio Team System-based products such as source control.

The movement to Visual Studio Team System is part of Microsoft’s dogfood philosophy and is part of a long term direction of replacing internal tools with Microsoft’s own commercial quality versions. Actually, in some cases, the internal tools themselves such as Product Studio, a bug database, and various static analysis tools were productized into Team System. So far, the only real team that has relied VSTS has been the VSTS Team itself.

Currently, product specs are being written for the next set of features for the upcoming Orcas. Scott showed an example of GridView spec, which is about 76 pages long, and kept updated throughout the product development cycle. The spec serves as a communication piece between PMs, developers, testers, localizers and documentation, and allows new team members, especially developers, to come late into the project and “load-balance” any remaining work and bugfixing. It also keeps product details outside the mind of the developer and protects against employee turnover. Specs have to be written somewhat comprehensively before development even begins and undergoes several reviews. They include a change history.

ASP.NET v2.0 is about a million lines of code and is a small project subtree within a single massive Visual Studio source tree that currently requires nine hours to build under powerful machines. Developers can build the whole tree with one command “build -c”, although normally just build their own project subtree to minimize time. Synchronizing source code is an extremely lengthy process and the VS team currently use high-speed terabyte network channels to handle the traffic.

Scott describe a complicated, four-level hierarchy of Virtual Build Labs (VBLs), all of this information has been detailed in the web (google VBL with pages restricted to blogs.msdn.com). Changes are propagated down the tree every day, but local changes are moved upward more slowly—in terms of weeks. All members of the VS division reside in about a handful of adjacent buildings for close collaboration.

Feature development occurs in milestones which span about six weeks. Developers spend about one-fifth of their time in actual development, and the remaining time building unit tests, testing or fixing bugs. Documentation and strings are kept in separate files. We were given a look into the source code of the Button class. Code contained very few comments except for references to bug id and descriptions. All code must pass a build verification test and be code-reviewed before checking in. One month of the every project at Microsoft is spent undergoing a security review, in which each line of code is checked by multiple eyes. A separate group of testers, 1.5 for every 2 developers and all of whom have a development background, build automation test suites contain hundreds of thousands of test cases.

At this point, my memory is fading. Scott went over terms like ZBB and ZBR, MSDN Feedback center, and the triage process for bugs as the product moves closer to shipping. Customers are often puzzled why legitimate bugs posted to the Feedback Center aren’t fixed closed to the ship date, and it’s because the development cycle has moved to a phase in which it must minimize code churn to reduce the likelihood of regressions.

Comments

[Matt]

Very informative posting. I have a few questions: http://mdavey.wordpress.com/2005/12/08/microsofts-development-process/

[Oskar Austegard]

Glad to see the focus of your excellent blog has shifted back to what (most) people come here to read.

Good move.

[Aaron Brethorst]

Eric talks more about the MQ milestone we're currently in on his blog: http://blogs.msdn.com/eric/archive/2005/11/04/489108.aspx

Brian Harry talks about the process by which we move changed source code up and down between VBLs. These are called Reverse Integrations and Forward Integrations, or RIs and FIs for short: http://blogs.msdn.com/bharry/archive/2005/11/12/492198.aspx

ZBB is Zero Bug Bounce, and it is well-described on the VS Release Team's blog: http://blogs.msdn.com/release_team/archive/2004/08/16/215139.aspx

ZRB is Zero Resolved Bugs, and it is well-described by Chris Rathjen: http://blogs.msdn.com/crathjen/archive/2005/11/08/490458.aspx

[Sunny]

Thanks for a good blog entry. This is what I like to read.

[sunny]

"ASP.NET v2.0 is about a million lines of code and is a small project subtree within a single massive Visual Studio source tree that currently requires nine hours to build "
- this is an ambiguous information. What takes 9 hours to build? ASP.NET or whole VS?

[Wesner Moise]

All of VS.NET takes 9 hours to build...