Saturday, November 11, 2006

The Power of Unit Testing, and Microsoft's Big Snafu

Automated unit testing has got to be one of the most powerful ways to produce rock-solid code. That's not to say that code-reviews and full testing by a dedicated quality assurance team don't have value, but automated unit testing has the virtue of being fast, easy-to-execute, preemptive, and (assuming the tests were properly written) extremely accurate.

The idea is simple. Before you write any code, you write a test that breaks. Then, you massage the code until the test passes. This makes you focus on the problem domain, and keeps you from getting side-tracked by fancy fluffy stuff that you might add and never actually use.

Now here's where the power comes from: the automated tests are run every time the software is built. These tests don't tell you if the software is syntactically unsound (that's the compiler's job); instead, they tell you whether or not the software is doing what it's supposed to do, and it's telling you this before the testers get their hands on it. The fact that it happens every time the software is built should give you a tremendous feeling of confidence: you'll know immediately if anyone introduced any code changes that broke your business logic, so you can be proactive about correcting them, instead of waiting for the testers to find it and rub your nose in it.

However, manually creating your own testing framework, and religiously executing the unit tests themselves with every build can be daunting. That's why projects like JUnit and NUnit are wildly popular: they simplify the process of creating and executing automated unit tests. The idea is so deeply entrenched in our development methodology these days that it has a name: Test-Driven Development (TDD). Kent Beck wrote what is arguably the definitive introductory work on the idea, the aptly named Test-Driven Development. (You can find that book on Amazon here. But if you don't want to read the whole book—even though it's a really good idea—you can take the crash course here.)

So now that I've made it clear that I'm a huge proponent of test-driven development, and of automated unit testing specifically, it's time we turned our steely gaze to Microsoft.

As many well know, Microsoft created its own unit testing framework. From what I've seen, it's pretty exhaustive. (Of course, like any Microsoft version of an existing "standard," their framework deviates pretty substantially from the NUnit and JUnit frameworks. So you can't just plop your existing test frameworks into Visual Studio 2005 and run them. No, you have to convert them to Microsoft's way of thinking.

How very Gatesian.

But that's not my main peeve. My main peeve is this: you can only access the unit testing capabilities of Visual Studio 2005 if you own Team System. Furthermore, the express editions of Visual Studio do not provide add-in support, so that tools like NUnit can't be snapped into the IDE, except as external tools.

Here's what I don't understand: Microsoft is all about security these days. Part of security is writing secure, stable software. Why on earth would you deny developers access to a technology that could help them to write stable, secure software? Furthermore, if you're releasing a free version of your IDE to the masses, why would you think it's a good idea not to support add-ins, so that new developers can't take advantage of things like NUnit?

And please don't tell me that you just expect everyone to upgrade to Team System. If you want them to do that, price your products reasonably. The average CIO will just about have a heart attack when he sees the estimate to upgrade from 2003 to Team Suite. I wouldn't be surprised if most companies can't afford it.

We can sit here and talk about how everything comes down to long-term cost versus short-term costs until we're blue in the face. Hey, I've read Steve McConnell, I know the facts. But the plain truth is, when you have someone sitting in the finance department who doesn't really get it, and who doesn't care to get it, because he's got a "bigger picture" than you do, your upgrade costs are going to get nixed in a heartbeat. You'll get the Professional Edition, or something close to it, because it's at least reasonably priced and does just about everything you need. I'm willing to bet that more companies are run that way than aren't, and a lot of poor developers are working for them.

So, as I see it, what Microsoft has done is shoot itself in the foot over unit testing. They're not fostering the growth of a corps of developers who write good, stable, well-tested code. They've placed the cost of unit testing using Microsoft's preferred framework so far out of reach that most companies will never embrace it. Those that do embrace automated unit testing will look for open-source third party products like NUnit. They'll find ways to make it work. If Microsoft was trying to squeeze that competition out, they certainly went about it the wrong way.

I would certainly like to sit down and talk to the brainiac that decided that unit testing wasn't going to be made available except for the elite few who could afford Team Suite. I'd also like to find the guy who decided that Add-In support was a no-go for the Express Editions, and confiscate his Geek Mafia membership card.

These two decisions together have pretty much guaranteed that most new users will find it too hard to learn how to unit test from the beginning, because Microsoft went out of its way to make it difficult.

When Microsoft had the opportunity to put the tools into the hands of new developers (many of whom couldn't afford pricey development tools) to make them better, more effective programmers, they opted instead to take those tools away, to keep them for the rich, and take away their ability to use 3rd party components in the IDE as well.

"Do it my way, at my price, or not at all."

Shame on you, Microsoft. Shame on you.

No comments: