Microsoft has recently put a release of their Code Contracts framework on DevLabs with a commercial license. We're interested on using them in our project (mostly C#, some C++/CLI) to gradually replace all the custom validation code, but I'm keen to know about the experience other people have had with it before we commit to it, specifically:
Do you think the framework is sufficiently mature for large and complex commercial projects?
What problems have you run into while using it?
What benefits have you got from it?
Is it currently more pain than it's worth?
I realise that this is a somewhat subjective question as it requires opinion, but given that this framework is a very significant part of .NET 4.0 and will (potentially) change the way we all write validation code, I hope that this question will be left open to gather experience on the subject to help me make a decision to a specific, answerable question:
Should we be starting to use it next month?
Note that we do not ship a code API, only a web service one, so for the majority of code breaking compatibility in terms of the exception type thrown is not a concern. However, as I'm hoping more people than just me will benefit from this post and its answers, any detail around this area is more than welcome.
The last mature response to this was in 2009, and .NET 4 is out. I figure we're due for an update:
Code Contracts might well be mature enough for your Debug releases.
I realise this is somewhat of an upgrade from “Harmless” to “Mostly Harmless”.
The Code Contracts home page links to quite thorough documentation in PDF format. The documentation outlines usage guidelines in section 5. To summarize, you can pick how brave you feel about the Contract Tools re-writing your IL in your Release builds.
We're using the “don't rewrite my Release IL” mode.
So far, I'm most enjoying this unexpected benefit: there's less code, thus less code to test. All your guard clauses melt away.
if(arg != null) {
throw new ArgumentNullException("arg");
}
// Blank line here insisted upon by StyleCop
becomes:
Contract.Requires(arg != null);
Your functions are shorter. Your intent is clearer. And, you no longer have to write a test named ArgumentShouldNotBeNull just to reach 100% coverage.
So far, I've run into two problems:
I had a unit test which relied on a contract failure to succeed. You might argue the existence of the test was a blunder, but I wanted to document this particular prohibition in the form of a test. The test failed on my build server because I didn't have the tools installed. Solution: install the tools.
We're using two tools that rewrite IL: Code Contracts and PostSharp. They didn't get along too well. PostSharp's 2.0.8.1283 fixed the problem. I'd cautiously evaluate how any two IL-rewriting tools get along, though.
So far, the benefits are outweighing the hazards.
Addressing out-of-date concerns raised in other answers:
- Code Contracts's documentation is quite thorough, though regrettably in PDF.
- There's at least one Code Contract forum hosted by Microsoft.
- Code Contracts Standard Edition is free if you have any VS2010 license.
- .NET 4 is out. I've run into Microsoft's contracts when implementing generic collection interfaces.
I've been playing around with the code contracts some more myself on a small but moderately complex standalone project, which needs to inherit from some BCL classes and use other ones.
The contracts thing seems great when you're working in a completely isolated environment with just your own code and primitive types, but as soon as you start using BCL classes (which until .NET 4.0 do not have their own contracts) the verifier cannot check whether they will violate any of the requires/ensures/invariants and so you get a lot of warnings about potentially unsatisfied constraints.
On the other hand, it does find some invalid or potentially unsatisfied constraints which could be real bugs. But it's very hard to find these because there is so much noise that it's hard to find out which ones you can fix. It's possible to suppress the warnings from the BCL classes by using the assume mechanism, but this is somewhat self-defeating as these classes will have contracts in the future and assumptions will lessen their worth.
So my feeling is that for now, because in 3.5 we're trying to build on a framework that the verifier does not sufficiently understand, that it's probably worth waiting for 4.0.
Judging by this thread I would say it is not quite mature enough to use for an enterprise level project. I haven't used it myself, but people are still running into bugs that would bring your contract-critical project to a halt. It seems like a really great framework and the example videos they've provided have been exciting, but I'd wait for:
- Existence of a community forum. You're going to want to be able to discuss inevitable problems you run into with other developers, and you want to know there is a decently strong base of developers out there to discuss solutions with.
- A successful pilot project release. Generally, when Microsoft Research releases something that they think is mature enough to be used in a commercial project, they will work with an organization to pilot it, and then release that project open source to as a proof of concept and trial-by-fire of all of the major features. This would give a lot of confidence that most of the common contract scenarios are covered and working.
- More complete documentation. Plain and simple, at some point you're going to want to do something with contracts that you can't do yet using Microsoft Code Contracts. You want to be able to quickly and clearly reason that your scenario is not yet supported. The current documentation is going to keep you guessing and trying different things, though, in my opinion, which will result in a lot of wasted time.
It's not mature enough.
It will be as soon as Microsoft releases it with the affordable editions of VS, but without the static code analysis it's not usable at all.
The editions of VS, that have it, are so insanely expensive that only a handful of people will ever be able to afford it.
It's a shame Microsoft killed this amazing idea with their pricing policy. I wish Code Contracts would become mainstream, but they won't.
Epic fail.
