I’ve had the privilege of working with many developers over the years on a diversity of projects. I like tests, i think they make the software development process more reliable and help keep code from being unmaintainable disasters. So i find it interesting to see why inspires people to write tests, and why they give up on them.
Recently i had a very similar conversation with two very different developers, one i’ll call TestFail and the other i’ll call TestHeavy.
Rabble: “I see there are tests on this project, but they don’t seem to pass anymore.”
TestFail: “Yes, I was writing tests but then things got busy and I couldn’t keep them up. The tests feel out of sync with the application 6 months ago. I was the only developer on the project and i just couldn’t keep up the tests and getting releases out the door.”
Then i had another conversation with a different developer on the same subject.
Rabble: “I see there are a lot of tests for this project.”
TestHeavy: “Yes. The thing is when a project gets really pressured for time it’s the tests which make it possible to add features and keep moving forward. Normally I just develop on my own and don’t tend to work with other developers on projects. I found having tests means i can keep getting releases out the door.”
In both cases i’m paraphrasing from memory, but the thing which struck me is that two smart developers used the same two reasons to come up with totally different conclusions around testing. The two principal points are, i’m working on my own, and there is heavy time pressure on releases.
The code bases in question, i’ll keep this all anonymous to respect privacy, are also really different. First off, the applications do totally different things, but it’s enough to say that they are both fairly complicated web applications built in rails.
TestFail’s application has 44075 lines of application code and 778 lines of tests. Over the last year the code base has grown by over %50. On the other hand, TestHeavy’s code base is 4014 lines of application code and 4802 lines of tests (technically specs as it uses rspec). TestHeavy’s code base is younger, but over the last 6 months it’s only added a couple hundred lines of new code. Many new features have been added, but the size of the application has stated more or less the same.
While i clearly prefer TestHeavy’s approach, what puzzles me is the path taken by TestFail. There are many interesting things to be learned by watching the techniques of an effective test driven developer, but that’s kind of an easy problem. Just watch what they do and document it.
Harder is to understand why somebody fails. When somebody starts writing tests but finds they aren’t a useful part of their software development practice. It could be considered the ‘falling off the wagon’ problem. Really to me it’s a question of why is testing not the path of least resistance. What needs to happen for a developer to internalize testing.
If tests are written, and they are relevant to the the functionality of the application, the code base stays smaller, faster, and is more maintainable. But telling people that it’s a good idea to write tests, even getting them started and using tests isn’t enough. Figuring how to help people continue is even more important. I suspect that a large part of the problem is that people partially start using tests, but they don’t ever get a development environment which is setup to encourage testing. It’s painful and difficult as opposed to making debugging easier and development faster.
Does anybody have experience with trying and failing to stick with testing. Why was it?
2 Comments, Comment or Ping
Does an order of magnitude difference in code base have something to do with it? The 40k l.o.c. site is either 10x more complicated, or is written pretty poorly. Leaving aside the question of code quality, just think about the difference between 4k and 40k. Are you sure that the dev with 4k l.o.c. would say the same thing if he had 40k l.o.c. and needed to add features on top of this?
In my experience, there are some projects which have managers who are more interested in features than in quality. If they can get the features added, even if they are buggy, they are happier than if they had a very stable system with less feature growth. I think that this attitude is difficult for most devs to deal with, but it should be acknowledged. And I think that developing without tests is a pretty rational response to these pressures. Yes, you are going to have real moments of frustration debugging, and your code is never going to be as solid, but I think that abandoning your tests is a rational short-term response to these pressures. Sure, you’ll want to change those attitudes in the long term, or move on, but I think some testing advocates forget that there is certainly a trade-off between quality, tested code and the amount of “features” that you can jam into a project.
April 25th, 2008
Is there any method that can be used while testing any software ? hey i can find some info here about the testing…..
April 25th, 2008
Reply to “Two conversations about tests and software development”