The very basis of our jobs as developers is to write code. If you can’t write code, there is no work for you to do as a software... more

The very basis of our jobs as developers is to write code. If you can’t write code, there is no work for you to do as a software engineer. We spend long years learning to write better, tighter, faster, more elegant code. There comes a time, however, when what we really need to do is not write code. It turns out to be one of the hardest parts of the job: learning when to say no and let someone else do the work. Why should you say no? Isn’t writing code what we do? Yes, it is what we do. Yet the reality of the business of software engineering is that business owners or project sponsors or whoever holds the purse strings don’t hire us to write code. Sounds odd, doesn’t it. But it’s true. They don’t hire us to write code. They hire us to solve problems using software. Sometimes (in fact, most of the time) that means we get to write code. But more often than we like to think, what we really should do is let someone else do the fun part.

As software systems have become increasingly complex, it has become impossible for any one developer to know everything there is to know about building a single system. We’d like to believe that isn’t the case, but unless you are working on a very small project devoted to a very narrow domain, it’s true. Someone who is an expert in crafting HTML to look exactly right in every browser and still knows how to write kernel-mode hardware drivers might exist somewhere in the world, but he certainly isn’t in the majority. There is simply too much to know, which naturally leads to specialization. So we end up with some developers who know the ins and outs of HTML, and others who write device drivers.

Even in the case of a moderate-sized commercial project, no average team of developers will be able to write all the software that it needs to accomplish its goals—nor should it. We get paid to solve problems that business people have, not to write code because it’s fun. Okay, the fact that for most of us writing code also happens to be fun comes as a nice bonus, but our business is really business.

So what’s the point of all that? It is that before we write a single line of code for any project, we should be asking ourselves one simple question: Is this code providing business value to my customers?

At the core of that question lies an understanding of what business value means to your organization. For example, if your company sells online banking software, then the core business value that you are delivering to your customers is banking. When it comes time to build your banking web site, and you need to ask the user what day he wants his bills paid, you might want to add a calendar control to your web page. That certainly makes sense, and it will make the user’s experience of your software better. It’s tempting (because we all like to write code) to immediately start writing a calendar control. After all, you can write one in no time, and you always wanted to. Besides, nobody else’s calendar control is as good as the one that you can write. Upon reflection, however, you realize that there is nothing about being able to write a calendar control that provides value directly to online banking customers. “Of course there is,” you say. “The users want a calendar control.” True, they do. But the business of a company that builds banking software is not to build calendar controls.

... less

Your first response to Test-Driven Development, or TDD, may be that it feels like a good way to encourage developers to write... more

Your first response to Test-Driven Development, or TDD, may be that it feels like a good way to encourage developers to write their unit tests. After you have spent some time with TDD, however, that becomes its least interesting aspect. Let’s take a quick look at what TDD is and where it came from, then move on to the fascinating parts.

Test-Driven Development arose out of early literature around XP and other agile methods and was put forth, first and foremost, as a way to support refactoring. The idea was that writing your tests first forced you into thinking about tests, which in turn encouraged you to write more. The more tests you had, the easier it was to practice refactoring, or the art of making small incremental changes to your code to improve its structure. Proper refactoring involves making very small organizational changes to your code, then making sure that your tests still pass. Without proper test coverage, refactoring is much more difficult.

Note

See Martin Fowler’s excellent book Refactoring: Improving the Design of Existing Code (Addison-Wesley, 1999) for more information on this practice.

The process of TDD is often described, in abbreviated form, as “Red, Green, Refactor,” meaning that first you write tests so that the tests fail (causing a “red light” in most testing GUIs), then you write code to make the tests pass (causing the “light” to turn green), and then you are free to refactor or improve the structure of your code, safe in the knowledge that the tests still pass. Once the refactoring is complete, the cycle begins again:

• Write tests that fail.

• Write code to make the tests pass.

• Refactor your code to improve its structure.

• Repeat . . .

The hardest part about making this cycle work is self-discipline. As developers, the thing we like to do most is write code—code that solves problems. Writing tests for many developers seems like drudgery or busy work. Some feel it’s a waste of time or that someone else should be writing their tests. It is always tempting to “just write a little code” before you start writing the tests for it. You see a problem, and you want to code your way out of it. It can be very difficult to stick to writing tests first, then writing code to make them pass. However, there are a number of compelling reasons for writing the tests first:

1. If you wait, you won’t really write the tests. This is just human nature. In almost every software development shop, the most important metric is success, which means that as soon as your code works you’ve accomplished your primary goal, and the organization will reward moving on to write more code above going back to write unit tests.

2. Writing the tests first forces you, as the developer, to think about your interfaces. It is very easy to write an interface in a vacuum and then discover that it doesn’t work when someone tries to consume your interface. If you write the tests first, you are both consumer and producer, and that leads to much better design. Tests help to define your contract.

3. Tests can provide documentation that is often more accessible to other developers than written documentation would be. Unit tests reflect the way the developer who wrote those tests thought about how the code under test was designed to be used. Tests define our expectations. If the same developer writes both the unit tests and the code to make those tests pass, then the tests reflect the intent of that developer. Writing the tests first expresses that intent more clearly than writing tests later to fit the code you have already written.

TDD can also be very useful when doing pair programming. The most effective way to use TDD to pair is for one person to write the test that expresses his or her intent and for the other person to write the code that makes that test pass. This can be done at a very physical level, passing the keyboard back and forth. First one developer writes the test and then passes the keyboard to the second developer, who then must make the test pass. The pair then jointly decides if any refactoring needs to be done, and the cycle begins again. This time the developers switch roles, trading test writing for implementation tasks. This method is not only fun but also leads to a very high level of productivity and very consistent interface design, because two pairs of eyes have been on the code, and two developers are forced to agree on the intent of their interface.

Test-Driven Development applies to your whole software development lifecycle, not just to the beginning of the process. Tests define the requirements of your system as understood at the time the tests were written. In fact, in some agile projects, tests might be the only concrete representation of the project’s requirements, if the developers have direct access to their customer and the customer doesn’t provide written requirements beyond story cards.

One of the most important ways to use TDD may come after the release of your project. Every time a defect in your code is reported, the first thing you should do is write a test that exposes the defect. If you get a bug report, and immediately create a test that causes the bug to happen, then you can be secure in the knowledge that when your tests pass, the defect is resolved. This provides a great way to ensure success in fixing the bug, and the test remains as part of the total unit test corpus, providing regression testing so that you know the bug won’t reoccur due to future changes. Those new tests may represent aspects of the system that were never properly tested before (this is why code coverage metrics are important; more on that later), or it might mean that the developer who wrote the initial code had an incorrect or incomplete understanding of the requirements.

... less

If you are working by yourself on all of your software projects, Continuous Integration may not be for you. Most of us don’t... more

If you are working by yourself on all of your software projects, Continuous Integration may not be for you. Most of us don’t have that luxury however. Modern software development is almost always done by teams, and sometimes by very large teams. Even in small companies, a single project may involve 5–10 developers, and those developers need to be able to work together in as efficient a manner as possible.

In the past, when most software development followed the waterfall model, developers worked largely independently. Each developer worked on one task, or a subsystem, or some other piece of the overall project. Individual developers kept from stepping on each other’s toes by using source control systems that relied on pessimistic locking. At some point in the waterfall project schedule, time was set aside for an integration phase, in which each developer checked in all his or her work, and the long process of integration began. That integration process is inherently unpredictable. If many developers are all working in isolation and then integrating their work, there is no way to predict ahead of time how long that process will take. Because integration traditionally happened at the end of a cycle, that unpredictability came at a critical juncture.

On even a medium-sized project, that integration phase might last a week or two, as everyone figured out how to make all their changes work together. Interfaces might have changed, new components might have to be integrated, or other incompatibilities might crop up. Only at the end of that integration phase could the code be tested, which meant testers were often idle.

More and more projects today are being run using Agile methodologies such as Scrum or XP. In those methodologies, a single iteration may only last a month, sometimes only a few weeks. If your entire cycle only lasts one month, there is no room in the schedule for a one-week integration phase. To make such a schedule work, incompatibilities must be detected as quickly as possible so that they can be addressed right away.

If a dedicated integration phase is to be avoided, integration must become a part of the developer’s routine. Integration has to happen at every level of the project, as often as possible. The more often integration happens, the less time it will take as a percentage of the project, since each integration will be much easier.

These new scheduling realities led to the philosophy of Continuous Integration.

... less

One of the most important sets of rules to establish with your team are those defining when a given task is “done” and ready... more

One of the most important sets of rules to establish with your team are those defining when a given task is “done” and ready to be turned over to testing. Typically, “done” means that the developer has completed coding to his or her satisfaction, and feels that it is time for testing to have a go at it. Sometimes that works, and sometimes it does not. If that is the only criteria, it is easy for developers to become sloppy, which in turn makes QA defensive, and no good can come of this.

So what does it mean to be done with a task, and how is that task defined?

How a task is defined depends on your project management methodology. If you are following a waterfall model, a task might be a task as defined by the project manager driving the schedule. If you are using Extreme Programming (XP), it might be a story or a task that supports a story; in Scrum, a backlog item or a specific task that is part of a backlog item.

Part of taking your project (and your personal skills) to the next level is raising the bar on what it means to be done with one of those tasks. So what kind of rules should you establish for your team (or yourself for that matter) to make sure that done really means done? We will examine a few rules that I think are important; you may have your own. Almost all of them come down to discipline in the end. “Done is done” means sticking to the rules, even if you don’t think it’s necessary in one particular case, or if you don’t see the point in a specific instance. Come up with a set of criteria that should always be applied to finishing a task. There are inevitably going to be exceptions, but it is worth trying to apply all of the rules first to find cases that really are exceptional, not just the ones you don’t feel like dealing with.

If you can come up with a solid set of criteria, and stick to them, you will end up with a much higher percentage of your work being accepted by QA. That means fewer defects due to oversight, omission, or just plain sloppiness. Fewer defects of this kind means that you will spend more time doing productive work, and testers will spend less time chasing bugs that could easily have been avoided. Even if it feels like it takes you more time to stick to the rules, when you consider all the time you won’t have to spend reading defect reports, trying to reproduce defects, and otherwise dealing with your defect tracking system, you will come out ahead in the end.

Whatever the set of rules you come up with, your whole team should be in agreement. There is no point in establishing rules that the team doesn’t buy into because no one will follow them if that is the case. It is worth taking the time to drive consensus around the “done is done” rules so that everyone is equally invested (as much as is possible) in seeing the rules followed.

It is also important that each of your rules be verifiable, so part of establishing your “done is done” rules is formulating a way to validate that the rules are being followed. If you can’t validate the rules, it will take a lot of extra work on the part of some person or persons to try and “police” the team’s work, and that is not what you want. Nobody wants to play the role of team rules validation cop. It works best if you can validate the rules as part of your build process because the output of the build is visible to the whole team, and you can take advantage of the power of peer pressure to keep everyone on track.

This chapter presents a basic set of rules:

• Any significant design decisions have been discussed and approved by the team.

• Every class in your system should have a corresponding test fixture in your unit test suite.

• Each one of your test fixture classes should test the functionality of only one class under test.

• Code coverage should be 90% or better by line.

• Set your compiler to treat warnings as errors.

• Any static analysis tools that run as part of the build should generate no errors/warnings.

• Before committing anything to source control, update to the latest code and compile/test.

• Get your documentation in order.

You’ll examine why these rules are important, what they mean to your development process, and how they can be validated.

... less

Testing, often referred to as quality assurance (QA), is one of the most important parts of the whole software development... more

Testing, often referred to as quality assurance (QA), is one of the most important parts of the whole software development process. Unfortunately, most of the time testing gets the least consideration, the fewest resources, and generally short shrift throughout the development organization.

Every study ever done on the effectiveness of software testing has shown that the more time you spend up front on testing, the less time and money it takes to complete your project to the level of quality you want to achieve. Everyone in the industry knows this to be true. Yet most developers don’t think about how they will test their software until after it is already written. Despite the fact that time and again all the evidence points to the fact that a bug found at the beginning of development is orders of magnitude less expensive and time-consuming to fix than one found at the end of the development cycle, and bugs are even more expensive and time-consuming to fix after the product has reached the customer.

Why should this be the case?

There are many reasons why developers don’t like testing. Testing is perceived as less “fun” than writing “real” code. Most software development organizations reward developers for producing features in the shortest amount of time, not for writing the highest-quality code. The majority (or at least a plurality) of software companies do not have a mature testing organization, which makes testing appear amateurish and not serious business to other developers. Because of the history of the industry, testing is often seen as an entry-level position, a low-status job. This causes testing to attract less qualified or less experienced developers, and many of them want to “get out” of testing as quickly as they can. Again, because most organizations favor and reward producing features over quality, testing usually receives the fewest resources and tends to get squeezed out of the schedule at the end of a project. This reinforces the impression that quality assurance is not valued.

The single most important thing that any developer can do to improve their skills and those of their team is to take testing seriously and internalize the importance of quality throughout their organization. Let me repeat that. The most important thing you as a developer can do to Code Up!, to take your skills to the next level, to increase your marketability, and to improve your job satisfaction is to embrace testing as a first class part of the software development process. Period.

Embracing testing won’t just improve the quality of your code. It will also improve your design skills. It will bring you closer to understanding the requirements of your customers, both internal and external. It will save you time and hassle throughout the development process, and long after it is over. It will keep your phone from ringing on the weekends or in the middle of the night when you would rather be doing something besides providing support to your customers.

Best of all, it will make you a better developer.

... less

For the vast majority of professional developers, source code control systems (SCCS or SCC) are a part of the landscape.... more

For the vast majority of professional developers, source code control systems (SCCS or SCC) are a part of the landscape. Almost every serious development project relies on an SCC system to provide backups, version control, revision history and archiving. Any source code control system (also known as version control or configuration management systems) will provide these basic features. However, very few organizations properly leverage their source control solution. A good SCC system can provide additional high-value features such as reporting, correlation with defect tracking, atomic commits, platform neutrality, and a host of others.

Not only can a good SCC system add a lot of value, even a basic solution can be used to do much more than simply store multiple versions of the same files. If used properly, features such as branching, tagging, and merging can provide a new level of capabilities that your project and team can take advantage of.

At its most basic, the primary purpose of using a source control system is to make sure that changes to your source code take place in as orderly a fashion as possible. SCC systems help to coordinate and integrate the work of multiple developers, so that each developer can make changes freely and rely on the SCC system to coordinate their changes with the changes of other developers working on the same code. As a secondary benefit, SCC provides a historical record of all the changes that have taken place since the project was started, and allows you to retrieve specific versions or roll back to previous revisions if changes turn out to be unfortunate.

How much benefit your team or your project will get from source control will depend on which system you use, and how well you use it. Not every source control platform supports the same feature set or working model, but almost any system can be used to great benefit if used properly.

Source control is one of the cornerstone tools in any good Continuous Integration (CI) process. For CI to work properly, it must be easy for developers to update to the latest changes in source control, and just as easy for them to merge and check in their changes. SCC systems that do atomic check-in operations are even better for use in a CI process, since each integration can take the form of a single atomic check-in operation. Checking in files in an atomic batch provides better change tracking because all the files associated with a single change are grouped into one change set. Such an atomic change set also makes it easier to speed up how often your CI server checks for changes. If your CI server can be assured that all the files necessary for a given feature or integration will be checked in as a batch, building from incomplete source is less of a concern. That means your CI server doesn’t have to wait as long after changes are detected before starting a new build.

A comprehensive branching strategy (which you'll learn in more detail later in the “Making the Most of Branching” section) can make your CI process easier to manage from a team perspective because it allows developers to continue to check in incremental changes without affecting the rest of the team using the integration branch.

A good source code control system can be just as important for developers working alone as it is for a team. Even if you don’t have to worry about coordinating source code changes with other developers, you can benefit from a source code control system. It is very valuable to keep a historical record of changes you have made during development. You might find yourself wanting to roll back to a previous version if a refactoring goes awry, or you might just want to be able to experiment, secure in the knowledge that you can easily get back to where you started without having to manually backup your files, and so on.

Even if you are working alone, you may want to keep a historical record representing versions of your software already released to customers. Keeping multiple versions of your source in an SCC system is much more efficient than keeping flat file copies and the SCC front end provides additional features such as branching and merging that would be difficult to do with file copies. If you have already shipped software to customers and they find bugs, you should have a simple way of correlating the version of the binaries they are using with the source that produced it. That is much easier to do with an SCC system than if you tried to maintain that information in a file system.

... less

Put simply, static analysis means “stuff you can learn about your code without running it.” There is a whole group of other... more

Put simply, static analysis means “stuff you can learn about your code without running it.” There is a whole group of other tools—profilers, debuggers, memory mappers, code coverage analyzers, and the like—that fall under the heading of dynamic analysis. They all require that your code be running to do their work. Static analysis tools look at the code itself, either using reflection in languages capable of using it or by reading source files much as a compiler would.

... less

What does it mean for software to have a contract? There turns out to be no hard-and-fast answer to that question. It can... more

What does it mean for software to have a contract? There turns out to be no hard-and-fast answer to that question. It can mean something very formal in some situations. Some programming languages such as Eiffel have raised the notion of a software contract to the level of a first-class language feature (meaning that the ability to define a complete contract is part of the language). In other languages, contracts have to be defined in less formal ways. Is it possible to write software without a contract? Not really. It is however possible to write software with a poorly defined or misunderstood contract.

... less

The fewer dependencies the code you write has on other code, the easier it will be to change and the more resilient it will... more

The fewer dependencies the code you write has on other code, the easier it will be to change and the more resilient it will be when faced with changes elsewhere. If your code depends directly on code that you don’t own, it is liable to be fragile and susceptible to breaking changes. That means that if you have a compile-time dependency on another package that you didn’t write, you are in some respects at the mercy of whoever developed that package.

... less

One of the biggest challenges in adopting Test-Driven Development and ultimately Continuous Integration is figuring out how... more

One of the biggest challenges in adopting Test-Driven Development and ultimately Continuous Integration is figuring out how to test the user interface portion of your application (if it has one). Test runner applications that simulate user mouse and keyboard events are costly, difficult to set up, and often require learning a proprietary scripting language. Additionally, it is hard to integrate such tools into a Continuous Integration process because they are often not designed for XML reporting. In the chapter on testing, you learned about functional testing platforms such as Watir, but those work only with web-based applications (rather than with desktop applications).

There are strategies that you can pursue, however, which make it easier to test your user interface (UI) without relying on simulating user interactions. One such strategy has come to be called the Model-View-Presenter (MVP) model. MVP is a design pattern for UI-based applications that makes it much easier to automate testing of almost the entire application.

The fundamental strategy employed by MVP is to separate the bulk of the application from the very thinnest layer of the user interface. Everything but that very thin layer can then be tested using the same unit testing frameworks discussed in Chapter 5 (“Testing”) such as NUnit. The remaining thin slice of user interface code can then be quickly inspected visually by human testers with relatively little effort.

... less

What is tracing? Ask any two developers, and you are likely to get different answers. I’m talking about tracing that means... more

What is tracing? Ask any two developers, and you are likely to get different answers. I’m talking about tracing that means, in a nutshell, reporting what happens inside an application while it is running, whether what happened was good or bad, happy or sad. That definition covers a lot of ground. It includes tracing (sometimes referred to as logging) meant for other developers, tracing meant for support personnel, and even error reporting meant for end users. These all involve reporting in some fashion about something that happened inside the application that you want one of those constituencies to know about. You may want them to know about it right away, or you may want to squirrel away the information for later reference. Tracing can be used to inform support personnel about steps they need to take right away, or it might be done for archival and research purposes to track the long-term behavior of a given application.

Because tracing can cover so many different situations, it’s important to come up with a comprehensive way to deal with tracing as a whole so that all the pieces are well integrated. If you use three different tracing systems to reach three different sets of users, it can become very difficult to keep track of what goes where and which system should be used when. And then it becomes hard to switch from one tracing mechanism to another or to report the same problems to different people.

... less

Dealing with errors is a vital part of any software development project, and like most aspects of development it can be done... more

Dealing with errors is a vital part of any software development project, and like most aspects of development it can be done very well, very poorly, or somewhere in between. Error handling sometimes gets short shrift because, of course, developers never make mistakes. Of course, you also know that isn’t true, never was, and never will be. As software has become more complex, it has also become increasingly important to handle errors carefully and consistently.

Good error handling is key to lowering the cost of ownership of software for your customers. The purchase price paid for software pales in comparison to the cost of supporting that same software. The better your error handling, the lower that cost of support.

Error handling can really be divided into two main categories: how errors are handled internally, and how errors are presented to the user. The way that errors are handled internally mainly affects developers. Good internal error handling makes finding and fixing defects easier, and makes code easier to read and to understand.

The way errors are handed off to the user makes the difference between easily supported software, and costly and difficult to support software. Errors should only be handed off to the user if there is something that the user can do to correct the problem, and that needs to be clearly communicated. As discussed in the last chapter, it is vitally important to make your error messages to the user actionable. If an error message doesn’t tell the user how to correct the problem, then there is no point in showing the user an error message.

Given the importance of error handling, it is not surprising that many developers have strong opinions about the subject. How errors should be handled programmatically and how they should be presented to the user can be quite contentious, and it is almost certain that 10 software developers in the same room will have at least 12 different opinions on how to go about it. Arguably the largest divide is between the “result code readers” and the “exception throwers.”

... less

Bob the developer comes to work in the morning, catches up on his email, and then sets to work on his project. Bob is working... more

Bob the developer comes to work in the morning, catches up on his email, and then sets to work on his project. Bob is working on the Calculator project, which has just recently started up. The Calculator team has just begun their first development iteration, and Bob is anxious to start writing code and adding value to the project.

The first thing that Bob does is decide which task to work on. He goes to his project management system and picks the highest-priority task that is currently unassigned.

Note

For the sake of this narrative, Bob’s team is using SCRUM to manage tasks and how they are assigned, but any development methodology would fit in here. The important point is that Bob chooses a task that has some form of task number associated with it.

Bob’s task for the day is Task 5: Add subtraction to the calculator. Bob has a pretty clear idea of what that task entails and how it fits with the rest of the project, so he doesn’t think that he needs to validate anything with the customer.

... less