Agile Testing: A Practical Guide for Testers and Agile Teams

Chapter 19 Wrap Up the Iteration

We’ve completed an iteration. What do testers do as the team wraps up this iteration and prepares for the next? We like to focus on how we and the rest of our team can improve and deliver a better product next time.

Iteration Demo

One of the pleasures of agile development is the chance to show completed stories to customers at the end of each iteration. Customers get to see a real, live, working application. They get to ask questions and give feedback. Everyone involved in the project, from both the business and technical sides, gets to enjoy a sense of accomplishment.

On Lisa’s team, the testers conduct the iteration review. Among all the team members, they’ve usually worked on the most stories. They have a natural role as information providers, and they have a good idea what the customers need to know about the new functionality. Having testers show off the deliverables is a common practice, although there is no hard and fast rule. The business experts on the team are a good choice for conducting the demo too, because they have the best understanding of how the software meets the business needs and they’ll feel greater ownership of the product. The ScrumMaster, a programmer, or a business analyst could demonstrate the new features and often does. Janet encourages rotating this honor.

Listening to the Customers

Pierre Veragen explains how his team uses iteration demonstrations.

“We shut up and listen to our customers. It’s all about the chemistry of the group’s presentation. Somehow, sharing the moment brings brains together—we look at things from a different perspective. The event gives birth to ideas and concepts. Some die as the next person speaks; some live on and become that great idea that differentiates the product.”

The demo is a chance to show off the new stories, but the feedback customers provide is the biggest reason to do them.

Anyone may note the comments made by customers as they participate in the demo, but testers are good candidates. They may notice previously undetected inconsistencies as the demo progresses. As questions come up, customers might decide they want to change something minor, such as help text, or something bigger, such as how a feature behaves. Minor changes can usually be made into tasks and dealt with in the next iteration, but some changes are big enough to turn into stories to plan into future releases.

Iteration demos (called sprint reviews in the Scrum world) are a super opportunity to get everyone talking and thinking about the application. Take advantage of it. Review meetings are usually short and can be under half an hour. If there’s time left over after demonstrating new stories, ask customers if they’ve experienced any problems with the previous release that they haven’t reported. Do they have any general concerns, do they need help understanding how to use a feature, or have any new issues arisen? Of course, you can talk to customers anytime, but having most of the stakeholders in the room with the development team can lead to interesting ideas.

Retrospectives

Agile development means continually improving the way you work, and retrospectives are an excellent place to start identifying what and how you can do better. We recommend taking time at the end of each iteration and release cycle to look back and talk about what went well, what didn’t, and what you might like to try in the next iteration. There are different approaches for conducting retrospective sessions. No matter what approach you use, it’s key that each team member feels safe, everyone is respected, and there’s no finger-pointing or blame.

The whole idea is to make the process better, one baby step at a time.

Start, Stop, Continue

One common exercise used in iteration retrospectives is “start, stop, continue.” The team asks itself: “What went well during this past iteration? What happened that shouldn’t happen again? What can we start doing to help with things that didn’t go well?” Each team member can suggest things to start doing to improve, things to stop doing that weren’t working, and things that are helping that should be continued. A facilitator or ScrumMaster lists them on a whiteboard or big piece of paper. Post them in a location where everyone can read them again during the iteration. Figure 19-1 shows a “stop, start, and continue” retrospective in progress. The ScrumMaster (standing) is writing stop, start, and continue suggestions on the big piece of paper on the story board.

Figure 19-1 A retrospective in progress. Used with permission of Mike Thomas. Copyright 2008.

Agile Retrospectives: Making Good Teams Great [2006] has imaginative ideas for making retrospectives more productive (see the bibliography).

Some teams start this process ahead of time. All team members write “start,” “stop,” and “continue” items on sticky notes, and then during the retrospective meeting they put the stickies on the board and group them by topic. “Start, stop, continue” is just one example of the terms you might use. Some other ideas are: “Things that went well,” “Things to improve,” “Enjoyable,” “Frustrating,” and “To Try.” Use whatever names that work for you. It can be hard to remember the past two weeks, much less an entire release, if that’s what your retrospective covers. Research different creative approaches to reflecting on your team’s experiences.

Here’s a sample “stop, start, continue” list from Lisa’s team:

Start:

Sending out next sprint’s stories to us earlier.

Don’t do lazy, single-record processing. Think of every service call as a remote call.

Communicate any database changes to everyone.

Stop:

Accepting stories without complete requirements.

Continue:

Running FitNesse tests for the code you’re working on.

Documenting what came up in meeting or informal discussions.

Communicating better with each other.

Showing mock-ups early.

Doing FitNesse driven development.

If the list of “start, stop, continue” items is long, it’s a good idea to choose one or two to focus on for the new iteration. To prioritize the items, give each team member “n” votes they can assign to items. The ten people on Lisa’s team each get three votes, and they can apply them all to one item if they feel that’s most important, or they can vote for two or three different items. The items with the most votes are noted as the focus items. Janet has had success with this way of prioritizing as well.

In addition to “start, stop, continue” items, the team may simply write task cards for actions to be undertaken the next iteration. For example, if the ongoing build is too slow, write a card to “get ongoing build under ten minutes.”

In the next iteration, take some time to look at the one or two focus items you wanted to improve. At the end of that iteration, take a checkpoint to see if you improved. If not, ask why. Should you try something different? Is it still important? It could be it has dropped in importance or really wasn’t important in the big picture. If you thought you improved on a problem area and it resurfaces, you’ll have to decide to do something about it or else quit talking about it.

We’ve found that retrospectives are a simple and highly effective way for teams to identify and address issues. The retrospective meeting is a perfect opportunity to raise testing-related issues. Bring up the issues in an objective, non-blaming way. The team can discuss each problem, what might be causing it, and write down some ideas to fix it.

Ideas for Improvements

Let’s take a look at some of those items that made it onto the list for improvement. Too many times, a team will identify really big issues but never follow up and actually do something about them. For example, maybe a lot of unit-level bugs are discovered after the programmers have claimed coding was complete.

The team may decide the programmers aren’t covering enough code with unit tests. They might write an action item to run the code coverage tool before they check in new code, or start writing a “unit tests” task card for each story to make sure they’re completed. Perhaps the team didn’t finish all the test automation tasks before the iteration ended. As they discuss the problem, the team finds that the initial executable tests were too complex, and they need to focus on writing and automating a simple test first, or pair for a better test design. Make sure the action items are concrete.

Agile teams try to solve their own problems and set guidelines to help themselves improve. Action items aimed at one problem may help with others. When Lisa’s team had trouble finishing stories and getting them tested during each iteration, it came up with various rules over the course of a few retrospectives:

Finish high-level test cases for all stories by the fourth day of the iteration.

Deliver one story to test by the fourth day of the iteration.

Focus on finishing one story at a time.

100% of features must be checked in by close of business on the next-to-last day of the iteration.

These rules did more than help the team finish testing tasks. They facilitated a flow and rhythm that helped the team work at a steady, sustainable pace over the course of each iteration.

Begin the next retrospective meeting by reviewing the action items to see what items were beneficial. Lisa’s team puts happy, sad, or neutral faces next to items to denote whether the team tried them and found them successful. The team should figure out the reasons behind any sad faces. Were some items simply forgotten? Did time constraints keep the team from trying a new activity? Did it just seem to be less of a good idea later? These discussions might lead to changing the improvement item or evolving it into a new one.

When the actions for improvement become a habit to the team, they no longer need to be written on the “stop, start, and continue” list. “Start” items that work well may be moved to the “Continue” column. Some ideas don’t work, or prove to be unnecessary, and those can also be taken off the list for the next iteration.

Refer to your ideas for improvement and action items during the iteration. Post them in a location (on a wall or online) where everyone sees them often. Lisa’s team sometimes goes through the list during a mid-iteration stand-up meeting. If you think of new improvement ideas during the iteration, write them down, possibly even on the existing list, so you won’t forget for the next iteration.

It’s a good idea to keep track of things that get in your way throughout the iteration. Keep an impediment backlog on some big visible chart. Talk about the impediments in each iteration, and write task cards or take action to eliminate them.

An Approach to Process Improvement

Rafael Santos, VP of Software Development and Chief ScrumMaster at Ultimate Software, and Jason Holzer, the Chief PSR (Performance, Security, Reliability) Architect, explained to us that their teams found retrospectives that used the “stop, start, and continue” model ineffective. They made “stop, start, and continue” lists, but those didn’t provide enough focus to address issues.

Instead, the ScrumMaster kept an impediment backlog, and the team found that worked better than retrospectives. Impediments may be related to testing or tools.

They also do value stream mapping to find the biggest “wait time,” and use the “five whys” from Toyota to understand which impediment is the biggest or which constraint needs to be addressed.

See the bibliography for good resources for lean development practices.

One example shared was that in a team with three programmers and one tester, the biggest problem was a testing bottleneck. Rafael asked the team what the tester does and wrote those items on a whiteboard. Then he asked the programmers which of those things on the board they couldn’t do. There was only one item they felt they couldn’t handle. This helped the programmers understand how everyone on the development team, not only the testers, could be responsible for testing tasks. This was a highly effective exercise.

Creative approaches like this help new agile teams tackle difficult testing challenges. Retrospectives are a good environment for experimenting.

Use retrospectives as an opportunity to raise testing-related issues and get the whole team thinking about possible solutions. We’ve been pleasantly surprised with the innovative ideas that come out of an entire team focusing on how to improve the way it works.

Celebrate Successes

Agile development practices tend to moderate the highs and lows that exist in more traditional or chaotic processes. If your waterfall team finally manages to push a release out the door after a year-long cycle ending in a two-month stressful fix-and-test cycle, everyone may be ready to celebrate the event with a big party—or they might just collapse for a couple of weeks. Agile teams that release every two weeks tend to stay in their normal coding and testing groove, starting on the next set of stories after drawing just enough breath to hold an iteration review and retrospective. This is nice, but you know what they say about all work and no play.

Make sure your team takes at least a little time to pat itself on the back and recognize its achievements. Even small successes deserve a reward. Enjoyment is a vital agile value, and a little motivation helps your team continue on its successful path. For some reason, this can be hard to do. Many agile teams have trouble taking time to celebrate success. Sometimes you’re eager to get going with the next iteration and don’t take time to congratulate yourselves on the previous accomplishments.

Lisa’s team ends an iteration every other Thursday and conducts its retrospective, iteration review, and release the following day. After their meetings conclude, they usually engage in something they call “Friday Fun.” This sometimes consists of playing a silly trivia or board game, going out for a drink, or playing a round of miniature golf. Getting a chance to relax and have a good laugh has a team-building side benefit.

For bigger milestones, such as a big release or achieving a test coverage goal, the whole company has a party to celebrate, bringing in catered food or going out to a restaurant on Friday afternoon. This is a nice reward and recognizes for everyone on both the business and technical teams.

If yours is a new agile team, motivate yourselves by rewarding small accomplishments. Cheer the rising number of unit tests passing in each build. Oooh and aaah over the chart showing actual burn down matching the projected burn down. Ring a bell when the broken unit tests in the build are fixed (okay, that one might be annoying, but recognize it in some way.)

Celebrate your individual successes, too. Congratulate your coworker for completing the project’s first performance test baseline. Give your DBA a gold star for implementing a production back-up system. Give yourself a treat for solving that hard test-automation problem. Bring cookies to your next meeting with the customers. Recognize the programmer who gave you a JavaScript harness that sped up testing of some GUI validations. Use your imagination.

The Shout-Out Shoebox

We love the celebration idea we got from Megan Sumrell, an agile trainer and coach. She shared this with an agile testing Open Space session at Agile 2007.

Celebrating accomplishments is something I am pretty passionate about on teams. On a recent project, we implemented the Shout-Out Shoebox. I took an old shoebox and decorated it. Then, I just cut a slit in the top of the lid so people could put their shout-outs in the box. The box is open to the entire team during the course of the sprint.

Anytime team members want to give a “shout-out” to another team member, they can write it on a card and put it in the box. They can range from someone helping you with a difficult task to someone going above and beyond the call of duty. If you have distributed team members, encourage them to email their shout-outs to your ScrumMaster who can then put them in the box as well.

At the end of our demo, someone from the team gets up and reads all of the cards out of the box. This is even better if you have other stakeholders at your demo. That way, folks on your team are getting public recognition for their work in front of a larger audience. You can also include small give-aways for folks, too.

It may be a cliché, but little things can mean a lot. The Shout-Out Shoebox is a great way to recognize the value different team members contribute.

Taking time to celebrate successes lets your team take a step back, get a fresh perspective, and renew its energy so it can keep improving your product. Give team members a chance to appreciate each other’s contributions. Don’t fall into a routine where everyone has their head down working all the time.

In agile development, we get a chance to stop and get a new perspective at the end of each short iteration. We can make minor course corrections, decide to try out a new test tool, think of better ways to elicit examples from customers, or identify the need for a particular type of testing expertise.

Summary

In this chapter, we looked at some activities to wrap up the iteration or release.

The iteration review is an excellent opportunity to get feedback and input from the customer team.

Retrospectives are a critical practice to help your team improve.

Look at all areas where the team can improve, but focus on one or two at a time.

Find a way to keep improvement items in mind during the iteration.

Celebrate both big and small successes, and recognize the contributions from different roles and activities.

Take advantage of the opportunity after each iteration to identify testing-related obstacles, and think of ways to overcome them.