Agile Testing: A Practical Guide for Testers and Agile Teams

Chapter 16 Hit the Ground Running

In agile development, we generally like to do tasks “just in time.” We can’t see around the curves in the road ahead, so we focus on the activities at hand. Then again, we want to hit the ground running when we start each new iteration. That may require a little preparation. Baking is a good analogy here. You decide you want to bake cookies because someone is coming over. Before you start, you make sure you have the right ingredients. If you don’t, you either go buy what you need, or you choose a different kind to make.

Don’t go overboard—if a pre-iteration activity doesn’t save time during the iteration, or help you do a better job, don’t do it before the iteration. Do what is appropriate for your team, and keep experimenting. Maybe you’ll do some of these activities after the iteration starts instead. Here are some ideas to think about that might help you “bake quality in” to your product.

Be Proactive

In Chapter 2, “Ten Principles of an Agile Tester,” we explained how agile testers have to shift their mind-set. Instead of waiting for work to come to us, we develop a proactive attitude where we get up and go look for ways to contribute. With the fast and constant pace of agile development, it’s easy to get immersed in the current iteration’s stories. We are so busy making sure we’ve covered the features with up-front tests, performing exploratory testing to be sure we’ve understood the business requirements, and automating adequate regression tests, it’s hard to think of anything else. However, it’s sometimes appropriate to take a bit of time to help our customers and our team prepare for the next iteration. When our team is about to break new ground, or work on complex and risky stories, some work before the iteration can help maximize our team’s velocity and minimize frustration.

We sure don’t want to spend all our time in meetings, or planning for stories that might be re-prioritized. However, if we can make our iteration planning go faster, and reduce the risk of the stories we’re about to undertake, it’s worth doing some research and brainstorming before we start the iteration.

Benefits

Working on stories in advance of the iteration may be especially useful for teams that are split across different geographic locations. By working ahead, there’s time to get information to everyone and give them a chance to give their input.

The problem is that we’re so busy during each short agile iteration that it’s hard to find time to meet about the next iteration’s stories, much less start writing test cases. If your iterations always go smoothly, with stories delivered incrementally and plenty of time to test, and the delivered software matches customer expectations, you may not need to take time to prepare in advance. If your team has trouble finishing stories, or ends up with big mismatches between actual and desired behavior of features, a little advance planning may save you time during the iteration.

Lisa’s Story

Our team used to feel we didn’t have time to plan in advance for the next iteration. After many experiences of misunderstanding stories and having them far exceed estimations, and finding most “bugs” were missed requirements, we decided to budget time in the iteration to start talking about the next one. Now the whole team, including the product owner and other customers as needed, meet for an hour or less the day before the planning meeting for our next sprint.

We laughingly call this the “pre-planning” meeting. We go over the stories for the next iteration. The product owner explains the purpose of each story. He goes over the business conditions of satisfaction and other items in his story checklists, and gives examples of desired behavior. We brainstorm about potential risks and dependencies, and identify steel threads where appropriate.

Sometimes it’s enough to spend a few minutes listening to the product owner’s explanation of the stories. At other times, we take time to diagram thin slices of a story on the whiteboard. Figure 16-1 shows an example diagram where we got into both the details of the UI flow and the database tables. Note the numbers for the “threads.” Thread #1 is our critical path. Thread #2 is the second layer, and so on. We upload photos of these diagrams to the wiki so our remote developer can see them too.

Figure 16-1 Sample planning whiteboard diagram

We can start thinking about what task cards we might write the next day and what approach we might take to each story. For some reason, being able to ruminate about the stories overnight makes us more productive in the actual iteration planning meeting the next day. After doing this for a few iterations, we were spending less time overall in planning the iteration, even though we had two meetings to do it.

Sometimes we pull in other customers to discuss stories that affect them directly. If they aren’t available right then, we still have time before our iteration planning meeting to talk to them to clarify the story.

In one pre-planning meeting, our product owner introduced a story about obtaining performance data for mutual funds. We would send a file to the vendor containing a list of mutual funds, and the vendor would provide an XML file on a website with all the latest performance information for those funds. We would then upload that data into our database.

In the pre-planning meeting, we asked questions such as, “What’s the format of the file we send the vendor?” “Is the ‘as-of’ date for each fund always the last day of the month?” “Is there any security on the website that contains the XML?” “Will we ever get a record for the same fund and “as-of” date that has new data, or can we ignore records with a date we already have in our database?”

By the next day’s iteration planning meeting, the product owner had obtained answers to all our questions. Writing task cards went quickly, and coding could proceed with correct assumptions.

Often, we find a much simpler solution to a story when we discuss it in the pre-iteration planning discussion. We found that to go fast, we needed to slow down first.

—Lisa

Do You Really Need This?

Your team may not need much or any advance preparation. Pierre Veragen and Erika Boyer described to Lisa how their teams at iLevel by Weyerhaeuser write user acceptance tests together at their iteration kickoff meeting.

These tests, which were written on a wiki page or some similar tool along with the story narrative, are used later when team members write task cards for each story and start writing more tests and code. Examples are turned into executable tests. Because the tests change as the team learns more about the story, the team may opt not to maintain the original ones that were written at the start. Keep it simple to start with, and dig into details later.

Lisa subsequently observed one of their planning sessions and saw first-hand how effective this technique was. Even when the product manager provides concrete examples, turning them into tests may flush out missing requirements. Their team did not need to do this before the iteration planning session, but it is not the case with all teams.

Lisa’s Story

My team liked the practice of writing tests together, but because we were writing task cards during iteration planning, we decided to write user acceptance tests together during the pre-planning meeting. We found this kept our discussions focused and we understood each story more quickly. We also did a better job of delivering exactly what the customer had in mind. Our customers noticed a difference in quality, and our product owner encouraged us to continue this practice.

—Lisa

Experiment with short pre-iteration discussions and test-writing sessions. It’ll take you several iterations to find your team’s rhythm, and find out if advance story discussions make you more productive during the iteration.

Potential Downsides to Advance Preparation

There’s a risk to “working ahead.” You could spend time learning more details about a feature only to have the business people re-prioritize at the last minute and put that feature off indefinitely. Invest preparation time when it’s appropriate. When you know you have a complex theme or story coming up, and it has a hard deadline such as Lisa’s team had with the statement story, consider spending some time up front checking out different viewpoints. The only reason to discuss stories in advance is to save time during iteration planning and during development. A deeper understanding of the feature behavior can speed up testing and coding, and can help make sure you deliver the right functionality.

If your situation is so dynamic that stories might be re-prioritized the day that the iteration starts, it isn’t worth trying to do this planning. Instead, make sure you budget time for these discussions during your planning meeting.

Advance Clarity

Lisa’s product owner, Steve Perkins, came up with the term “advance clarity.” Different parts of each organization have different priorities and agendas. For example, Business Development is looking for new features to attract new business, while Operations is prioritizing features that would reduce the number of phone calls from users. The development team tries to understand the range of business needs and get a feel for each individual’s job.

With many different agendas, someone needs to decide what stories should be implemented in the next iteration. Because there are many ways to implement any given story, someone has to decide the specific requirements and capture them in the form of examples, conditions of satisfaction, and test cases. Steve gets everyone together to agree on the value they want from each story, and to provide “advance clarity.”

Customers Speak with One Voice

Scrum provides the helpful role of the product owner to help all the customers “Speak with One Voice.” Whether or not you’re on a Scrum team, find some way to help your customers agree on the priority of the stories and how the components of each story ought to be implemented. Management support is crucial, because any person in this role needs time and authority to get everyone on the same page.

Other teams use business analysts to help flesh out the stories before the next iteration. In one organization Janet worked with, the customers were not available full-time to answer questions, but each team had a business analyst that worked with the customers to flesh out the requirements before the iteration planning meeting. If there were any questions that she could not answer at the meeting, the team either called the customer directly or the analyst followed up immediately after the meeting.

As a tester, you want to sit in on story writing and prioritization meetings. Ask questions that help the customers focus on the core functionality, the critical business value they need. Help participants stay focused on concrete examples that crystallize the meaning of the stories. In meetings that involve multiple customers, it is critical to have a strong facilitator and a method for determining consensus.

As with code, stories are best if they have the bare minimum. For example, an Internet shopping cart needs some way to delete unwanted items, but the ability to move items from the cart to a “save for later” list can probably wait. It may be helpful to talk about this before the iteration, so that the team is clear on what tasks need to be planned. Focus on the simplest thing first and use an example to make it clear.

Get All Viewpoints

Getting requirements from different customers for a story, each of whom has a different agenda, might create chaos. That’s why it’s essential for someone on the customer team to get consensus and coordinate all points of view. This doesn’t mean we shouldn’t get input from different customers. As a tester, you’re considering each story from multiple points of view. It helps to know what the story means to people in different roles.

Lisa’s Story

When my company decided to redesign the retirement plan participants’ quarterly account statements, different people on the business side wanted changes for different reasons. The plan administrators wanted a clearly understandable layout that would minimize the number of calls from confused participants to customer support.

For example, they wanted the statement to show the date and amount of the participant’s most recent contribution. This helps the participant know whether her employer is late in posting contributions to the accounts. Business development wanted jazzy new features that they could sell to potential customers, such as graphs of performance by fund category. Our legal person needed some new text and data on the statement to satisfy federal regulations.

While the product owner balanced all the different needs and presented the final statement layout, it was still important for our team to understand the purpose behind each new piece of information. We needed to talk directly to business experts in the plan administration, business development, and legal areas, and to the product owner. A tester and a programmer met with each group to gather the different viewpoints. By doing this before starting on stories to gather and display data, we understood the requirements much more clearly and even made suggestions to produce the information more efficiently.

—Lisa

Make sure you are as efficient as possible in collecting this data. Sometimes it is important for the whole team to understand the need, and sometimes it is sufficient for one or two of the team members to do the research.

Story Size

As you discuss stories for the next iteration with the customer team members, ask questions to help them make sure each story delivers the value needed. This is a good time to identify new stories they might need to write. Even though the team sized the stories previously, you might find a story is bigger than previously thought. You might even discover that a feature can be implemented more simply than planned, and the story can be smaller.

Sometimes assumptions are made when the story is sized and on further investigation turn out to be false. Even simple stories deserve a closer look. It’s hard for any one person to remember all the details of an application.

Lisa’s Story

Here are some examples of stories that turned out to be significantly bigger or smaller than originally thought.

1. The story was to produce a file of account statements for all participants in a given company retirement plan, which was to be sent to a vendor who would print and mail the statements. It was originally sized with the assumption that all statements were exactly three pages long. Upon further investigation, we discovered that some participants had four-page statements, but the vendor required that all statements be the same length. Our business experts had to decide whether to have a feature to flag any plans whose participants had four-page statements and deal with those manually, or change the statements to make them all four pages long. That’s a much bigger effort than the original story. After we started developing the story, the customers revealed another requirement: If any participant’s address was missing or invalid, the statement should be mailed to the employer instead. It’s reasonable, but we didn’t know about it when we sized the story.

2. Our customers wanted to start displaying the sales phone number in various locations in the UI. There is a different sales phone number for each partner’s site, and at the time there were about 25 different partner sites. This sounded like such a straightforward story that it wasn’t even given to the team to size. The development manager just assigned it a small point value, and it was just “added” to the iteration. He had assumed the phone number was stored in the database, when in fact it was hard-coded in the HTML of each partner’s “contact” page. Storing the correct number for each partner in the database, and changing the code to retrieve the value, made the story twice as big, and there wasn’t room for it in that iteration, so it did not get done.

3. We sized a story for the user interface to allow administrators to submit a request for a batch job to rebalance participant accounts that met a certain condition. It included a confirmation page displaying the number of participants affected. Because the request was queued to run as an asynchronous batch job, the code to determine which participants were affected was in the batch job’s code. Refactoring the code to obtain the number of participants at request time was a big job. After we started working on the story, we asked the primary user of the feature whether he really needed that number upon submitting the request, and he decided it wasn’t necessary. The story became much smaller than originally thought. We always ask questions to find out the true business value that the customers want and eliminate components that don’t have a good ROI.

—Lisa

These stories show that a few questions up front might save time during the iteration that could be spent figuring out what to do with new discoveries. However, we recognize that not all discoveries can be found early. For example, on the first story, a simple question about statement size may have prevented last-minute confusion about how to handle four-page statements, but the inaccurate address issue may not have been considered until it was being coded or tested.

We know there will always be discoveries along the way, but if we can catch the big “gotchas” first, that will help the team work as effectively as possible.

Geographically Dispersed Teams

Some preparation for the next iteration may be useful for teams that are split across different locations. Teams that are distributed in multiple locations may do their iteration planning by conference call, online meeting, or teleconference. One practice, which a team of Lisa’s used, is to assign each team a subset of the upcoming stories and have them write task cards in advance. During the planning meeting, everyone can review all the task cards and make changes as needed. The up-front work enhances communication, makes the stories and tasks visible to everyone, and speeds up the planning process.

Of course, this assumes that the team is using an electronic story or task board. Lisa’s team uses Thoughtwork’s Mingle, but there are many other products out there that serve this purpose.

Coping with Geographic Diversity

We talked to a team we know at a software company that has customers, developers, and testers spread all over the globe. Not only are the customers far away from the technical team but they don’t have bandwidth to be available to answer the development team’s questions. Instead, the team relies on functional analysts who understand both the business side of the application at a detailed level and the technical implementation of the software. These functional analysts act as liaisons between the business and technical teams.

Patrick Fleisch and Apurva Chandra are consultants who were working with this company and served as functional analysts on a project to develop web-based entitlement software, because they are experts in this domain. They traveled between locations to facilitate communication between stakeholders and developers.

The functional analysts worked in advance of the iteration, sizing and getting stories ready to size, helping the technical team to understand the stories. They entered stories into an online tool and built on them by defining test cases, edge conditions, and other information that helped the technical team understand the story. They documented high-level functionality on a wiki aimed at the business users.

Apurva and Patrick played a key role in making the decisions that the technical team needed to get started with the new stories. Their deep business and technical understanding allowed them to provide the team with requirements they needed to get coding, because the actual customers weren’t available to them. David Reed, a tester and automation engineer, told us how he relied on Apurva and Patrick for the information he needed to perform and automate tests. While agile principles say to collaborate closely with the customer, in some situations you have to be creative and find another way to get clear business requirements.

If customers aren’t readily available to answer questions and make decisions, other domain experts who are accessible at all times should be empowered to guide the team by determining priorities and expressing desired system behavior with examples. Testers and business analysts are often called upon to do these activities.

Examples

You may notice that we talk about examples in just about every chapter of this book. Examples are an effective way to learn about and illustrate desired (and undesired) functionality; it’s worth using them throughout your development cycle. Our motto was coined by Brian Marick: “An example would be handy right about now.” (See Figure 16-2.) Start your discussions about features and stories with a realistic example. The idea has taken off, so that at a recent workshop for functional testing we were discussing ideas around calling it “Example-Driven Development.”

Figure 16-2 Brian Marick’s example sticker

When Lisa’s team members meet with their product owner to talk about the next iteration, they ask him for examples of desired behavior for each story. This keeps the discussion at a concrete level and is a fast way to learn how the new features should work. Have a whiteboard handy while you do this, and start drawing. If some team members are in a distant location, consider using tools that allow everyone to see whiteboard diagrams and participate in the discussion. Go through real examples with your customers or their proxies. As during release planning, consider different points of view: the business, end users, developers, and business partners. Unlike release planning, you are looking at far more detail because these are the stories you are planning for the next iteration.

Using examples, you can write high-level tests to flesh out each story a bit more. You may not need to do this before the iteration starts, but for complex stories, it can be a good idea to write at least one happy path and one negative path test case in advance. Let’s consider the story in Figure 16-3.

Figure 16-3 Story for deleting items from shopping cart

The product owner sketches out the desired UI on the whiteboard. There’s a “delete” checkbox next to each item and an “update cart” button. The user can select one or more items and click the button to remove the items. The high-level tests might be:

When the user clicks the delete checkbox next to the item and clicks the “update cart” button, the page refreshes showing the item is no longer in the cart.

When the user clicks the delete checkboxes next to every item in the cart and clicks the “update cart” button, the page refreshes showing an empty cart. (This will generate questions—should the user be directed to another page? Should a “keep shopping” button display?)

When the user clicks the “update cart” button without checking an item for delete, the page is refreshed and nothing is removed from the cart.

Ask your customers to write down examples and high-level test cases before the iteration. This can help them think through the stories more and help define their conditions of satisfaction. It also helps them identify which features are critical, and which might be able to wait. It also helps to define when the story is done and manage expectations among the team.

Figure 16-4 shows a sample mock-up, where the product owner marked changes on the existing page. Be careful about using an existing screenshot from an old system, because you will run the risk of having a new system look exactly like the old one even if that is not what you wanted.

Figure 16-4 Sample customer mock-up

Mock-ups are essential for stories involving the UI or a report. Ask your customers to draw up their ideas about how the page should look. Share these ideas with the team. One idea is to scan them in and upload them on the wiki so everyone has access. Use those as a starting point and do more paper prototypes, or draw them on the whiteboard. These can be photographed and uploaded for remote team members to see.

Test Strategies

As you learn about the stories for the next iteration, think about how to approach testing them. Do they present any special automation challenge? Are any new tools needed?

Lisa’s Story

Recently, our company needed to replace the voice response unit hardware and interactive voice interface software. A contractor was to provide the software to operate the voice application, but it needed to interact via stored procedures with the database.

This was a big departure from any software we’d worked on before, so it was helpful to have an extra day to research how other teams have tested this type of application before the first iteration planning that involved a story related to this project. During the iteration planning session, we were able to write tasks that were pertinent to the testing needed and give better estimates.

—Lisa

When your team embarks on a new type of software, you may decide to do a development spike to see what you can learn about how to develop it. At the same time, try a test spike to help make sure you’ll know how to drive the development with tests and how to test the resulting software. If a major new epic or feature is coming up, write some cards to research it and hold brainstorming meetings an iteration or two in advance. That helps you know what stories and tasks to plan when you actually start coding. One idea is to have a “scout” team that looks at what technical solutions might work for upcoming stories or themes.

Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” Chapter 10, “Business-Facing Tests that Critique the Product,” and Chapter 11, “Critiquing the Product Using Technology-Facing Tests,” provide examples of tools for different types of testing.

Prioritize Defects

In our ideal world, we want zero defects at the end of each iteration and definitely at the end of the release. However, we recognize that we don’t live in an ideal world. Sometimes we have legacy system defects to worry about, and sometimes fixing a defect is just not high enough value for the business to fix. What happens to these defects? We’ll talk about strategies in Chapter 18, “Coding and Testing,” but for now, let’s just consider that we have defects to deal with.

Before the next iteration is an ideal time to review outstanding issues with the customer and triage the value of fixing versus leaving them in the system. Those that are deemed necessary to be fixed should be scheduled into the next iteration.

Resources

Another thing to double-check before the iteration is whether your team has all the resources you need to complete any high-risk stories. Do you need any experts who are shared with other projects? For example, you may need a security expert if one of the stories poses a security risk or is for a security feature. If load testing will be done, you may need to have a special tool, or have help from a load testing specialist from another team, or even a vendor who provides load testing services. This is your last chance to plan ahead.

Summary

Your team may or may not need to do any preparation in advance of an iteration. Because priorities change fast in agile development, you don’t want to waste time planning stories that may be postponed. However, if you’re about to implement some new technology, embark on a complex new theme, hope to save time in iteration planning, or your team is divided into different locations, you might find some up-front planning and research to be productive. As a tester, you can do the following:

Help the customers achieve “advance clarity”—consensus on the desired behavior of each story—by asking questions and getting examples.

Be proactive, learn about complex stories in advance of the iteration, and make sure they’re sized correctly.

You don’t always need advance preparation to be able to hit the ground running in the next iteration. Don’t do any preparation that doesn’t save time during the iteration or ensure more success at meeting customer requirements.

Coordinate between different locations and facilitate communication. There are many tools to help with this.

Obtain examples to help illustrate each story.

Develop test strategies in advance of the next iteration for new and unusual features.

Triage and prioritize existing defects to determine whether any should be scheduled for the next iteration.

Determine whether any necessary testing resources not currently at hand need to be lined up for the next iteration.