Table of Contents
Foreword by Mike Cohn
Foreword by Brian Marick
Preface
Acknowledgments
About the Authors
Part I. Introduction
Chapter 1. What Is Agile Testing, Anyway?
Agile Values
What Do We Mean by “Agile Testing”?
A Little Context for Roles and Activities on an Agile Team
Customer Team
Developer Team
Interaction between Customer and Developer Teams
How Is Agile Testing Different?
Working on Traditional Teams
Working on Agile Teams
Traditional vs. Agile Testing
Whole-Team Approach
Summary
Chapter 2. Ten Principles for Agile Testers
What’s an Agile Tester?
The Agile Testing Mind-Set
Applying Agile Principles and Values
Provide Continuous Feedback
Deliver Value to the Customer
Enable Face-to-Face Communication
Have Courage
Keep It Simple
Practice Continuous Improvement
Respond to Change
Self-Organize
Focus on People
Enjoy
Adding Value
Summary
Part II. Organizational Challenges
Chapter 3. Cultural Challenges
Organizational Culture
Quality Philosophy
Sustainable Pace
Customer Relationships
Organization Size
Empower Your Team
Barriers to Successful Agile Adoption by Test/QA Teams
Loss of Identity
Additional Roles
Lack of Training
Not Understanding Agile Concepts
Past Experience/Attitude
Cultural Differences among Roles
Introducing Change
Talk about Fears
Give Team Ownership
Celebrate Success
Management Expectations
Cultural Changes for Managers
Speaking the Manager’s Language
Change Doesn’t Come Easy
Be Patient
Let Them Feel Pain
Build Your Credibility
Work On Your Own Professional Development
Beware the Quality Police Mentality
Vote with Your Feet
Summary
Chapter 4. Team Logistics
Team Structure
Independent QA Teams
Integration of Testers into an Agile Project
Agile Project Teams
Physical Logistics
Resources
Tester-Developer Ratio
Hiring an Agile Tester
Building a Team
Self-Organizing Team
Involving Other Teams
Every Team Member Has Equal Value
Performance and Rewards
What Can You Do?
Summary
Chapter 5. Transitioning Typical Processes
Seeking Lightweight Processes
Metrics
Lean Measurements
Why We Need Metrics
What Not to Do with Metrics
Communicating Metrics
Metrics ROI
Defect Tracking
Why Should We Use a Defect Tracking System (DTS)?
Why Shouldn’t We Use a DTS?
Defect Tracking Tools
Keep Your Focus
Test Planning
Test Strategy vs. Test Planning
Traceability
Existing Processes and Models
Audits
Frameworks, Models, and Standards
Summary
Part III. The Agile Testing Quadrants
Chapter 6. The Purpose of Testing
The Agile Testing Quadrants
Tests that Support the Team
Tests that Critique the Product
Knowing When a Story Is Done
Shared Responsibility
Managing Technical Debt
Testing in Context
Summary
Chapter 7. Technology-Facing Tests that Support the Team
An Agile Testing Foundation
The Purpose of Quadrant 1 Tests
Supporting Infrastructure
Why Write and Execute These Tests?
Lets Us Go Faster and Do More
Making Testers’ Jobs Easier
Designing with Testing in Mind
Timely Feedback
Where Do Technology-Facing Tests Stop?
What If the Team Doesn’t Do These Tests?
What Can Testers Do?
What Can Managers Do?
It’s a Team Problem
Toolkit
Source Code Control
IDEs
Build Tools
Build Automation Tools
Unit Test Tools
Summary
Chapter 8. Business-Facing Tests that Support the Team
Driving Development with Business-Facing Tests
The Requirements Quandary
Common Language
Eliciting Requirements
Advance Clarity
Conditions of Satisfaction
Ripple Effects
Thin Slices, Small Chunks
How Do We Know We’re Done?
Tests Mitigate Risk
Testability and Automation
Summary
Chapter 9. Toolkit for Business-Facing Tests that Support the Team
Business-Facing Test Tool Strategy
Tools to Elicit Examples and Requirements
Checklists
Mind Maps
Spreadsheets
Mock-Ups
Flow Diagrams
Software-Based Tools
Tools for Automating Tests Based on Examples
Tools to Test below the GUI and API Level
Tools for Testing through the GUI
Strategies for Writing Tests
Build Tests Incrementally
Keep the Tests Passing
Use Appropriate Test Design Patterns
Keyword and Data-Driven Tests
Testability
Code Design and Test Design
Automated vs. Manual Quadrant 2 Tests
Test Management
Summary
Chapter 10. Business-Facing Tests that Critique the Product
Introduction to Quadrant 3
Demonstrations
Scenario Testing
Exploratory Testing
Session-Based Testing
Automation and Exploratory Testing
An Exploratory Tester
Usability Testing
User Needs and Persona Testing
Navigation
Check Out the Competition
Behind the GUI
API Testing
Web Services
Testing Documents and Documentation
User Documentation
Reports
Tools to Assist with Exploratory Testing
Test Setup
Test Data Generation
Monitoring Tools
Simulators
Emulators
Summary
Chapter 11. Critiquing the Product Using Technology-Facing Tests
Introduction to Quadrant 4
Who Does It?
When Do You Do It?
“ility” Testing
Security
Maintainability
Interoperability
Compatibility
Reliability
Installability
“ility” Summary
Performance, Load, Stress, and Scalability Testing
Scalability
Performance and Load Testing
Performance and Load-Testing Tools
Baseline
Test Environments
Memory Management
Summary
Chapter 12. Summary of Testing Quadrants
Review of the Testing Quadrants
A System Test Example
The Application
The Team and the Process
Tests Driving Development
Unit Tests
Acceptance Tests
Automation
The Automated Functional Test Structure
Web Services
Embedded Testing
Critiquing the Product with Business-Facing Tests
Exploratory Testing
Testing Data Feeds
The End-to-End Tests
User Acceptance Testing
Reliability
Documentation
Documenting the Test Code
Reporting the Test Results
Using the Agile Testing Quadrants
Summary
Part IV. Automation
Chapter 13. Why We Want to Automate Tests and What Holds Us Back
Why Automate?
Manual Testing Takes Too Long
Manual Processes Are Error Prone
Automation Frees People to Do Their Best Work
Automated Regression Tests Provide a Safety Net
Automated Tests Give Feedback, Early and Often
Tests and Examples that Drive Coding Can Do More
Tests Are Great Documentation
ROI and Payback
Barriers to Automation—Things that Get in the Way
Bret’s List
Our List
Programmers’ Attitude—“Why Automate?”
The “Hump of Pain” (The Learning Curve)
Initial Investment
Code that’s Always in Flux
Legacy Code
Fear
Old Habits
Can We Overcome These Barriers?
Summary
Chapter 14. An Agile Test Automation Strategy
An Agile Approach to Test Automation
Automation Test Categories
Test Automation Pyramid
What Can We Automate?
Continuous Integration, Builds, and Deploys
Unit and Component Tests
API or Web Services Testing
Testing behind the GUI
Testing the GUI
Load Tests
Comparisons
Repetitive Tasks
Data Creation or Setup
What Shouldn’t We Automate?
Usability Testing
Exploratory Testing
Tests that Will Never Fail
One-Off Tests
What Might Be Hard to Automate?
Developing an Automation Strategy—Where Do We Start?
Where Does It Hurt the Most?
Multi-Layered Approach
Think about Test Design and Maintenance
Choosing the Right Tools
Applying Agile Principles to Test Automation
Keep It Simple
Iterative Feedback
Whole-Team Approach
Taking the Time to Do It Right
Learn by Doing
Apply Agile Coding Practices to Tests
Supplying Data for Tests
Data Generation Tools
Avoid Database Access
When Database Access Is Unavoidable or Even Desirable
Understand Your Needs
Evaluating Automation Tools
Identifying Requirements for Your Automation Tool
One Tool at a Time
Choosing Tools
Agile-Friendly Tools
Implementing Automation
Managing Automated Tests
Organizing Tests
Organizing Test Results
Go Get Started
Summary
Part V. An Iteration in the Life of a Tester
Chapter 15. Tester Activities in Release or Theme Planning
The Purpose of Release Planning
Sizing
How to Size Stories
The Tester’s Role in Sizing Stories
An Example of Sizing Stories
Prioritizing
Why We Prioritize Stories
Testing Considerations While Prioritizing
What’s in Scope?
Deadlines and Timelines
Focus on Value
System-Wide Impact
Third-Party Involvement
Test Planning
Where to Start
Why Write a Test Plan?
Types of Testing
Infrastructure
Test Environments
Test Data
Test Results
Test Plan Alternatives
Lightweight Test Plans
Using a Test Matrix
Test Spreadsheet
A Whiteboard
Automated Test List
Preparing for Visibility
Tracking Test Tasks and Status
Communicating Test Results
Release Metrics
Summary
Chapter 16. Hit the Ground Running
Be Proactive
Benefits
Do You Really Need This?
Potential Downsides to Advance Preparation
Advance Clarity
Customers Speak with One Voice
Story Size
Geographically Dispersed Teams
Examples
Test Strategies
Prioritize Defects
Resources
Summary
Chapter 17. Iteration Kickoff
Iteration Planning
Learning the Details
Considering All Viewpoints
Writing Task Cards
Deciding on Workload
Testable Stories
Collaborate with Customers
High-Level Tests and Examples
Reviewing with Customers
Reviewing with Programmers
Test Cases as Documentation
Summary
Chapter 18. Coding and Testing
Driving Development
Start Simple
Add Complexity
Assess Risk
Coding and Testing Progress Together
Identify Variations
Power of Three
Focus on One Story
Tests that Critique the Product
Collaborate with Programmers
Pair Testing
“Show Me”
Talk to Customers
Show Customers
Understand the Business
Completing Testing Tasks
Dealing with Bugs
Is It a Defect or Is It a Feature?
Technical Debt
Zero Bug Tolerance
It’s All about Choices
Decide Which Bugs to Log
Choose When to Fix Your Bugs
Choose the Media You Should Use to Log a Bug
Alternatives and Suggestions for Dealing with Bugs
Start Simple
Facilitate Communication
Testers Facilitate Communication
Distributed Teams
Regression Tests
Keep the Build “Green”
Keep the Build Quick
Building a Regression Suite
Checking the “Big Picture”
Resources
Iteration Metrics
Measuring Progress
Defect Metrics
Summary
Chapter 19. Wrap Up the Iteration
Iteration Demo
Retrospectives
Start, Stop, Continue
Ideas for Improvements
Celebrate Successes
Summary
Chapter 20. Successful Delivery
What Makes a Product?
Planning Enough Time for Testing
The End Game
Testing the Release Candidate
Test on a Staging Environment
Final Nonfunctional Testing
Integration with External Applications
Data Conversion and Database Updates
Installation Testing
Communication
What If It’s Not Ready?
Customer Testing
UAT
Alpha/Beta Testing
Post-Development Testing Cycles
Deliverables
Releasing the Product
Release Acceptance Criteria
Release Management
Packaging
Production Support
Understand Impact to Business
Customer Expectations
Summary
Part VI. Summary
Chapter 21. Key Success Factors
Success Factor 1: Use the Whole-Team Approach
Success Factor 2: Adopt an Agile Testing Mind-Set
Success Factor 3: Automate Regression Testing
Success Factor 4: Provide and Obtain Feedback
Success Factor 5: Build a Foundation of Core Practices
Continuous Integration
Test Environments
Manage Technical Debt
Working Incrementally
Coding and Testing Are Part of One Process
Synergy between Practices
Success Factor 6: Collaborate with Customers
Success Factor 7: Look at the Big Picture
Summary
Glossary
Bibliography
Index