Agile tester : Skills Needed

In last post we discussed about how the role of a tester changes in agile world. To cope with the changed world, a tester needs to acquire a few new skills. This post discusses what exactly needs to be done

Whenever the two words agile and tester come in one sentence, the next word to be expected is automation. While automation does form a part of the skill repository of tester, its not the only skill they need to acquire to fit in agile world.The diagram above showcases snapshot what needs to be learnt new by a tester in order to thrive in this new world!

  1. Scrum Framework – This category is self explanatory 🙂 A good tester needs to know the framework he or she is operating in. In most of the agile projects – the framework is scrum. The Tester needs to know the same
  2. Common Agile Practices Agile projects use some practices not mandated in scrum. Knowing these practices helps the tester. Some practices like TDD/BDD or Integrating testing in CICD pipeline in fact keep more focus on testing. Testers need to learn these practices well
  3. Agile mindset More than the events and artifacts, agile projects are set apart by the principles and values which the team adheres to. It is critical that the testers really need to understand how the thought process differs. An agile project will focus on getting user usable deliverable out at every sprint. This singular goal drives all activities including testing. A tester needs to now wear the hat where he helps the developer ship out defect free code – The focus now is not finding maximum defects but on ensuring defect free delivery.
  4. Feedback loop A tester in agile project works with frequently changing requirements. This necessitates a ongoing discussion between tester and business community as well as developer. The tester should continually update the business community on the testing impact the new requirements have! He also needs to work with developer to pass on insights that he has from his testing till date. These can be typical defects , areas which the tester think will have down stream impact etc. Testers have a wealth of knowledge that can enable the developer to deliver a best in class code. A tester should be conscious of his responsibility to have this constant feedback going. On the other hand, a tester needs to actively seek inputs on how his tests are impacting project. Are they helping to avoid defects in later stages? or they are just a time waste? Having this feed back loop going in both directions will help a tester to do his job efficiently
  5. Involvement In Events – I have seen cases where test team is absent from events like planning , daily scrum or the sprint review. Tester is very much a part of the whole team and needs to be an active participant in each event.
  6. Automation Automation is the word one hears often whenever agile is mentioned. A tester needs to learn not only functional automation but also need to automate test triggers.

Agile Testing:: Role Of tester

The role of a tester in an Agile team includes activities that generate and provide feedback not only on test status, test progress, and product quality, but also on process quality. These activities include:

  • Understanding, implementing, and updating the test strategy
  • Measuring and reporting test coverage across all applicable coverage dimensions o Ensuring proper use of testing tools
  • Configuring, using, and managing test environments and test data o Reporting defects and working with the team to resolve them
  • Coaching other team members in relevant aspects of testing
  • Ensuring the appropriate testing tasks are scheduled during release and iteration planning
  • Actively collaborating with developers and business stakeholders to clarify requirements, especially in terms of testability, consistency, and completeness
  • Participating proactively in team retrospectives, suggesting and implementing improvements

Within an Agile team, each team member is responsible for product quality and plays a role in performing test-related tasks.

Agile organizations may encounter some test-related organizational risks:

  • Testers work so closely to developers that they lose the appropriate tester mindset
  • Testers become tolerant of or silent about inefficient, ineffective, or low-quality practices within the team
  • Testers cannot keep pace with the incoming changes in time-constrained iterations
  • To mitigate these risks, organizations may consider variations for preserving independence.

The Role of a Tester in a Scrum Lifecycle

Throughout this study content, general reference has been made to Agile methods and techniques, and the role of a tester within various Agile lifecycles. This subsection looks specifically at the role of a tester in a project following a Scrum lifecycle .

Teamwork

Teamwork is a fundamental principle in Agile development. Agile emphasizes the whole-team approach consisting of developers, testers, and business representatives working together. The following are organizational and behavioural best practices in Scrum teams:

  • Cross-functional: Each team member brings a different set of skills to the team. The team works together on test strategy, test planning, test specification, test execution, test evaluation, and test results reporting.
  • Self-organizing: The team may consist only of developers, but ideally there would be one or more testers.
  • Co-located: Testers sit together with the developers and the product owner.
  • ·         Collaborative: Testers collaborate with their team members, other teams, the stakeholders, the product owner, and the Scrum Master.
  • Empowered: Technical decisions regarding design and testing are made by the team as a whole (developers, testers, and Scrum Master), in collaboration with the product owner and other teams if needed.
  • Committed: The tester is committed to question and evaluate the product’s behavior and characteristics with respect to the expectations and needs of the customers and users.
  • Transparent: Development and testing progress is visible on the Agile task board.
  • Credible: The tester must ensure the credibility of the strategy for testing, its implementation, and execution; otherwise the stakeholders will not trust the test results. This is often done by providing information to the stakeholders about the testing process.
  • Open to feedback: Feedback is an important aspect of being successful in any project, especially in Agile projects. Retrospectives allow teams to learn from successes and from failures.
  • Resilient: Testing must be able to respond to change, like all other activities in Agile projects.

These best practices maximize the likelihood of successful testing in Scrum projects.

Initial Set Up Activities

Like all areas of project, testers need to do some preparatory work in first iteration of the project . The tester collaborates with the team on the following activities during this iteration:

  • Identify the scope of the project (i.e., the product backlog)
  • Create an initial system architecture and high-level prototypes
  • Plan, acquire, and install needed tools (e.g., for test management, defect management, test automation, and continuous integration)
  • Create an initial test strategy for all test levels, addressing (among other topics) test scope, technical risks, test types, and coverage goals
  • Perform an initial quality risk analysis
  • Define test metrics to measure the test process, the progress of testing in the project, and product quality
  • Specify the definition of “done”
  • Create the task board
  • Define when to continue or stop testing before delivering the system to the customer
  • Sprint zero sets the direction for what testing needs to achieve and how testing needs to achieve it throughout the sprints.

Integration

In Agile projects, the objective is to deliver customer value on a continuous basis (preferably in every sprint). To enable this, the integration strategy should consider both design and testing. To enable a continuous testing strategy for the delivered functionality and characteristics, it is important to identify all dependencies between underlying functions and features.

Test Planning

Since testing is fully integrated into the Agile team, test planning should start during the release planning session and be updated during each sprint. Test planning for the release and each sprint should address the issues.

Sprint planning results in a set of tasks to put on the task board, where each task should have a length of one or two days of work. In addition, any testing issues should be tracked to keep a steady flow of testing.

Agile Testing

The Differences between Testing in Traditional and Agile Approaches

Testers must understand the differences between testing in traditional lifecycle models (e.g., sequential such as the V-model or iterative such as RUP) and Agile lifecycles in order to work effectively and efficiently. The Agile models differ in terms of the way testing and development activities are integrated, the project work products, the names, entry and exit criteria used for various levels of testing, the use of tools, and how independent testing can be effectively utilized.

Testers should remember that organizations vary considerably in their implementation of lifecycles. Deviation from the ideals of Agile lifecycles may represent intelligent customization and adaptation of the practices. The ability to adapt to the context of a given project, including the software development practices actually followed, is a key success factor for testers.

Testing and Development Activities

One of the main differences between traditional lifecycles and Agile lifecycles is the idea of very short iterations, each iteration resulting in working software that delivers features of value to business stakeholders. At the beginning of the project, there is a release planning period. This is followed by a sequence of iterations. At the beginning of each iteration, there is an iteration planning period. Once iteration scope is established, the selected user stories are developed, integrated with the system, and tested. These iterations are highly dynamic, with development, integration, and testing activities taking place throughout each iteration, and with considerable parallelism and overlap. Testing activities occur throughout the iteration, not as a final activity.

Testers, developers, and business stakeholders all have a role in testing, as with traditional lifecycles.

  • Developers perform unit tests as they develop features from the user stories.
  • Testers then test those features. Business stakeholders also test the stories during implementation.
  • Business stakeholders might use written test cases, but they also might simply experiment with and use the feature in order to provide fast feedback to the development team.

In some cases, hardening or stabilization iterations occur periodically to resolve any lingering defects and other forms of technical debt. However, the best practice is that no feature is considered done until it has been integrated and tested with the system. Another good practice is to address defects remaining from the previous iteration at the beginning of the next iteration, as part of the backlog for that iteration (referred to as “fix bugs first”).

However, some complain that this practice results in a situation where the total work to be done in the iteration is unknown and it will be more difficult to estimate when the remaining features can be done. At the end of the sequence of iterations, there can be a set of release activities to get the software ready for delivery, though in some cases delivery occurs at the end of each iteration.

When risk-based testing is used as one of the test strategies, a high-level risk analysis occurs during release planning, with testers often driving that analysis. However, the specific quality risks associated with each iteration are identified and assessed in iteration planning. This risk analysis can influence the sequence of development as well as the priority and depth of testing for the features. It also influences the estimation of the test effort required for each feature.

In some Agile practices (e.g., Extreme Programming), pairing is used. Pairing can involve testers working together in twos to test a feature. Pairing can also involve a tester working collaboratively with a developer to develop and test a feature. Pairing can be difficult when the test team is distributed, but processes and tools can help enable distributed pairing.

Testers may also serve as testing and quality coaches within the team, sharing testing knowledge and supporting quality assurance work within the team. This promotes a sense of collective ownership of quality of the product.

Test automation at all levels of testing occurs in many Agile teams, and this can mean that testers spend time creating, executing, monitoring, and maintaining automated tests and results. Because of the heavy use of test automation, a higher percentage of the manual testing on Agile projects tends to be done using experience-based and defect-based techniques such as software attacks, exploratory testing, and error guessing. While developers will focus on creating unit tests, testers should focus on creating automated integration, system, and system integration tests. This leads to a tendency for Agile teams to favour testers with a strong technical and test automation background.

One core Agile principle is that change may occur throughout the project. Therefore, lightweight work product documentation is favoured in Agile projects. Changes to existing features have testing implications, especially regression testing implications. The use of automated testing is one way of managing the amount of test effort associated with change.

However, it’s important that the rate of change not exceed the project team’s ability to deal with the risks associated with those changes.

Project Work Products

Project work products of immediate interest to Agile testers typically fall into three categories:

  • Business-oriented work products that describe what is needed (e.g., requirements specifications) and how to use it (e.g., user documentation)
  • Development work products that describe how the system is built (e.g., database entity-relationship diagrams), that actually implement the system (e.g., code), or that evaluate individual pieces of code (e.g., automated unit tests)
  • Test work products that describe how the system is tested (e.g., test strategies and plans), that actually test the system (e.g., manual and automated tests), or that present test results (e.g., test dashboards)

In a typical Agile project, it is a common practice to avoid producing vast amounts of documentation. Instead, focus is more on having working software, together with automated tests that demonstrate conformance to requirements. This encouragement to reduce documentation applies only to documentation that does not deliver value to the customer. In a successful Agile project, a balance is struck between increasing efficiency by reducing documentation and providing sufficient documentation to support business, testing, development, and maintenance activities. The team must make a decision during release planning about which work products are required and what level of work product documentation is needed.

Typical business-oriented work products on Agile projects include user stories and acceptance criteria. User stories are the Agile form of requirements specifications, and should explain how the system should behave with respect to a single, coherent feature or function.

A user story should define a feature small enough to be completed in a single iteration. Larger collections of related features, or a collection of sub-features that make up a single complex feature, may be referred to as “epics”. Epics may include user stories for different development teams. For example, one user story can describe what is required at the API-level (middleware) while another story describes what is needed at the UI-level (application). These collections may be developed over a series of sprints. Each epic and its user stories should have associated acceptance criteria.

Epics can be easily broken down into user stories by slicing them vertically and they comply to INVEST criteria. Each Epic and user story contains acceptance criteria with necessary business rules and mockups attached to them.

Typical developer work products on Agile projects include code. Agile developers also often create automated unit tests. These tests might be created after the development of code. In some cases, though, developers create tests incrementally, before each portion of the code is written, in order to provide a way of verifying, once that portion of code is written, whether it works as expected. While this approach is referred to as test first or test-driven development, in reality the tests are more a form of executable low-level design specifications rather than tests.

Typical tester work products on Agile projects include automated tests, as well as documents such as test plans, quality risk catalogs, manual tests, defect reports, and test results logs. The documents are captured in as lightweight a fashion as possible, which is often also true of these documents in traditional lifecycles. Testers will also produce test metrics from defect reports and test results logs, and again there is an emphasis on a lightweight approach.

In some Agile implementations, especially regulated, safety critical, distributed, or highly complex projects and products, further formalization of these work products is required. For example, some teams transform user stories and acceptance criteria into more formal requirements specifications. Vertical and horizontal traceability reports may be prepared to satisfy auditors, regulations, and other requirements.

In some regulated agile projects, more inspection of the documentation that is being produced during the project execution may be required.

For example: Federal drug administration (FDA) projects require lot of audit reports, strategy reports and compliance reports that need to be produced during the project execution.