What We Overlook When Building Our End-to-End Test Strategy

It’s a no-brainer that documenting a neatly articulated test strategy aids in the process of assuring the software quality. There are so many blogs on the subject, and there are core principles that should be followed when approaching a test strategy.

Typically, a test strategy should have:

• Testing objectives/Scope
• Clear overview of the project
• Risks associated with the project
• Testing Resources
• Test schedule

It should also help identify areas where additional testing may be required. Writing an effective test strategy is skill testers gain with experience. Hence, the possibility of missing any test activity is very low when a proper test strategy is defined.

End-End Test Strategy

End-End Test Strategy lays out the test coverage for the entire application and ensures that the flow of data is maintained for all kinds of end-user actions.

Testers should always ensure that the End-End Test Strategy covers everything a client needs and, at times, can even go one step further than required to provide them with a satisfactory product.

This article will provide an overview of what should be included in a test strategy, what we overlook when building a test strategy, and how to create one that will work for your project.

Formulating an E2E test strategy

A picture representing the components of test strategy

#Testing Scope and Overview

This should contain the main Scope of the project and the overview information about who will be accessing this document, and it includes the details such as: who will review and approve these documents.

People who will usually review the test strategy:

• Test Team Lead
• Development Manager
• Quality Analyst Manager
• Product Manager

#Testing Approach

The next component in test strategy is about the testing approach. The following might help you get started.

• Testing processes
• Role and responsibility across QA team
• Change management processes
• Types of Testing that can be presented to the client
• Test techniques
• Defect handling mechanisms
• Test sign-off

#Testing Environments

The testing environment specifies – environments in which testing types will take place and also configurations.

It should specify instructions on creating test data and the data recovery process. A weak test strategy with incomplete information can result in final deployments plagued by defects and errors.

#Testing Tools

An essential piece of a test strategy document as it bears the information on the test automation and management tools to perform test execution.

Determines the ratio of open-source and commercial tools to assure non-functional parameters like security, performance, usability, and the number of users who’ll support.

#Test Releases

This part of the test strategy defines the release management plan and the version history to ensure that all the modifications that happened in that release are included.

This lends itself to building a management process to determine where the new build should be available when it will be delivered, who will deploy it, how to stop the release in the event of errors, and so on.

#Resource Planning

As the name mentions, it determines how many resources will be involved in the testing process and their responsibilities for the task assigned to them.

#Risk Analysis

Risk analysis – to estimate the possible risks that could occur during test execution and a clear plan to mitigate the risk and a contingency plan when risks happen in real-time.

#Reviews and Approvals

This is the last piece of information in the test strategy document where the project management and development team reviews all these activities and sign off.

A summary of review changes should be traced at the beginning of the document, along with an approved date, name, and comment.

The next section walks you through a different discussion on test strategy by Harini Mohan, our Test Lead.

What do testers overlook when building a test strategy?

Test estimation and test planning involve a lot of dependencies which may cause variance in test estimation.

When building a test strategy, managers and testers should be aware of the coordination and miscommunication challenges and plan a suitable strategy for a project.

Our Senior test engineer, Harini Mohan, shares some of the factors that test managers overlook when building a test strategy.

Changing requirements

An application with changing requirements causes inaccurate test estimates. It’s imperative to have in place a good change management process to keep track of the requirements to minimize business risks.

A screenshot highlighting the change management practices of our client:

No test data/Wrong test data

Test data is the most pre-requisite to be considered before starting the test execution. Some projects don’t have proper test data, resulting in chaos in testing.

Therefore, testers must continuously explore, learn and apply the most efficient approaches for data collection, generation, maintenance, automation, and comprehensive data management for any functional and non-functional testing type.

Project communication/coordination

Some project managers forget to allocate time for collaboration in their test strategy, which, especially in an offshore/onsite model, would result in more effort being spent on the project than planned.

Ambiguity in planning

Lack or ambiguity during the planning stage sets up for a failure upfront. When enough care is not put in the planning stage, there’s a possibility of arriving at an inefficient test strategy resulting in investing a lot of time and effort in the wrong places and high-risk areas getting left out.

Test strategy in agile

In an agile world where testers work on multiple projects/user stories in two-week sprints, it’s natural to think if we need a test strategy for such short sprints. If we create a test strategy document for each project, a lot of time would be spent iterating the documentation.

So, we identified some questions from testing communities on this topic and tried to answer them.

Disclaimer: You do you and what’s best for your team

Is a “per project” test strategy document the right approach in an agile environment?

In the Waterfall environment, you must be familiar with all the heavy documentation associated with the testing process. You are aware of the format and content of those documents, and as we know, it is a hugely time-consuming task to develop all those documents.

But the strategy you put together describes your test approaches, your test reporting methods, your strategy for managing environments, your strategy for reporting bugs, key stakeholders and decision-makers, etc.

However, a lean test strategy should work for both waterfall and agile models. The document can be for the whole project instead of sprints in the agile model. Only then does it make sense for the team leaders or managers to have enough information, make plans for test designs, etc.

Would a global test strategy document be the right approach which is more of a static company-wide test strategy document, acting more as governance, as opposed to doing this for each project?

The idea of a global document needs to be revised, as the teams should be able to decide what is appropriate for their project at any point.

 Is a test strategy document even necessary at all?

OfCourse Yes! Every project must have a test strategy as it helps set up the testing framework for any project.

Reference

https://club.ministryoftesting.com/t/test-strategy-documentation-in-an-agile-world/10724