An INFJ personality wielding brevity in speech and writing.
Running Non-Functional Tests In Continuous Testing Mode – Part 3
An INFJ personality wielding brevity in speech and writing.
The often stated challenges in considering non-functional tests in any model (let alone continuous testing) are:
- Non-functional tests are inconsistently defined and poorly planned
- Non-functional tests are often treated as lower priority
- Lack of suitable skillset
- Cost of tools & environment involved tend to be prohibitive
This further gets complicated when considering non-functional tests in a continuous testing context:
- Non-functional test asserts are more fragile and requires substantial maintenance in rapid iterative model
- Automated analysis tends to be more objective; whereas for most of the non-functional test we need subjective analysis
- Test environment, test data and test duration plays a key role in obtaining closer to accurate test results. Maintaining a production like environment with realistic test data for detailed non-functional tests requires mature DevOps processes and significant investment in setting up the environment
With that being said, considering automation of basic non-functional tests (would it be appropriate to call them checks?) for load, latency, throughput, security and adding them to the Continuous Testing suite helps to a great extent. The question is why? To answer it, such tests can give us early feedback on the problem areas and provides us the flexibility to choose the subset of tests we want to run.
Continuous Testing mode
SQL injection vulnerabilities, for which an example was given in the previous part of the blog is one such test to consider. We also commonly see that known weaknesses and misconfigurations such as lack of the HttpOnly flag on session cookies or use of known weak SSL suites and ciphers can be automated and run in Continuous Testing mode. Such tests can be automated using browser automation tools such as Selenium Webdriver in combination with others like OWASP ZAP.
Similarly, DB & component / service level load tests and performance regressing tests for business critical user journey can help in identifying the “performance degradation” point for the “system under construction” can be ideal candidates for continuous testing. More intense/ specific tests like Ethical hacking, capacity planning and stress testing can be designed and run based on the feedback received. In short, understanding both application technical stack & business risk, concise / limited non-functional test suite can be added to continuous test suite for frequent repeated feedback cycle; whereas targeted non-functional tests can be saved for less frequent / one time milestone.
The goal behind running non-functional tests in continuous testing mode is “to be able to detect the exact moment when someone enters a line of code that affects the non-functional aspects of the system.” To get there, we need a strong Continuous Testing model not just with the right kind of tests but also with the right processes, components and tools.
Continuous Delivery is a discipline that is successful only when those involved feel responsible and accountable. A typical Continuous Delivery pipeline looks similar to the one below from dzone.com
As you can see Continuous Testing sits within “L3 – Continuous Delivery” and as mentioned above, it is not just the kind of tests alone that make Continuous Testing successful. It is about following a process based approach that is in line with the organization’s needs and quality expectations and constantly improving it based on the results obtained.
“Continuous Improvement” is what true practitioners focus on!
,
Read more
>
Running Non-Functional Tests in Continuous Testing Mode – Part 2
Running Non-Functional Tests in Continuous Testing Mode
Similar posts
Related Posts
Kavya Ravichandran is a skilled content writer with a flair for crafting narratives that educate and engage. Driven by a love for words and an innate curiosity, she explores various topics in the digital space, focusing on application development and modernization, UI/UX design, and emerging technologies like DevOps, AI, and more. She is adept at tailoring her narratives to suit different audiences and platforms, ensuring her work is both relevant and insightful.
Pavithra Anandan is a solution driven IT specialist with over 12 years of experience, including 7 years in manual and automation testing and 5 years as a Business Analyst. She excels in understanding customer business needs and translating them into actionable requirements. Proficient in Agile methodology, she is adept at requirements gathering, epic and user story development, backlog management, and fostering collaboration with cross-functional teams. Her consulting experience spans various industries, including Postal, E-commerce, Automotive, and Airline sectors. Currently, as a Product Owner at Zuci, she focuses on advancing postal logistics by enhancing operational efficiency and driving customer satisfaction through innovative delivery solutions
Minna is a content developer specializing in software testing and Robotic Process Automation (RPA). She enjoys exploring the intricacies of cutting-edge software and knits comprehensible content that resonates with the audience. PS, she is a book lover.
Kavya Ravichandran is a skilled content writer with a flair for crafting narratives that educate and engage. Driven by a love for words and an innate curiosity, she explores various topics in the digital space, focusing on application development and modernization, UI/UX design, and emerging technologies like DevOps, AI, and more. She is adept at tailoring her narratives to suit different audiences and platforms, ensuring her work is both relevant and insightful.
An experienced and adaptable IT leader, Gopalakrishna Raju boasts over 18.5 years of expertise in service delivery management, project management, and database administration. A strong advocate for continuous service improvement and automation, he strives to bring productivity and cost benefits for clients. Certified in Oracle, AWS, and Microsoft Azure, he has received numerous accolades, including the Top Achiever FY23 Spot Award at Zensar and multiple awards at Wipro. When not busy setting up operational models, and delivering successful outcomes, he enjoys playing badminton and cricket.
Simran is a professional with over 18 years of diversified experience in business intelligence and data analytics, strategy planning, key account management and new product development. She has worked in the technology industry, consumer goods industry, retail and market research.
Gayathri Krishnan is a seasoned IT professional with over 15 years of experience, spanning 4 years in manual testing and 9+ years as a business analyst in the General Insurance and Logistics sectors. With more than 2 years as a delivery lead, she has a proven ability to manage end-to-end project lifecycles and transform business requirements into effective solutions. Her expertise covers multiple lines of business within general insurance, including Motor, Health, Personal Accident, Fire, Marine, Engineering, and Rural insurance. Skilled in Agile methodologies, Gayathri excels in requirements gathering, backlog management, client engagement, and leading cross-functional teams. As a Product Owner at Zuci, Gayathri specializes in aligning business and technical requirements to enhance operational efficiency in postal logistics services.
Sona Jayakumar is a Senior Business Analyst with three years of experience in the ESG (Environmental, Social, and Governance) and Healthcare sectors. She specializes in digital transformation and process optimization, focusing on aligning business strategies with innovative solutions. Her expertise in stakeholder management and cross-functional collaboration has consistently delivered impactful results and improved operational efficiency.
Minna is a content developer specializing in software testing and Robotic Process Automation (RPA). She enjoys exploring the intricacies of cutting-edge software and knits comprehensible content that resonates with the audience. PS, she is a book lover.
Minna is a content developer specializing in software testing and Robotic Process Automation (RPA). She enjoys exploring the intricacies of cutting-edge software and knits comprehensible content that resonates with the audience. PS, she is a book lover.
Rama Jayaraman is a Certified Public Accountant (USA), CA (Intermediate) and Commerce graduate. She is certified in Professional Scrum Master I and has working knowledge of multiple tools like Tableau, Power BI, Smartsheet, Azure Devops and multiple other Microsoft tools. She is a Seasoned Professional with 8+ years of experience with a proven track record in Project Management and Risk Management through a solution-oriented approach. She has managed and lead key projects in the areas of Automation, Analytics, Auditing, Financial Reporting and Internal Control. She has worked for companies like KPMG, Maersk and Standard Chartered Bank. Currently supporting The Officer of Inspector General (The Global Fund) as part of the Zuci family. She is a trained singer who has participated and won competitions. During her spare time, she has also volunteered as a coach and conducted swimming classes for the underprivileged and visually challenged.
Ameena Siddiqa is a seasoned marketer with hands-on experience in curating captivating content on the latest cloud, devops and enterprise technology trends. With a keen eye for emerging trends and a passion for storytelling, she has a knack for transforming complex concepts into engaging narratives that resonate with audiences across industries.
A web-analytics nerd, speaker - here delving into (Big)-data.
A web-analytics nerd, speaker - here delving into (Big)-data.
Minna is a content developer specializing in software testing and Robotic Process Automation (RPA). She enjoys exploring the intricacies of cutting-edge software and knits comprehensible content that resonates with the audience. PS, she is a book lover.
Rajalakshmi Sivaramakrishnan is a Lead Business Analyst with 17 years of experience in various fields, including Business Analysis, Identity Access Management, Requirement Engineering, and Business Intelligence. She excels in automating processes, aligning business and technology, and has domain expertise in retail banking and capital markets.
