{"id":79532,"date":"2026-10-28T05:43:40","date_gmt":"2026-10-28T05:43:40","guid":{"rendered":"https:\/\/www.devopsschool.com\/blog\/?p=79532"},"modified":"2026-10-28T05:43:42","modified_gmt":"2026-10-28T05:43:42","slug":"software-testing-and-quality-maturity-the-complete-tutorial-series","status":"publish","type":"post","link":"https:\/\/www.devopsschool.com\/blog\/software-testing-and-quality-maturity-the-complete-tutorial-series\/","title":{"rendered":"Software Testing and Quality Maturity: The Complete Tutorial Series"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Seven in-depth technical guides covering frameworks #11\u2013#17 \u00b7 Beginner to Advanced \u00b7 Software QA, Quality Engineering, DevSecOps, and Process Improvement<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Software quality is not simply about executing test cases, finding bugs, or achieving a high automation percentage. A mature quality engineering organization systematically defines quality, prevents defects, measures risks, improves testing practices, and provides objective evidence that its products are fit for purpose.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The seven frameworks in this series address different parts of that challenge.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>#<\/th><th>Framework<\/th><th>Primary purpose<\/th><\/tr><tr><td>11<\/td><td>TMMi<\/td><td>Assess and improve organizational testing maturity<\/td><\/tr><tr><td>12<\/td><td>TPI NEXT<\/td><td>Identify and prioritize practical test process improvements<\/td><\/tr><tr><td>13<\/td><td>ISO\/IEC\/IEEE 29119<\/td><td>Standardize software testing processes, techniques, and documentation<\/td><\/tr><tr><td>14<\/td><td>ISO\/IEC 25010<\/td><td>Define and evaluate software product quality<\/td><\/tr><tr><td>15<\/td><td>IEEE 730<\/td><td>Establish software quality assurance processes and governance<\/td><\/tr><tr><td>16<\/td><td>Six Sigma<\/td><td>Reduce process variation, recurring defects, and waste<\/td><\/tr><tr><td>17<\/td><td>CISQ<\/td><td>Measure software structural quality and technical risk<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Important standards updates: TMMi released Model 2.0 in June 2026, adding updated Agile, DevOps, quality engineering, and AI guidance. IEEE 730-2026 was published in August 2026 and supersedes IEEE 730-2014. ISO\/IEC 25010:2023 uses nine product quality characteristics, rather than the eight in its earlier edition. These distinctions matter when using published tutorials as implementation or assessment references. TMMi+2<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The tutorials below use a fictional software product throughout, so the examples connect to a realistic engineering workflow. Implementation checklists, sample targets, and templates are practical guidance\u2014not quotations from, or substitutes for, the full licensed standards.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tutorial 11 \u2014 TMMi: A Complete Guide to Test Maturity Model Integration<\/h2>\n\n\n\n<h1 class=\"wp-block-heading\">TMMi Explained: How to Assess, Improve, and Mature Your Software Testing Process<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">A software development team can have hundreds of automated tests, experienced QA engineers, and sophisticated CI\/CD pipelines\u2014and still struggle with software quality.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Why? Because testing maturity is not determined by the number of tests or the tools an organization owns. It depends on whether testing activities are planned, repeatable, measurable, integrated into development, and continuously improved.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Test Maturity Model integration (TMMi)<\/strong> provides a structured approach to developing these capabilities.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">1. What is TMMi?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">TMMi is a test process improvement and assessment model maintained by the TMMi Foundation. It provides maturity levels, process areas, goals, and practices that help organizations understand how effectively they manage software testing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">TMMi is especially useful for:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Software development organizations operating multiple teams.<\/li>\n\n\n\n<li>QA and quality engineering departments.<\/li>\n\n\n\n<li>Enterprises introducing Agile and DevOps.<\/li>\n\n\n\n<li>Organizations seeking consistent testing governance.<\/li>\n\n\n\n<li>Outsourcing organizations evaluating suppliers.<\/li>\n\n\n\n<li>Engineering teams preparing for formal test maturity assessments.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">TMMi Model 2.0 incorporates contemporary Agile and DevOps practices, quality engineering concepts, and guidance related to artificial intelligence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The model is <strong>technology-neutral<\/strong>. It does not mandate Selenium, Playwright, Jenkins, or a specific test management platform.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2. Understanding the five TMMi maturity levels<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Level<\/th><th>Name<\/th><th>Meaning<\/th><\/tr><tr><td>1<\/td><td>Initial<\/td><td>Testing practices are inconsistent, reactive, or dependent on individual effort.<\/td><\/tr><tr><td>2<\/td><td>Managed<\/td><td>Teams establish testing policies, planning, controls, and repeatable execution.<\/td><\/tr><tr><td>3<\/td><td>Defined<\/td><td>Consistent organizational testing practices are established and integrated with development.<\/td><\/tr><tr><td>4<\/td><td>Measured<\/td><td>Testing and product quality are managed using quantitative objectives and measurement.<\/td><\/tr><tr><td>5<\/td><td>Optimization<\/td><td>The organization continuously improves its practices through defect prevention and process optimization.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Level 1 is the starting point and has no required process areas. Progression through the other levels depends on demonstrating that their applicable process goals have been achieved.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Level 2: Managed<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">At this level, testing stops being an informal activity performed immediately before release.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The six process areas in TMMi 2.0 are:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Test Policy and Strategy<\/li>\n\n\n\n<li>Test Planning<\/li>\n\n\n\n<li>Test Monitoring and Control<\/li>\n\n\n\n<li>Test Design and Execution<\/li>\n\n\n\n<li>Test Environment<\/li>\n\n\n\n<li>Implementation and Habit<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Practical example:<\/strong> An API development team introduces a risk-based test strategy, defines test environments, records test results, identifies accountable owners, and monitors unresolved defects before deployment.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Level 3: Defined<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Testing practices become consistent across the organizational unit.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The five process areas are:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Test Organization<\/li>\n\n\n\n<li>Test Training Program<\/li>\n\n\n\n<li>Test Process and Integration<\/li>\n\n\n\n<li>Non-Functional Testing<\/li>\n\n\n\n<li>Peer Reviews<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">At this stage, teams share organizational testing guidance while retaining the ability to tailor their implementation to different products.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Non-functional testing\u2014including performance, reliability, and security-related testing\u2014becomes an organized part of engineering work rather than an occasional exercise.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Level 4: Measured<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The two process areas are:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Test Measurement<\/li>\n\n\n\n<li>Product Quality Evaluation<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Organizations establish meaningful quantitative objectives and use collected evidence to evaluate testing effectiveness and product quality.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The distinction is important: merely collecting dashboards does not make a testing process quantitatively managed.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Level 5: Optimization<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The three process areas are:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Defect Prevention<\/li>\n\n\n\n<li>Quality Control<\/li>\n\n\n\n<li>Test Process Optimization<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">The focus moves beyond finding defects to understanding why they occur and reducing their recurrence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">TMMi 2.0 therefore covers <strong>16 process areas across Levels 2\u20135<\/strong>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">3. How to implement TMMi step by step<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Consider a fictional connected-vehicle platform called FleetLink. The product includes a mobile application, cloud APIs, and vehicle-status synchronization services.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The engineering organization wants fewer production incidents and more predictable releases.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 1: Establish the assessment scope<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Decide whether the initiative covers one delivery team, a business unit, or the broader engineering organization.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Document:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Systems and services in scope.<\/li>\n\n\n\n<li>Teams and development locations.<\/li>\n\n\n\n<li>Product criticality and customer expectations.<\/li>\n\n\n\n<li>Applicable external requirements.<\/li>\n\n\n\n<li>Existing test tools and test environments.<\/li>\n\n\n\n<li>Assessment sponsor and improvement owners.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Deliverable:<\/strong> A test maturity improvement charter.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A small pilot is often more manageable than attempting to standardize every team simultaneously.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 2: Establish the testing baseline<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Collect evidence of how testing currently works.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Assessment topic<\/th><th>Evidence to inspect<\/th><th>Example weakness<\/th><\/tr><tr><td>Test strategy<\/td><td>Approved strategy and risk classification<\/td><td>No risk-based testing<\/td><\/tr><tr><td>Test planning<\/td><td>Sprint and release test plans<\/td><td>Testing starts too late<\/td><\/tr><tr><td>Test execution<\/td><td>Test logs, CI results, defect reports<\/td><td>Inconsistent results<\/td><\/tr><tr><td>Test environment<\/td><td>Environment ownership and configuration<\/td><td>Unreliable environments<\/td><\/tr><tr><td>Test measurement<\/td><td>Defect and quality trends<\/td><td>Metrics without definitions<\/td><\/tr><tr><td>Governance<\/td><td>Responsibilities and process reviews<\/td><td>No accountable owner<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Use interviews alongside evidence from engineering repositories, CI systems, documentation, and issue tracking tools.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Do not assign an official TMMi maturity rating from a homemade checklist.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 3: Define a testing policy and strategy<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A useful policy answers four questions:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>What quality risks matter most?<\/li>\n\n\n\n<li>What testing is required?<\/li>\n\n\n\n<li>Who is responsible?<\/li>\n\n\n\n<li>What evidence supports a release decision?<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">For example, FleetLink may classify its API operations into critical, high, medium, and low-risk categories.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Critical operations might require additional integration testing, failure recovery tests, security analysis, and documented release approval.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 4: Introduce risk-based test planning<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Each feature should have an identified test approach based on business impact and failure likelihood.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A simple risk-prioritization method is:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>Risk score = Likelihood \u00d7 Impact<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Feature<\/th><th>Likelihood (1\u20135)<\/th><th>Impact (1\u20135)<\/th><th>Score<\/th><\/tr><tr><td>Vehicle-status synchronization<\/td><td>4<\/td><td>4<\/td><td>16<\/td><\/tr><tr><td>Notification preferences<\/td><td>3<\/td><td>2<\/td><td>6<\/td><\/tr><tr><td>Account authorization<\/td><td>3<\/td><td>5<\/td><td>15<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This is an illustrative prioritization technique, not a prescribed TMMi scoring formula.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Risk classification should influence test depth, test environments, release acceptance, and review requirements.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 5: Improve test design and execution<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Create a repeatable testing workflow:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Analyze requirements and relevant risks.<\/li>\n\n\n\n<li>Design test cases using suitable test techniques.<\/li>\n\n\n\n<li>Prepare representative environments and controlled test data.<\/li>\n\n\n\n<li>Execute automated and exploratory tests.<\/li>\n\n\n\n<li>Record results and investigate failures.<\/li>\n\n\n\n<li>Resolve defects and perform regression testing.<\/li>\n\n\n\n<li>Report residual risks before release.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">A test is not adequately evidenced merely because a CI job displays a green status.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Retain enough information to identify the tested software version, environment, test data, execution outcome, and relevant defects.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 6: Establish useful quality metrics<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Start with a small set of meaningful indicators.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Defect escape rate<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>Escaped defects \/ (Pre-release defects + Escaped defects) \u00d7 100<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Use defects discovered within a clearly defined release and observation window.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Critical requirement coverage<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>Critical requirements with accepted verification evidence \/ Total critical requirements \u00d7 100<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Test automation stability<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Track inconsistent test outcomes, environment failures, and recurring false alarms.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Defect resolution time<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Measure elapsed time from defect reporting to verified resolution, preferably segmented by severity.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Change failure rate<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Track the proportion of production changes causing defined failures requiring remediation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Never use a single metric to declare product quality acceptable.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 7: Build continuous improvement into operations<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Review testing weaknesses regularly. Assign owners to improvement actions and establish measurable outcomes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Problem: Many tests fail because test data is inconsistent.<\/li>\n\n\n\n<li>Root cause: Teams share mutable integration test accounts.<\/li>\n\n\n\n<li>Improvement: Generate isolated test data for each test run.<\/li>\n\n\n\n<li>Validation: Compare test instability before and after implementation.<\/li>\n\n\n\n<li>Sustainment: Add ownership and regular monitoring.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">This moves the organization toward sustainable maturity.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. Example TMMi implementation roadmap<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Period<\/th><th>Activities<\/th><th>Deliverables<\/th><\/tr><tr><td>Days 1\u201315<\/td><td>Define scope, interview teams, inspect evidence<\/td><td>Initial gap assessment<\/td><\/tr><tr><td>Days 16\u201330<\/td><td>Establish policy, risk categories, ownership<\/td><td>Test policy and strategy<\/td><\/tr><tr><td>Days 31\u201345<\/td><td>Improve planning, environments, and execution evidence<\/td><td>Repeatable test workflow<\/td><\/tr><tr><td>Days 46\u201360<\/td><td>Integrate testing with CI\/CD and peer reviews<\/td><td>Quality gates and review process<\/td><\/tr><tr><td>Days 61\u201375<\/td><td>Implement metrics and dashboards<\/td><td>Measurement baseline<\/td><\/tr><tr><td>Days 76\u201390<\/td><td>Verify improvements and plan next cycle<\/td><td>Evidence-based improvement roadmap<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This is a suggested improvement schedule, not a guaranteed timeframe for reaching or certifying a maturity level.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">5. Recommended supporting tools<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Purpose<\/th><th>Example tools<\/th><\/tr><tr><td>Test automation<\/td><td>Playwright, pytest, Robot Framework<\/td><\/tr><tr><td>CI\/CD<\/td><td>Jenkins, GitHub Actions, GitLab CI\/CD<\/td><\/tr><tr><td>Performance testing<\/td><td>k6, Apache JMeter<\/td><\/tr><tr><td>Static analysis<\/td><td>Semgrep, SonarQube<\/td><\/tr><tr><td>Test reporting<\/td><td>Allure Report, JUnit XML<\/td><\/tr><tr><td>Defect tracking<\/td><td>GitHub Issues, OpenProject<\/td><\/tr><tr><td>Documentation<\/td><td>Git-based Markdown, internal knowledge bases<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">TMMi evaluates the effectiveness of testing practices, not the popularity of the tools selected.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">6. Informal assessment versus formal assessment<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">An informal assessment helps identify improvement opportunities and provides an indicative view of testing maturity.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A formal assessment involves additional rigor, corroborating evidence, and accredited assessment practices. Formal maturity ratings and recognized certification follow the TMMi Foundation&#8217;s assessment requirements.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Organizations should pursue formal assessment when customer, contractual, supplier, or strategic requirements justify it.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">7. Common implementation mistakes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Treating automation coverage as maturity.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Automation supports testing. It does not replace governance, risk management, process integration, and defect prevention.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Creating documentation that nobody follows.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A lightweight, consistently applied process is more useful than a large collection of abandoned procedures.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Targeting Level 5 immediately.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Improvement should build on stable foundational practices.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Focusing only on QA engineers.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Software quality depends on developers, product owners, operations, security engineers, and business stakeholders.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">8. How to measure success<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A successful TMMi improvement program should demonstrate better risk visibility, reliable test execution, more predictable quality decisions, and measurable reductions in important quality failures.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The ultimate objective is not to achieve a maturity label. It is to develop a testing capability that consistently supports business and engineering objectives.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">TMMi provides a structured way to transform inconsistent testing activities into a managed, measurable, continuously improving quality engineering capability.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Start with objective evidence, establish Level 2 fundamentals, integrate testing into development, and gradually expand quantitative management and defect prevention.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Official resources<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.tmmi.org\/tmmi-documents\/\">TMMi Foundation \u2014 Framework, Model 2.0 and assessment resources<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.tmmi.org\/tmmi-model\/\">TMMi Foundation \u2014 Maturity model<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.tmmi.org\/assessment-types\/\">TMMi Foundation \u2014 Assessment types<\/a><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Verified against the TMMi Foundation&#8217;s Model 2.0 and assessment guidance. TMMi Framework model V2.0+2<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tutorial 12 \u2014 TPI NEXT: Business-Driven Test Process Improvement<\/h2>\n\n\n\n<h1 class=\"wp-block-heading\">TPI NEXT Tutorial: Assess Your Testing Process, Find Weaknesses, and Build an Improvement Roadmap<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">Every software engineering organization experiences testing bottlenecks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Some teams struggle with unreliable environments. Others have unclear testing responsibilities, inconsistent defect reporting, insufficient test planning, or poor coordination between developers and testers.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These problems do not necessarily require a complete organizational transformation. Often, targeted improvements in a few important areas produce meaningful results.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>TPI NEXT (Test Process Improvement NEXT)<\/strong> provides an approach for identifying these weaknesses and improving testing capabilities in a controlled, business-driven manner.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">1. What is TPI NEXT?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">TPI NEXT is a test process improvement model developed by Sogeti. It enables organizations to assess different aspects of testing and prioritize improvements according to business objectives.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Unlike a model that concentrates primarily on one organization-wide maturity level, TPI NEXT provides a granular view of individual testing capabilities.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An organization may have sophisticated defect management while its test environments remain poorly controlled.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">TPI NEXT helps make that imbalance visible.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its principal elements include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Key areas<\/li>\n\n\n\n<li>Maturity levels<\/li>\n\n\n\n<li>Checkpoints<\/li>\n\n\n\n<li>Improvement suggestions<\/li>\n\n\n\n<li>Enablers<\/li>\n\n\n\n<li>Clusters<\/li>\n\n\n\n<li>Test maturity matrix<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Business drivers such as quality, delivery speed, cost, and operational risk influence improvement priorities.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2. Understand the 16 TPI NEXT key areas<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The model contains 16 key areas spanning stakeholder relationships, test management, and testing practices.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>#<\/th><th>Key area<\/th><th>Main concern<\/th><\/tr><tr><td>1<\/td><td>Stakeholder commitment<\/td><td>Management and business support for effective testing<\/td><\/tr><tr><td>2<\/td><td>Degree of involvement<\/td><td>Integration of testing into development activities<\/td><\/tr><tr><td>3<\/td><td>Test strategy<\/td><td>Testing objectives, priorities, and risk-based approach<\/td><\/tr><tr><td>4<\/td><td>Test organization<\/td><td>Test responsibilities and organizational structure<\/td><\/tr><tr><td>5<\/td><td>Communication<\/td><td>Effective communication between stakeholders<\/td><\/tr><tr><td>6<\/td><td>Reporting<\/td><td>Communication of progress, results, and risks<\/td><\/tr><tr><td>7<\/td><td>Test process management<\/td><td>Coordinating and controlling test activities<\/td><\/tr><tr><td>8<\/td><td>Estimating and planning<\/td><td>Reliable test effort and resource planning<\/td><\/tr><tr><td>9<\/td><td>Metrics<\/td><td>Objective measurement of testing and quality<\/td><\/tr><tr><td>10<\/td><td>Defect management<\/td><td>Defect identification, tracking, and resolution<\/td><\/tr><tr><td>11<\/td><td>Testware management<\/td><td>Control of testing artifacts and their relationships<\/td><\/tr><tr><td>12<\/td><td>Methodology practice<\/td><td>Consistent practical application of test methods<\/td><\/tr><tr><td>13<\/td><td>Tester professionalism<\/td><td>Competencies and skills required for testing<\/td><\/tr><tr><td>14<\/td><td>Test case design<\/td><td>Appropriate methods for identifying effective test cases<\/td><\/tr><tr><td>15<\/td><td>Test tools<\/td><td>Selection and effective use of supporting tools<\/td><\/tr><tr><td>16<\/td><td>Test environment<\/td><td>Availability, suitability, and control of test environments<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">A key advantage is that the model highlights capabilities that are sometimes neglected when improvement programs focus exclusively on automation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, better test tools will not solve a fundamental lack of stakeholder involvement.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">3. The TPI NEXT maturity levels<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Each key area can progress through four maturity levels.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Level<\/th><th>Description<\/th><\/tr><tr><td>Initial<\/td><td>Activities are informal or insufficiently controlled.<\/td><\/tr><tr><td>Controlled<\/td><td>The capability is established and performed consistently.<\/td><\/tr><tr><td>Efficient<\/td><td>The capability operates effectively with better use of resources.<\/td><\/tr><tr><td>Optimizing<\/td><td>Continuous improvement sustains and enhances the capability.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">TPI NEXT uses checkpoints to assess whether the relevant practices are established.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A key area&#8217;s maturity is determined by fulfilling the applicable checkpoints, not by assigning an arbitrary numerical average.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What are checkpoints?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Checkpoints describe conditions that should exist in the testing process.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, an assessment may inspect whether a team has a documented testing approach, communicates material testing risks, or maintains controlled test environments.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Assessors examine evidence to determine whether those conditions are actually met.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What are clusters?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Clusters group related improvements across multiple key areas into a sensible progression.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">They help prevent teams from implementing advanced practices before the supporting fundamentals exist.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What are enablers?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Enablers highlight relationships between testing and adjacent development processes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, effective automated regression testing depends on suitable deployment mechanisms, test data, environment stability, and development integration.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Improving those dependencies can make testing more effective.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. Step-by-step TPI NEXT implementation<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Step 1: Identify the business problem<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Do not begin by asking, &#8220;How do we achieve the highest maturity level?&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Begin with a real business concern.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Examples:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Production defects are increasing.<\/li>\n\n\n\n<li>Releases repeatedly miss planned dates.<\/li>\n\n\n\n<li>Testing costs are rising.<\/li>\n\n\n\n<li>Developers wait too long for test environments.<\/li>\n\n\n\n<li>Customers experience regressions after deployments.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">For this tutorial, assume FleetLink experiences frequent delays because integration test environments are unreliable.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 2: Define the assessment scope<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Select the relevant team, product, project, or organizational unit.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Identify the systems, development processes, responsible stakeholders, and available evidence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Make sure the assessment covers the work that directly affects the identified business objective.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 3: Assess the current capabilities<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Use the official TPI NEXT maturity matrix and checkpoint guidance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A simplified diagnostic summary might look like this:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Key area<\/th><th>Illustrative current state<\/th><th>Desired improvement<\/th><\/tr><tr><td>Stakeholder commitment<\/td><td>Controlled<\/td><td>Faster decisions on critical risks<\/td><\/tr><tr><td>Test strategy<\/td><td>Controlled<\/td><td>More effective risk prioritization<\/td><\/tr><tr><td>Estimating and planning<\/td><td>Initial<\/td><td>Repeatable release test estimates<\/td><\/tr><tr><td>Metrics<\/td><td>Initial<\/td><td>Consistent definitions and reporting<\/td><\/tr><tr><td>Defect management<\/td><td>Controlled<\/td><td>Faster root cause identification<\/td><\/tr><tr><td>Test environment<\/td><td>Initial<\/td><td>Stable, reproducible environments<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This is a practical summary for planning. It is not a substitute for completing the official checkpoint assessment.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 4: Perform evidence-based gap analysis<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">For each weakness, examine the symptoms, contributing factors, and business consequences.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Consider the test environment problem:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Observed symptom:<\/strong> Integration tests frequently fail during deployment.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Evidence:<\/strong> CI logs show failed connections to services that are intermittently unavailable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Possible contributing factors:<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Shared infrastructure without clear ownership.<\/li>\n\n\n\n<li>Test data overwritten by parallel execution.<\/li>\n\n\n\n<li>Environments not provisioned consistently.<\/li>\n\n\n\n<li>Configuration drift.<\/li>\n\n\n\n<li>Missing environment health checks.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Do not treat these hypotheses as confirmed causes until investigation supports them.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 5: Prioritize the improvements<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A simple workshop prioritization method can consider:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Business impact<\/li>\n\n\n\n<li>Urgency<\/li>\n\n\n\n<li>Implementation effort<\/li>\n\n\n\n<li>Dependencies<\/li>\n\n\n\n<li>Confidence in the proposed solution<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">For example:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Improvement<\/th><th>Business impact<\/th><th>Effort<\/th><th>Priority<\/th><\/tr><tr><td>Make CI test environments reproducible<\/td><td>High<\/td><td>Medium<\/td><td>First<\/td><\/tr><tr><td>Isolate automated test data<\/td><td>High<\/td><td>Low<\/td><td>First<\/td><\/tr><tr><td>Introduce test result trend reporting<\/td><td>Medium<\/td><td>Low<\/td><td>Second<\/td><\/tr><tr><td>Expand UI automation coverage<\/td><td>Medium<\/td><td>High<\/td><td>Later<\/td><\/tr><tr><td>Implement advanced defect prediction<\/td><td>Uncertain<\/td><td>High<\/td><td>Investigate<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">These priorities are illustrative. The official TPI NEXT clusters and checkpoints should guide the actual improvement sequence.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 6: Implement targeted changes<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">For the FleetLink example:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Establish environment ownership.<\/li>\n\n\n\n<li>Define standardized environment configurations.<\/li>\n\n\n\n<li>Provision consistent test dependencies.<\/li>\n\n\n\n<li>Introduce health checks before test execution.<\/li>\n\n\n\n<li>Isolate test data between concurrent runs.<\/li>\n\n\n\n<li>Record environmental failures separately from application defects.<\/li>\n\n\n\n<li>Monitor reliability over multiple release cycles.<\/li>\n<\/ol>\n\n\n\n<h3 class=\"wp-block-heading\">Step 7: Reassess and measure benefits<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Compare outcomes using consistent definitions and comparable periods.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Useful measurements include:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Environment-related failure percentage<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>Environment-related failed runs \/ Total eligible runs \u00d7 100<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Test cycle duration<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Elapsed time between test execution beginning and completion.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Test execution predictability<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Difference between estimated and actual effort or elapsed time, interpreted in context.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Defect escape rate<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The proportion of discovered defects first identified after release, using a consistent observation window.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If an improvement reduces environmental failures but production regressions remain unchanged, the next improvement cycle should target another limiting capability.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">5. Example improvement plan<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Phase<\/th><th>Main activity<\/th><th>Outcome<\/th><\/tr><tr><td>Assess<\/td><td>Review key areas and checkpoints<\/td><td>Current maturity matrix<\/td><\/tr><tr><td>Diagnose<\/td><td>Analyze bottlenecks and dependencies<\/td><td>Root cause hypotheses<\/td><\/tr><tr><td>Prioritize<\/td><td>Apply business drivers and cluster guidance<\/td><td>Improvement backlog<\/td><\/tr><tr><td>Implement<\/td><td>Introduce selected improvements<\/td><td>Updated working practices<\/td><\/tr><tr><td>Measure<\/td><td>Evaluate outcomes<\/td><td>Evidence of improvement<\/td><\/tr><tr><td>Repeat<\/td><td>Reassess affected key areas<\/td><td>Updated improvement roadmap<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">TPI NEXT should operate as a continuous improvement activity rather than a one-time audit.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">6. Applying TPI NEXT in Agile and DevOps<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Modern teams can implement TPI NEXT practices without introducing extensive manual documentation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Examples include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Test strategies stored as version-controlled Markdown.<\/li>\n\n\n\n<li>Quality requirements included in backlog refinement.<\/li>\n\n\n\n<li>Automated tests integrated into CI\/CD.<\/li>\n\n\n\n<li>Defect trends discussed during retrospectives.<\/li>\n\n\n\n<li>Test environment configuration managed through infrastructure as code.<\/li>\n\n\n\n<li>Improvement activities tracked as engineering backlog items.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Sogeti also provides <strong>Agile Test Practice Improvement (Agile TPI)<\/strong> guidance for adapting improvement assessments to Agile practices.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Agile TPI is related to TPI NEXT but uses an adapted assessment structure, so its maturity presentation should not be confused with the original TPI NEXT model.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">7. TPI NEXT versus TMMi<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Dimension<\/th><th>TPI NEXT<\/th><th>TMMi<\/th><\/tr><tr><td>Main orientation<\/td><td>Targeted, business-driven test process improvement<\/td><td>Structured organizational test maturity<\/td><\/tr><tr><td>Assessment structure<\/td><td>Individual key areas and checkpoints<\/td><td>Staged maturity levels and process areas<\/td><\/tr><tr><td>Improvement mechanism<\/td><td>Business drivers, clusters, enablers<\/td><td>Process goals and maturity progression<\/td><\/tr><tr><td>Typical starting point<\/td><td>Specific testing bottlenecks<\/td><td>Organizational testing capability baseline<\/td><\/tr><tr><td>Useful outcome<\/td><td>Prioritized improvement roadmap<\/td><td>Defined maturity improvement path<\/td><\/tr><tr><td>Formal TMMi certification<\/td><td>Not provided by TPI NEXT<\/td><td>Available through the TMMi assessment framework<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The models are complementary. TPI NEXT can identify focused improvements while TMMi provides a broader organizational maturity direction.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">8. Common mistakes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Improving every key area simultaneously.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Prioritize based on business impact and dependencies.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Treating maturity as a percentage score.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The official checkpoints and maturity rules must be used when representing actual TPI NEXT maturity.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Ignoring development processes.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Testing quality depends on requirements, architecture, code review, configuration management, deployment, and operational feedback.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Measuring only activity.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">More test cases and more execution hours do not necessarily mean greater testing effectiveness.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">TPI NEXT offers a practical way to identify testing weaknesses, understand their relationships, and improve them in a business-driven order.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Organizations achieve the greatest value when they connect testing improvements to meaningful outcomes such as lower production risk, faster feedback, and more predictable delivery.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Official resources<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.tmap.net\/building-blocks\/test-process-improvement-tpi\/\">TMAP \u2014 Test Process Improvement and TPI NEXT tools<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.tmap.net\/book\/tpi-next\/\">TMAP \u2014 TPI NEXT methodology<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.tmap.net\/building-blocks\/test-process-improvement-tpi\/\">TMAP \u2014 Agile Test Practice Improvement<\/a><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Verified against Sogeti\/TMAP&#8217;s TPI NEXT model and its published 16-key-area guidance. TMAP+2<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tutorial 13 \u2014 ISO\/IEC\/IEEE 29119: Complete Software Testing Standards Guide<\/h2>\n\n\n\n<h1 class=\"wp-block-heading\">ISO\/IEC\/IEEE 29119 Explained: Software Testing Processes, Documentation, Techniques, and Practical Implementation<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">Software testing becomes difficult to manage when every team follows different conventions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">One team creates detailed test plans, another relies on informal discussions, and a third maintains automated tests without clear traceability to requirements.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These differences create challenges in test governance, communication, evidence collection, and release decisions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>ISO\/IEC\/IEEE 29119<\/strong> provides an internationally recognized family of software testing standards that addresses these problems.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It defines common concepts, test processes, documentation, and testing techniques that can be applied across different organizations, technologies, and development methodologies.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">1. What is ISO\/IEC\/IEEE 29119?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ISO\/IEC\/IEEE 29119 is a standards series for software and systems testing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its purpose is to establish a consistent foundation for planning, designing, executing, managing, and documenting software testing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The series supports traditional development, Agile, DevOps, and other lifecycle approaches.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It does not prescribe one testing tool or require that all test information be maintained in separate Word documents.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Organizations can use appropriately managed electronic records, version-controlled documents, test management systems, and CI\/CD evidence.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2. Understand the different parts<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Standard<\/th><th>Subject<\/th><th>Main purpose<\/th><\/tr><tr><td>29119-1:2022<\/td><td>General concepts<\/td><td>Establish common testing terminology and concepts<\/td><\/tr><tr><td>29119-2:2021<\/td><td>Test processes<\/td><td>Define organizational, management, and dynamic testing processes<\/td><\/tr><tr><td>29119-3:2021<\/td><td>Test documentation<\/td><td>Specify testing documentation templates<\/td><\/tr><tr><td>29119-4:2021<\/td><td>Test techniques<\/td><td>Define systematic test design techniques<\/td><\/tr><tr><td>29119-5:2024<\/td><td>Keyword-driven testing<\/td><td>Define a structured approach to keyword-based test specification and automation<\/td><\/tr><tr><td>ISO\/IEC TR 29119-6:2021<\/td><td>Agile testing guidance<\/td><td>Explain application of the series in Agile projects<\/td><\/tr><tr><td>ISO\/IEC TR 29119-11:2020<\/td><td>Testing AI-based systems<\/td><td>Provide guidance for challenges specific to AI testing<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Parts 1\u20135 form the core of the series. The technical reports provide additional application guidance.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Part 1: General concepts<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Part 1 establishes terminology and fundamental ideas used throughout the series.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These include test plans, strategies, test design, execution, defect management, and communication.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Part 2: Test processes<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Part 2 defines processes covering:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Organizational test governance<\/li>\n\n\n\n<li>Test management<\/li>\n\n\n\n<li>Dynamic test implementation<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">It provides a process-based approach rather than treating testing as an isolated execution activity.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Part 3: Test documentation<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Part 3 describes templates for test documentation produced by the testing processes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Examples of useful testing records include plans, specifications, execution evidence, incident information, status reports, and completion information.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Part 4: Test techniques<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Part 4 addresses systematic techniques for deriving test cases.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Instead of choosing test inputs randomly, engineers use an appropriate test model and defined techniques to produce meaningful coverage.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Part 5: Keyword-driven testing<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Part 5 describes an approach in which test specifications use reusable keywords representing test operations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This supports modular testing and automation frameworks.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">3. Implement ISO 29119 step by step<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Consider an API that accepts a vehicle battery charge percentage and publishes it to a companion application.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The input must be numeric and lie between 0 and 100, inclusive.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 1: Define the testing objectives<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Before execution, identify:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Functional requirements.<\/li>\n\n\n\n<li>Security and privacy expectations.<\/li>\n\n\n\n<li>Reliability and performance requirements.<\/li>\n\n\n\n<li>Applicable regulatory or contractual constraints.<\/li>\n\n\n\n<li>Test boundaries and exclusions.<\/li>\n\n\n\n<li>Required evidence and acceptance criteria.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">For the battery-status API:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Requirement BAT-001:<\/strong> The service must accept valid numeric charge percentages between 0 and 100.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Requirement BAT-002:<\/strong> The service must reject invalid charge values.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Requirement BAT-003:<\/strong> Failed validation must not update the stored battery status.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These requirements create testable expectations.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 2: Establish a test strategy<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Determine which types of testing are appropriate.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Test type<\/th><th>Objective<\/th><\/tr><tr><td>Unit testing<\/td><td>Validate input processing logic<\/td><\/tr><tr><td>Integration testing<\/td><td>Verify API and database interactions<\/td><\/tr><tr><td>API testing<\/td><td>Validate requests, responses, and error handling<\/td><\/tr><tr><td>Security testing<\/td><td>Verify authorization and input handling<\/td><\/tr><tr><td>Performance testing<\/td><td>Verify behavior under defined load<\/td><\/tr><tr><td>Regression testing<\/td><td>Confirm that existing behavior remains correct<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Prioritize the risks that could cause serious business or technical consequences.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 3: Create a test plan<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A lightweight test plan can be stored in the repository.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Example:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-1\" data-shcb-language-name=\"CSS\" data-shcb-language-slug=\"css\"><span><code class=\"hljs language-css\"><span class=\"hljs-selector-tag\">test_plan<\/span>:\n  <span class=\"hljs-selector-tag\">id<\/span>: <span class=\"hljs-selector-tag\">TP-BATTERY-001<\/span>\n  <span class=\"hljs-selector-tag\">component<\/span>: <span class=\"hljs-selector-tag\">battery-status-api<\/span>\n\n  <span class=\"hljs-selector-tag\">objectives<\/span>:\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">Validate<\/span> <span class=\"hljs-selector-tag\">battery<\/span> <span class=\"hljs-selector-tag\">percentage<\/span> <span class=\"hljs-selector-tag\">inputs<\/span>\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">Verify<\/span> <span class=\"hljs-selector-tag\">API<\/span> <span class=\"hljs-selector-tag\">response<\/span> <span class=\"hljs-selector-tag\">behavior<\/span>\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">Prevent<\/span> <span class=\"hljs-selector-tag\">invalid<\/span> <span class=\"hljs-selector-tag\">data<\/span> <span class=\"hljs-selector-tag\">persistence<\/span>\n\n  <span class=\"hljs-selector-tag\">test_levels<\/span>:\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">unit<\/span>\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">integration<\/span>\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">system<\/span>\n\n  <span class=\"hljs-selector-tag\">test_environment<\/span>:\n    <span class=\"hljs-selector-tag\">type<\/span>: <span class=\"hljs-selector-tag\">isolated-integration<\/span>\n    <span class=\"hljs-selector-tag\">test_data<\/span>: <span class=\"hljs-selector-tag\">synthetic<\/span>\n\n  <span class=\"hljs-selector-tag\">entry_criteria<\/span>:\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">Requirements<\/span> <span class=\"hljs-selector-tag\">reviewed<\/span>\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">Test<\/span> <span class=\"hljs-selector-tag\">environment<\/span> <span class=\"hljs-selector-tag\">available<\/span>\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">Candidate<\/span> <span class=\"hljs-selector-tag\">build<\/span> <span class=\"hljs-selector-tag\">identified<\/span>\n\n  <span class=\"hljs-selector-tag\">exit_criteria<\/span>:\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">Required<\/span> <span class=\"hljs-selector-tag\">critical<\/span> <span class=\"hljs-selector-tag\">tests<\/span> <span class=\"hljs-selector-tag\">completed<\/span>\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">No<\/span> <span class=\"hljs-selector-tag\">unresolved<\/span> <span class=\"hljs-selector-tag\">blocker<\/span> <span class=\"hljs-selector-tag\">defects<\/span>\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">Residual<\/span> <span class=\"hljs-selector-tag\">risks<\/span> <span class=\"hljs-selector-tag\">reviewed<\/span>\n\n  <span class=\"hljs-selector-tag\">evidence<\/span>:\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">test-results<\/span><span class=\"hljs-selector-class\">.xml<\/span>\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">defect-report<\/span>\n    <span class=\"hljs-selector-tag\">-<\/span> <span class=\"hljs-selector-tag\">test-completion-summary<\/span><\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-1\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">CSS<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">css<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">This is a simplified practical template, not a complete reproduction of ISO\/IEC\/IEEE 29119-3 requirements.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For formal conformance, the applicable standard must be reviewed and its requirements satisfied or formally tailored where permitted.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 4: Apply test design techniques<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Consider equivalence partitioning and boundary value analysis.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For the battery percentage requirement, useful partitions are:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Input class<\/th><th>Example<\/th><th>Expected result<\/th><\/tr><tr><td>Below lower boundary<\/td><td>-1<\/td><td>Reject<\/td><\/tr><tr><td>Lower boundary<\/td><td>0<\/td><td>Accept<\/td><\/tr><tr><td>Valid interior<\/td><td>50<\/td><td>Accept<\/td><\/tr><tr><td>Upper boundary<\/td><td>100<\/td><td>Accept<\/td><\/tr><tr><td>Above upper boundary<\/td><td>101<\/td><td>Reject<\/td><\/tr><tr><td>Missing value<\/td><td>None<\/td><td>Reject<\/td><\/tr><tr><td>Unsupported type<\/td><td>True<\/td><td>Reject<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Boundary analysis is useful because errors commonly occur near the limits of accepted ranges.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Additional techniques can be selected for more complex logic.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, a notification rule involving severity, account permissions, and user preferences may benefit from decision-table testing.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. Practical automation with Python and pytest<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Create a project directory and install pytest:<\/p>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">mkdir iso29119-demo\ncd iso29119-demo\n\npython3 -m venv .venv\nsource .venv\/bin\/activate\n\npython -m pip install pytest<\/code><\/span><\/pre>\n\n\n<p class=\"wp-block-paragraph\">On Windows PowerShell, activate the virtual environment using:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-2\" data-shcb-language-name=\"CSS\" data-shcb-language-slug=\"css\"><span><code class=\"hljs language-css\"><span class=\"hljs-selector-class\">.venv<\/span>\\<span class=\"hljs-selector-tag\">Scripts<\/span>\\<span class=\"hljs-selector-tag\">Activate<\/span><span class=\"hljs-selector-class\">.ps1<\/span><\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-2\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">CSS<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">css<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">Create <code>battery.py<\/code>:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-3\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\">def is_valid_percentage(value):\n    <span class=\"hljs-keyword\">return<\/span> (\n        type(value) <span class=\"hljs-keyword\">in<\/span> (int, float)\n        and <span class=\"hljs-number\">0<\/span> &lt;= value &lt;= <span class=\"hljs-number\">100<\/span>\n    )<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-3\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">Create <code>test_battery.py<\/code>:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-4\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">import pytest\nfrom battery import is_valid_percentage\n\n\n@pytest.mark.parametrize(\n    <span class=\"hljs-string\">\"value,expected\"<\/span>,\n    &#91;\n        (<span class=\"hljs-number\">-1<\/span>, <span class=\"hljs-keyword\">False<\/span>),\n        (<span class=\"hljs-number\">0<\/span>, <span class=\"hljs-keyword\">True<\/span>),\n        (<span class=\"hljs-number\">1<\/span>, <span class=\"hljs-keyword\">True<\/span>),\n        (<span class=\"hljs-number\">50<\/span>, <span class=\"hljs-keyword\">True<\/span>),\n        (<span class=\"hljs-number\">99<\/span>, <span class=\"hljs-keyword\">True<\/span>),\n        (<span class=\"hljs-number\">100<\/span>, <span class=\"hljs-keyword\">True<\/span>),\n        (<span class=\"hljs-number\">101<\/span>, <span class=\"hljs-keyword\">False<\/span>),\n        (None, <span class=\"hljs-keyword\">False<\/span>),\n        (<span class=\"hljs-keyword\">True<\/span>, <span class=\"hljs-keyword\">False<\/span>),\n    ],\n)\ndef test_percentage_validation(value, expected):\n    assert is_valid_percentage(value) is expected<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-4\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">Run the tests:<\/p>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">python -m pytest -v<\/code><\/span><\/pre>\n\n\n<p class=\"wp-block-paragraph\">Generate machine-readable test evidence:<\/p>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">python -m pytest \\\n  --junitxml=test-results.xml<\/code><\/span><\/pre>\n\n\n<p class=\"wp-block-paragraph\">This example demonstrates how formal test design concepts translate into executable tests.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It does not establish compliance with the entire ISO 29119 series.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">5. Establish requirements traceability<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Every important requirement should have a clear relationship to its verification evidence.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Requirement<\/th><th>Test case<\/th><th>Expected behavior<\/th><th>Evidence<\/th><\/tr><tr><td>BAT-001<\/td><td>TC-001<\/td><td>Accept valid charge percentage<\/td><td>Unit test report<\/td><\/tr><tr><td>BAT-002<\/td><td>TC-002<\/td><td>Reject invalid charge percentage<\/td><td>Unit and API reports<\/td><\/tr><tr><td>BAT-003<\/td><td>TC-003<\/td><td>Invalid input does not update persistence<\/td><td>Integration test report<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Traceability becomes especially important when teams must demonstrate testing completeness to customers or assessors.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The objective is not to create more paperwork. It is to make verification auditable and reproducible.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">6. Test execution and defect management<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">When a test fails, distinguish between:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>A genuine software defect.<\/li>\n\n\n\n<li>An incorrect test expectation.<\/li>\n\n\n\n<li>Invalid test data.<\/li>\n\n\n\n<li>An environment problem.<\/li>\n\n\n\n<li>An automation problem.<\/li>\n\n\n\n<li>An unresolved issue requiring investigation.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">A useful defect report contains:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Defect ID:<\/strong> BUG-104<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Affected build:<\/strong> Identified release candidate<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Requirement:<\/strong> BAT-002<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Expected:<\/strong> Invalid percentage is rejected.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Actual:<\/strong> API accepts an out-of-range value.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Reproduction:<\/strong> Submit a value of 101.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Impact:<\/strong> Invalid data may be displayed or persisted.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Evidence:<\/strong> Request, response, logs, and linked test result.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The report should also identify severity, ownership, and resolution status.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">7. Produce meaningful test reports<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A useful test status report should answer:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>What was tested?<\/li>\n\n\n\n<li>What passed and failed?<\/li>\n\n\n\n<li>Which planned tests remain incomplete?<\/li>\n\n\n\n<li>Which significant defects remain unresolved?<\/li>\n\n\n\n<li>What risks remain?<\/li>\n\n\n\n<li>Is the available evidence sufficient for a release decision?<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Avoid reporting only an overall pass percentage.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A 99% pass rate can still conceal failure of a business-critical test.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">8. Apply the standard in Agile and DevOps<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A lightweight implementation can map the testing process into development activities.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Development activity<\/th><th>Testing activity<\/th><\/tr><tr><td>Backlog refinement<\/td><td>Identify risks and testable acceptance criteria<\/td><\/tr><tr><td>Pull request<\/td><td>Execute unit tests and static checks<\/td><\/tr><tr><td>Integration build<\/td><td>Execute API and integration tests<\/td><\/tr><tr><td>Release candidate<\/td><td>Perform system and regression testing<\/td><\/tr><tr><td>Release review<\/td><td>Examine residual risks and test completion evidence<\/td><\/tr><tr><td>Production operation<\/td><td>Capture operational feedback for future testing<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The test plan can be a repository artifact, test cases can be executable, and test reports can be generated automatically.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The important issue is whether the necessary process objectives and applicable requirements are satisfied\u2014not whether the organization uses a particular document format.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">9. Common mistakes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Treating ISO 29119 as a test case template standard only.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The series includes test management, processes, and techniques.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Writing tests without identifying expected behavior.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Every meaningful test needs a credible way to determine success or failure.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Using coverage percentages without analyzing risk.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Coverage helps describe what was exercised but does not by itself demonstrate adequate product quality.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Declaring conformance from tool usage alone.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Using pytest, Playwright, or a test management application does not establish ISO 29119 conformance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Ignoring formal tailoring.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Part 1 explains that the normative parts have conformance requirements and that tailored conformance may be appropriate when the tailoring and rationale are defined and agreed as required.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ISO\/IEC\/IEEE 29119 provides a consistent foundation for software testing across different engineering environments.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its strongest practical value comes from combining clear testing objectives, systematic test design, controlled execution, evidence management, and risk-based reporting.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Official resources<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/committee.iso.org\/sites\/jtc1sc7\/home\/projects\/flagship-standards\/isoiecieee-29119-series.html\">ISO \u2014 Software testing standards series<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.iso.org\/obp\/ui?_escaped_fragment_=iso%3Astd%3Aiso-iec-ieee%3A29119%3A-1%3Aed-2%3Av1%3Aen\">ISO\/IEC\/IEEE 29119-1 \u2014 General concepts<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.iso.org\/standard\/79429.html\">ISO\/IEC\/IEEE 29119-3 \u2014 Test documentation<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.iso.org\/standard\/79430.html\">ISO\/IEC\/IEEE 29119-4 \u2014 Test techniques<\/a><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Verified against ISO&#8217;s standards catalogue, the official software testing series guidance, and the pytest documentation. Software and systems engineering+4<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tutorial 14 \u2014 ISO\/IEC 25010: Software Product Quality Models<\/h2>\n\n\n\n<h1 class=\"wp-block-heading\">ISO\/IEC 25010 Tutorial: How to Define, Measure, and Improve Software Product Quality<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">A software application can pass all its functional tests and still provide a poor experience.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It might be slow, difficult to maintain, inaccessible to users, vulnerable to attacks, or unreliable under demanding conditions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These are different dimensions of software product quality, and each requires a suitable evaluation approach.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>ISO\/IEC 25010<\/strong> provides a structured quality model that helps organizations identify and evaluate those dimensions.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">1. What is ISO\/IEC 25010?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ISO\/IEC 25010 belongs to the SQuaRE family\u2014Systems and software Quality Requirements and Evaluation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The standard defines a model for the quality characteristics of ICT and software products.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Organizations can use the model when:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Establishing product requirements.<\/li>\n\n\n\n<li>Designing systems and software.<\/li>\n\n\n\n<li>Defining non-functional testing objectives.<\/li>\n\n\n\n<li>Developing release acceptance criteria.<\/li>\n\n\n\n<li>Evaluating software suppliers.<\/li>\n\n\n\n<li>Reviewing technical quality.<\/li>\n\n\n\n<li>Prioritizing quality engineering investment.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The standard provides a classification model, not a universal list of performance thresholds or an automatic software certification mechanism.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2. The nine software product quality characteristics<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ISO\/IEC 25010:2023 defines nine primary product quality characteristics.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Characteristic<\/th><th>What it evaluates<\/th><th>Example verification<\/th><\/tr><tr><td>Functional suitability<\/td><td>Whether the product provides appropriate and correct functions<\/td><td>Requirements-based testing<\/td><\/tr><tr><td>Performance efficiency<\/td><td>Response behavior, resource consumption, and capacity<\/td><td>Load and performance testing<\/td><\/tr><tr><td>Compatibility<\/td><td>Ability to coexist and interoperate with other systems<\/td><td>Integration and compatibility testing<\/td><\/tr><tr><td>Interaction capability<\/td><td>How effectively people or other entities can interact with the product<\/td><td>Usability and accessibility-related evaluations<\/td><\/tr><tr><td>Reliability<\/td><td>Ability to perform consistently and recover from failures<\/td><td>Fault injection and recovery testing<\/td><\/tr><tr><td>Security<\/td><td>Protection against relevant threats and unauthorized actions<\/td><td>Security testing and threat analysis<\/td><\/tr><tr><td>Maintainability<\/td><td>Ease of analysis, modification, testing, and reuse<\/td><td>Code review and maintainability analysis<\/td><\/tr><tr><td>Flexibility<\/td><td>Ability to adapt, scale, install, and operate in changing environments<\/td><td>Scaling and deployment testing<\/td><\/tr><tr><td>Safety<\/td><td>Ability to avoid unacceptable harm in relevant operating conditions<\/td><td>Hazard-based and fail-safe validation<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Each characteristic contains more specific subcharacteristics.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This helps engineers turn the broad idea of quality into concrete engineering objectives.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">3. Important differences from the older quality model<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Many online tutorials still describe eight ISO 25010 characteristics.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That reflects ISO\/IEC 25010:2011, which has been replaced.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The revised product quality model introduced important changes:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Safety became a separate quality characteristic.<\/li>\n\n\n\n<li>Usability was replaced by Interaction capability in the product quality model.<\/li>\n\n\n\n<li>Portability was replaced by Flexibility.<\/li>\n\n\n\n<li>Several subcharacteristics were revised or added.<\/li>\n\n\n\n<li>Quality-in-use is addressed separately in ISO\/IEC 25019:2023.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The distinction between product quality and quality-in-use is useful.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Product quality concerns the characteristics of the product itself. Quality-in-use concerns the consequences and outcomes of using the product in a specified context.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A technically responsive interface does not automatically guarantee that users complete their tasks successfully.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. How to implement ISO 25010<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Assume FleetLink provides a mobile application and cloud services for viewing connected-vehicle information.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The product team wants to convert broad quality expectations into measurable requirements.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 1: Identify stakeholders and their expectations<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Consider the needs of:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>End users.<\/li>\n\n\n\n<li>Product management.<\/li>\n\n\n\n<li>Engineering teams.<\/li>\n\n\n\n<li>Security and privacy stakeholders.<\/li>\n\n\n\n<li>Operations teams.<\/li>\n\n\n\n<li>Support teams.<\/li>\n\n\n\n<li>Customers or contractual partners.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Different stakeholders may prioritize quality differently.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, operations teams may emphasize reliability and observability, while end users may emphasize interaction capability and responsiveness.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 2: Identify critical quality characteristics<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Do not assume every product requires an identical evaluation depth for every characteristic.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Evaluate product purpose, architecture, operating conditions, business consequences, and potential hazards.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For FleetLink, reliability, security, performance efficiency, and interaction capability may require significant attention.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If the product includes functions with potential safety consequences, safety engineering must be addressed through suitable hazard analysis and applicable domain-specific requirements.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 3: Write measurable quality requirements<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A weak requirement says:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&#8220;The application must be fast.&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A stronger requirement says:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&#8220;Under the agreed reference workload and environment, the API must respond within the specified latency threshold for at least 95% of successful requests.&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The requirement becomes testable when the workload, measurement point, environment, threshold, exclusions, and acceptance method are defined.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 4: Build a product quality matrix<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The following targets are illustrative engineering examples, not ISO-mandated values.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Characteristic<\/th><th>Example objective<\/th><th>Example evidence<\/th><\/tr><tr><td>Functional suitability<\/td><td>All identified critical functions satisfy approved acceptance criteria<\/td><td>Functional test results<\/td><\/tr><tr><td>Performance efficiency<\/td><td>API p95 latency \u2264 300 ms under a defined workload<\/td><td>Load test report<\/td><\/tr><tr><td>Compatibility<\/td><td>Supported integration versions pass defined contract tests<\/td><td>Contract test report<\/td><\/tr><tr><td>Interaction capability<\/td><td>Critical user journeys meet agreed usability and accessibility criteria<\/td><td>UX evaluation and accessibility checks<\/td><\/tr><tr><td>Reliability<\/td><td>Recovery completes within the agreed service recovery objective<\/td><td>Recovery test results<\/td><\/tr><tr><td>Security<\/td><td>No unresolved critical findings in the agreed assessment scope<\/td><td>Security review and test reports<\/td><\/tr><tr><td>Maintainability<\/td><td>Critical changes meet review and maintainability criteria<\/td><td>Code review and static analysis<\/td><\/tr><tr><td>Flexibility<\/td><td>System satisfies approved horizontal-scaling scenarios<\/td><td>Scaling test report<\/td><\/tr><tr><td>Safety<\/td><td>Safety-related behavior meets approved hazard-derived requirements<\/td><td>Safety verification evidence<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Quality objectives should be reviewed whenever the product&#8217;s architecture, operating conditions, or risks change.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">5. Measuring performance efficiency<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Performance efficiency involves more than response time.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Relevant measures may include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Response latency.<\/li>\n\n\n\n<li>Throughput.<\/li>\n\n\n\n<li>CPU consumption.<\/li>\n\n\n\n<li>Memory utilization.<\/li>\n\n\n\n<li>Storage I\/O.<\/li>\n\n\n\n<li>Concurrent workload capacity.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Suppose the team defines:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>A target workload of 500 requests per second.<\/li>\n\n\n\n<li>An API p95 response time of at most 300 milliseconds.<\/li>\n\n\n\n<li>A defined error-rate threshold.<\/li>\n\n\n\n<li>A reproducible test environment.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Engineers can use tools such as k6 or Apache JMeter to execute the workload and collect evidence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The workload model must reflect expected usage rather than an arbitrary collection of HTTP requests.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">6. Measuring reliability<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Reliability evaluation may include availability, faultlessness, fault tolerance, and recoverability.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Consider a service dependency becoming unavailable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Useful test scenarios include:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Introduce a defined dependency failure.<\/li>\n\n\n\n<li>Observe application behavior.<\/li>\n\n\n\n<li>Verify that the system handles the failure as specified.<\/li>\n\n\n\n<li>Restore the dependency.<\/li>\n\n\n\n<li>Measure recovery time.<\/li>\n\n\n\n<li>Confirm that relevant data remains consistent.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Realistic failure scenarios reveal weaknesses that functional happy-path tests cannot identify.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">7. Measuring security<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Security testing should address relevant threats, architecture, and business risks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Practical activities include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Authentication and authorization testing.<\/li>\n\n\n\n<li>Secure configuration review.<\/li>\n\n\n\n<li>Source code security analysis.<\/li>\n\n\n\n<li>Dependency vulnerability review.<\/li>\n\n\n\n<li>API security testing.<\/li>\n\n\n\n<li>Security-related logging verification.<\/li>\n\n\n\n<li>Threat modeling.<\/li>\n\n\n\n<li>Independent testing where appropriate.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">A security test result must be interpreted against the tested scope, configuration, and supported threat model.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Passing an automated scanner does not prove that software is secure.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">8. Measuring maintainability<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Maintainability concerns how effectively software can be understood, changed, tested, and maintained.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Useful engineering indicators include:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Change lead time:<\/strong> Time required to implement, review, verify, and deliver a defined change.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Complexity:<\/strong> Complexity measures applied to relevant components with contextual interpretation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Code duplication:<\/strong> Repeated implementation patterns that create maintenance risk.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Testability:<\/strong> Ability to validate components and behavior without unreasonable effort.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Architecture constraints:<\/strong> Evidence that component dependencies respect intended boundaries.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These metrics should guide engineering decisions rather than become isolated targets that teams optimize without improving the product.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">9. Measuring interaction capability<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Interaction capability is particularly relevant for mobile applications, dashboards, websites, and interactive systems.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Evaluate:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Clarity of system behavior.<\/li>\n\n\n\n<li>Ease of learning.<\/li>\n\n\n\n<li>Prevention of user errors.<\/li>\n\n\n\n<li>User guidance and assistance.<\/li>\n\n\n\n<li>Support for users with diverse capabilities.<\/li>\n\n\n\n<li>Effectiveness of interaction design.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">For example, a vehicle-status application should clearly distinguish between fresh vehicle data and data that has not synchronized recently.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This prevents users from misinterpreting outdated information as real-time status.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Tools such as axe-core and Lighthouse can support selected checks, but human evaluation remains important.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">10. Evaluating safety<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Safety must be considered when software behavior can contribute to harm.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, a connected system controlling or influencing physical operations may require evidence that:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Relevant hazards have been identified.<\/li>\n\n\n\n<li>Fault conditions are handled appropriately.<\/li>\n\n\n\n<li>Safety constraints are enforced.<\/li>\n\n\n\n<li>Failure behavior is appropriately controlled.<\/li>\n\n\n\n<li>Interfaces do not introduce unacceptable hazards.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">ISO\/IEC 25010 supplies a quality model. It does not replace domain-specific functional safety standards such as ISO 26262 for applicable automotive safety-related systems.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Safety requirements and verification methods must be derived from the actual application and relevant safety engineering activities.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">11. Connect quality objectives to DevSecOps<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Quality requirements should influence development from the beginning.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Engineering stage<\/th><th>Quality activities<\/th><\/tr><tr><td>Requirements<\/td><td>Establish measurable quality objectives<\/td><\/tr><tr><td>Architecture<\/td><td>Evaluate tradeoffs and quality risks<\/td><\/tr><tr><td>Development<\/td><td>Apply secure coding, reviews, and component testing<\/td><\/tr><tr><td>Continuous integration<\/td><td>Execute automated tests and relevant static checks<\/td><\/tr><tr><td>Pre-release<\/td><td>Perform targeted reliability, security, and performance validation<\/td><\/tr><tr><td>Production<\/td><td>Monitor operational quality and customer impact<\/td><\/tr><tr><td>Improvement<\/td><td>Adjust requirements and verification practices using evidence<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This approach moves quality from a final testing event into the software development lifecycle.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">12. Common mistakes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Using the outdated eight-characteristic model without checking the edition.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Use the applicable edition and document which version governs the project.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Treating every quality metric as a universal threshold.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">ISO 25010 does not establish one pass\/fail value appropriate for all products.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Testing only functional suitability.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Functional correctness does not guarantee reliability, security, maintainability, or other quality characteristics.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Using static analysis as the complete quality measurement strategy.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Structural analysis should complement behavioral testing, operational evidence, and stakeholder evaluation.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ISO\/IEC 25010 provides a powerful foundation for defining what software product quality actually means.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its greatest value comes from translating quality characteristics into measurable requirements, selecting suitable verification methods, and continuously evaluating the resulting evidence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Official resources<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.iso.org\/standard\/78176.html\">ISO\/IEC 25010 \u2014 Product quality model<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.iso.org\/standard\/78177.html\">ISO\/IEC 25019 \u2014 Quality-in-use model<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.iso.org\/standard\/35747.html\">ISO\/IEC 25023 \u2014 Measurement of system and software product quality<\/a><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Verified against ISO&#8217;s published product quality model and quality-in-use standard. Product quality model+3<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tutorial 15 \u2014 IEEE 730: Software Quality Assurance Processes<\/h2>\n\n\n\n<h1 class=\"wp-block-heading\">IEEE 730 Tutorial: How to Build an Effective Software Quality Assurance Process and SQA Plan<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">Software testing is essential, but testing alone cannot guarantee software quality.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A test team may identify defects while the wider engineering organization struggles with inconsistent requirements, inadequate reviews, uncontrolled changes, supplier risks, or incomplete release evidence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Software Quality Assurance (SQA)<\/strong> addresses these broader concerns.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">IEEE 730 provides a framework for establishing and managing software quality assurance processes throughout the software lifecycle.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">1. What is IEEE 730?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">IEEE 730 is a standard for software quality assurance processes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The current edition is <strong>IEEE 730-2026<\/strong>, titled <em>IEEE Standard for Software Quality Assurance Processes<\/em>. It supersedes IEEE 730-2014.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The standard addresses initiating, planning, controlling, and executing software quality assurance activities. Its scope extends beyond software testing to quality governance and assurance across software development and maintenance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The updated standard is harmonized with relevant software lifecycle and engineering standards, including ISO\/IEC\/IEEE 12207 and IEEE 2675.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When referring to the current publication, use the designation <strong>IEEE 730-2026<\/strong>, rather than assuming that the designation ISO\/IEC\/IEEE 730 identifies the current IEEE publication.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2. Understand QA, QC, and testing<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">These terms are related but not interchangeable.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Discipline<\/th><th>Main concern<\/th><th>Example<\/th><\/tr><tr><td>Quality Assurance<\/td><td>Establishing confidence that appropriate quality processes are implemented and followed<\/td><td>Process assessment and assurance reviews<\/td><\/tr><tr><td>Quality Control<\/td><td>Evaluating whether products meet specified criteria<\/td><td>Product inspection and acceptance evaluation<\/td><\/tr><tr><td>Software Testing<\/td><td>Evaluating software behavior and discovering defects or risks<\/td><td>API, integration, and regression testing<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Software testing can contribute to quality control, while SQA evaluates the wider process and evidence supporting the delivery of quality software.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, SQA may verify that changes receive appropriate review, critical requirements have verification evidence, and release exceptions are formally considered.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">3. What is a Software Quality Assurance Plan?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A Software Quality Assurance Plan (SQAP) describes how quality assurance responsibilities and activities will be organized and performed for a project or organizational context.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A practical SQAP should make the assurance approach understandable, repeatable, and auditable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The plan should answer:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>What quality objectives apply?<\/li>\n\n\n\n<li>Which lifecycle processes are in scope?<\/li>\n\n\n\n<li>Who is accountable for assurance activities?<\/li>\n\n\n\n<li>Which standards and procedures apply?<\/li>\n\n\n\n<li>What evidence must be maintained?<\/li>\n\n\n\n<li>How are nonconformities handled?<\/li>\n\n\n\n<li>What happens when quality expectations are not met?<\/li>\n\n\n\n<li>How will the assurance process be evaluated and improved?<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The exact plan structure and required contents for a claim of conformance must be checked against the applicable IEEE 730 edition.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. Step-by-step implementation<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Step 1: Establish software quality objectives<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Start with business, contractual, regulatory, and product requirements.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For the fictional FleetLink platform, example objectives might include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Reliable delivery of vehicle-status information.<\/li>\n\n\n\n<li>Protection of customer and vehicle-related data.<\/li>\n\n\n\n<li>Predictable release quality.<\/li>\n\n\n\n<li>Effective management of software changes.<\/li>\n\n\n\n<li>Traceable verification evidence.<\/li>\n\n\n\n<li>Timely management of significant quality issues.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Quality objectives should be actionable and connected to measurable evidence.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 2: Define the scope of assurance<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Identify relevant lifecycle activities.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These may include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Requirements development and review.<\/li>\n\n\n\n<li>Software architecture and design.<\/li>\n\n\n\n<li>Implementation.<\/li>\n\n\n\n<li>Testing and verification.<\/li>\n\n\n\n<li>Configuration and change management.<\/li>\n\n\n\n<li>Software acquisition and supplier management.<\/li>\n\n\n\n<li>Deployment and release.<\/li>\n\n\n\n<li>Maintenance and incident management.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Quality assurance should not begin only after development is complete.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 3: Define roles and responsibilities<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A simplified responsibility model could be:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Activity<\/th><th>Accountable role<\/th><th>Supporting roles<\/th><\/tr><tr><td>Quality objectives<\/td><td>Engineering or product leadership<\/td><td>SQA, security, stakeholders<\/td><\/tr><tr><td>Code review process<\/td><td>Engineering lead<\/td><td>Developers<\/td><\/tr><tr><td>Test strategy<\/td><td>Quality engineering lead<\/td><td>Developers, product owners<\/td><\/tr><tr><td>Security assurance<\/td><td>Security owner<\/td><td>Development and operations<\/td><\/tr><tr><td>Release decision<\/td><td>Authorized release owner<\/td><td>SQA, engineering, operations<\/td><\/tr><tr><td>Quality process assessment<\/td><td>SQA owner<\/td><td>Independent reviewers as appropriate<\/td><\/tr><tr><td>Corrective action tracking<\/td><td>Assigned process owner<\/td><td>Relevant engineering teams<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The organization should ensure that assurance findings can be reported and escalated without inappropriate conflicts of interest.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A small organization may use designated reviewers and structured escalation rather than a large dedicated QA department.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 4: Prepare the SQA plan<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The following is an illustrative plan outline.<\/p>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">Software Quality Assurance Plan\n\n1. Purpose and scope\n2. Product and system context\n3. Quality objectives\n4. Applicable standards and procedures\n5. Roles and responsibilities\n6. Lifecycle assurance activities\n7. Requirements and design reviews\n8. Verification and validation evidence\n9. Configuration and change management\n10. Security, privacy, and reliability assurance\n11. Supplier and third-party quality controls\n12. Nonconformity and corrective action process\n13. Quality measurement and reporting\n14. Release quality review\n15. Records and evidence retention\n16. Assurance process evaluation and improvement<\/code><\/span><\/pre>\n\n\n<p class=\"wp-block-paragraph\">This example is an implementation starting point, not an official IEEE 730 template or exhaustive compliance checklist.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 5: Establish quality checkpoints<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Integrate assurance activities into the development lifecycle.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>During requirements development<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Verify that critical quality expectations are identified, reviewed, and testable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>During architecture and design<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Review important security, reliability, performance, and maintainability risks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>During implementation<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Verify the application of relevant development practices, source code review, and configuration management.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>During testing<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Examine the adequacy of test coverage, environments, outcomes, and unresolved risks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Before release<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Review acceptance evidence, significant defects, operational readiness, and approved exceptions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>After deployment<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Evaluate production incidents and feedback for corrective and preventive actions.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 6: Establish a nonconformity management process<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A nonconformity is a failure to satisfy an applicable requirement, process expectation, or agreed criterion.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A production deployment is completed without a required security review.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The corrective process should identify:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>The applicable requirement.<\/li>\n\n\n\n<li>What was not satisfied.<\/li>\n\n\n\n<li>The impact and current risk.<\/li>\n\n\n\n<li>Necessary containment.<\/li>\n\n\n\n<li>The root cause.<\/li>\n\n\n\n<li>Corrective action.<\/li>\n\n\n\n<li>Verification of effectiveness.<\/li>\n\n\n\n<li>Final closure evidence.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">A process nonconformity is not always the same thing as a software defect.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Both may require tracking, but they concern different aspects of quality.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">5. Implement quality assurance in CI\/CD<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Software assurance can be supported through automated evidence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A simplified workflow is:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-5\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">Source Change\n    |\n    v\nCode Review\n    |\n    v\n<span class=\"hljs-keyword\">Static<\/span> Analysis <span class=\"hljs-keyword\">and<\/span> Security Checks\n    |\n    v\nUnit <span class=\"hljs-keyword\">and<\/span> Integration Testing\n    |\n    v\nBuild <span class=\"hljs-keyword\">and<\/span> Artifact Verification\n    |\n    v\nRelease Readiness Assessment\n    |\n    v\nAuthorized Deployment\n    |\n    v\nProduction Monitoring\n    |\n    v\nCorrective <span class=\"hljs-keyword\">and<\/span> Preventive Improvement<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-5\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">CI\/CD automation improves consistency and evidence collection.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">However, automated tools cannot independently determine whether every applicable product, contractual, or assurance requirement has been satisfied.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">6. Example quality assurance checklist<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Assurance question<\/th><th>Evidence<\/th><th>Responsible party<\/th><\/tr><tr><td>Are critical requirements reviewed?<\/td><td>Review records<\/td><td>Product and engineering<\/td><\/tr><tr><td>Are relevant architecture risks assessed?<\/td><td>Design review and threat model<\/td><td>Architecture and security<\/td><\/tr><tr><td>Are code changes controlled?<\/td><td>Pull requests and approvals<\/td><td>Engineering<\/td><\/tr><tr><td>Are required tests executed?<\/td><td>Test results<\/td><td>Quality engineering<\/td><\/tr><tr><td>Are significant defects managed?<\/td><td>Defect records<\/td><td>Engineering and QA<\/td><\/tr><tr><td>Are release exceptions approved?<\/td><td>Exception and risk acceptance records<\/td><td>Authorized decision maker<\/td><\/tr><tr><td>Is configuration traceable?<\/td><td>Build and release metadata<\/td><td>Release engineering<\/td><\/tr><tr><td>Are production incidents reviewed?<\/td><td>Incident and corrective action records<\/td><td>Operations and engineering<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This is a practical assurance checklist rather than a standards conformance assessment.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">7. Recommended metrics<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Quality assurance metrics may include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Completion of applicable reviews.<\/li>\n\n\n\n<li>Open and overdue nonconformities.<\/li>\n\n\n\n<li>Corrective action closure time.<\/li>\n\n\n\n<li>Repeated process nonconformities.<\/li>\n\n\n\n<li>Verification evidence completeness.<\/li>\n\n\n\n<li>Release exceptions and their risk levels.<\/li>\n\n\n\n<li>Production incidents related to process weaknesses.<\/li>\n\n\n\n<li>Supplier quality issues.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Avoid using process compliance scores as a substitute for product outcomes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A well-documented process can still produce software that needs improvement.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">8. A practical 30-day adoption plan<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Week 1: Assess the current state<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Identify assurance responsibilities, applicable requirements, existing procedures, and major gaps.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Week 2: Establish the assurance plan<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Define scope, roles, review activities, evidence requirements, and nonconformity handling.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Week 3: Integrate assurance into engineering<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Introduce review checkpoints, evidence collection, quality reporting, and release risk decisions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Week 4: Evaluate effectiveness<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Review a sample of actual changes and releases. Identify whether the new process detects important weaknesses and produces useful evidence.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The goal is to establish a functioning baseline, not to claim full standards conformance after one month.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">9. Common mistakes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Treating SQA as another name for software testing.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Testing is one of several activities contributing to quality assurance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Assigning quality responsibility exclusively to a QA team.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Quality outcomes require active participation across engineering and the business.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Maintaining an SQA plan without operational evidence.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A procedure is valuable only when it influences actual practices.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Creating excessive manual approval gates.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Controls should be proportionate to risk and integrated into existing delivery workflows where possible.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Assuming IEEE 730 certification from an internal checklist.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Compliance claims require examination of the actual standard and relevant assessment or contractual mechanisms.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">IEEE 730 provides a foundation for establishing systematic software quality assurance across the engineering lifecycle.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An effective SQA program combines clear responsibilities, documented objectives, risk-based reviews, reliable evidence, nonconformity management, and continuous improvement.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The result should be greater confidence in software delivery\u2014not additional bureaucracy without measurable value.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Official resource<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/standards.ieee.org\/ieee\/730\/10854\/\">IEEE Standards Association \u2014 IEEE 730-2026<\/a><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Verified against the current IEEE 730-2026 publication and its supersession record. IEEE 730-2026+2<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tutorial 16 \u2014 Six Sigma: Defect Reduction and Process Improvement<\/h2>\n\n\n\n<h1 class=\"wp-block-heading\">Six Sigma for Software Engineering: A Practical Guide to DMAIC, Defect Reduction, and Continuous Improvement<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">Software engineering teams frequently encounter recurring quality problems.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An application may repeatedly fail deployment checks. Integration tests may be unreliable. Production incidents may follow similar patterns, and teams may spend significant time fixing issues that could have been prevented.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Solving individual incidents is necessary, but repeatedly fixing the same categories of problems is expensive.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Six Sigma<\/strong> provides a disciplined approach to understanding process variation, identifying root causes, and implementing measurable improvements.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Although originally developed in industrial quality management, its principles can be applied to software engineering, testing, DevOps, and operational processes.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">1. What is Six Sigma?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Six Sigma is a data-driven quality improvement methodology focused on reducing defects, errors, and process variation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It emphasizes:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Clearly defined problems.<\/li>\n\n\n\n<li>Reliable measurement.<\/li>\n\n\n\n<li>Root cause analysis.<\/li>\n\n\n\n<li>Evidence-based changes.<\/li>\n\n\n\n<li>Verification of improvement.<\/li>\n\n\n\n<li>Sustaining results over time.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">A commonly cited Six Sigma performance convention is <strong>3.4 defects per million opportunities (DPMO)<\/strong> at the six-sigma level under the conventional long-term sigma-shift model.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is not a universal software quality target or a guarantee that a product will be safe or defect-free.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Six Sigma is most useful when an organization has a recurring, measurable process problem that warrants systematic investigation.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2. Lean versus Six Sigma<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Lean and Six Sigma are closely related but emphasize different improvement concerns.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Approach<\/th><th>Main emphasis<\/th><th>Software engineering example<\/th><\/tr><tr><td>Lean<\/td><td>Reduce waste and improve flow<\/td><td>Eliminate unnecessary waiting during releases<\/td><\/tr><tr><td>Six Sigma<\/td><td>Reduce variation and recurring defects<\/td><td>Reduce inconsistent deployment outcomes<\/td><\/tr><tr><td>Lean Six Sigma<\/td><td>Improve flow while reducing defects and variation<\/td><td>Improve deployment speed and reliability together<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">These approaches can be combined with Agile, DevOps, TMMi, and other engineering process improvement practices.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">3. Understand DMAIC<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">DMAIC is the primary structured improvement cycle for existing processes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The five phases are:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Define<\/li>\n\n\n\n<li>Measure<\/li>\n\n\n\n<li>Analyze<\/li>\n\n\n\n<li>Improve<\/li>\n\n\n\n<li>Control<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Each phase has a distinct purpose and expected outcome.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. Step 1 \u2014 Define the problem<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Begin with a measurable problem statement.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Weak statement:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&#8220;Our deployments are unreliable.&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Stronger statement:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&#8220;A significant proportion of production deployments require remediation because the deployment process experiences recurring configuration and integration failures.&#8221;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Define:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>The process being examined.<\/li>\n\n\n\n<li>The measurable problem.<\/li>\n\n\n\n<li>The affected stakeholders.<\/li>\n\n\n\n<li>The business consequences.<\/li>\n\n\n\n<li>The improvement objective.<\/li>\n\n\n\n<li>The investigation boundaries.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Example project charter<\/h3>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Field<\/th><th>Example<\/th><\/tr><tr><td>Project<\/td><td>Deployment Reliability Improvement<\/td><\/tr><tr><td>Problem<\/td><td>Recurring deployment validation failures<\/td><\/tr><tr><td>Process<\/td><td>Build, test, and deployment workflow<\/td><\/tr><tr><td>Business impact<\/td><td>Delayed releases and engineering rework<\/td><\/tr><tr><td>Objective<\/td><td>Reduce preventable deployment defects<\/td><\/tr><tr><td>Sponsor<\/td><td>Engineering leadership<\/td><\/tr><tr><td>Process owner<\/td><td>Release engineering<\/td><\/tr><tr><td>Success measure<\/td><td>Fewer comparable deployment failures<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The objective should be specific enough to test after implementing changes.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">5. Step 2 \u2014 Measure the baseline<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Six Sigma requires trustworthy data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For a software deployment improvement project, useful measures include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Deployment attempts.<\/li>\n\n\n\n<li>Failed deployments.<\/li>\n\n\n\n<li>Deployment-related defects.<\/li>\n\n\n\n<li>Recovery time.<\/li>\n\n\n\n<li>Rework effort.<\/li>\n\n\n\n<li>Causes of failed validation checks.<\/li>\n\n\n\n<li>Change failure rate.<\/li>\n\n\n\n<li>Deployment duration.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Measurement definitions must remain consistent across the baseline and improvement periods.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Understanding DPMO<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Defects per million opportunities is calculated as:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[<br>DPMO=\\frac{\\text{Number of defects}}{\\text{Units}\\times\\text{Opportunities per unit}}\\times1{,}000{,}000<br>]<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An opportunity is a clearly defined chance for a particular defect to occur.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In software engineering, this definition requires care because opportunities are not always obvious or independent.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Worked example<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Suppose an engineering team examines 1,000 build executions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Each build contains four clearly defined validation opportunities.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">During the baseline period, the team records 72 qualifying validation defects.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Units: 1,000 builds<\/li>\n\n\n\n<li>Opportunities per unit: 4<\/li>\n\n\n\n<li>Total opportunities: 4,000<\/li>\n\n\n\n<li>Defects: 72<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">[<br>DPMO=\\frac{72}{4000}\\times1{,}000{,}000=18{,}000<br>]<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Now suppose a later, comparable period contains 20 qualifying defects across the same number of opportunities.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">[<br>DPMO=\\frac{20}{4000}\\times1{,}000{,}000=5{,}000<br>]<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&lt;text preserveWhitespace&gt;{&#8220;&#8221;}&lt;\/text&gt;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The illustrative DPMO reduction is approximately <strong>72.2%<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This suggests improvement, but a sound analysis should also consider workload comparability, changing system complexity, measurement accuracy, and whether the change is statistically distinguishable from ordinary variation.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Calculate DPMO with Python<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Create <code>quality_metrics.py<\/code>:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-6\" data-shcb-language-name=\"PHP\" data-shcb-language-slug=\"php\"><span><code class=\"hljs language-php\">def calculate_dpmo(defects, units, opportunities_per_unit):\n    <span class=\"hljs-keyword\">if<\/span> units &lt;= <span class=\"hljs-number\">0<\/span> <span class=\"hljs-keyword\">or<\/span> opportunities_per_unit &lt;= <span class=\"hljs-number\">0<\/span>:\n        raise ValueError(<span class=\"hljs-string\">\"Units and opportunities must be positive\"<\/span>)\n\n    <span class=\"hljs-keyword\">if<\/span> defects &lt; <span class=\"hljs-number\">0<\/span>:\n        raise ValueError(<span class=\"hljs-string\">\"Defect count cannot be negative\"<\/span>)\n\n    total_opportunities = units * opportunities_per_unit\n\n    <span class=\"hljs-keyword\">return<\/span> (defects \/ total_opportunities) * <span class=\"hljs-number\">1<\/span>_000_000\n\n\nbaseline = calculate_dpmo(<span class=\"hljs-number\">72<\/span>, <span class=\"hljs-number\">1000<\/span>, <span class=\"hljs-number\">4<\/span>)\nimproved = calculate_dpmo(<span class=\"hljs-number\">20<\/span>, <span class=\"hljs-number\">1000<\/span>, <span class=\"hljs-number\">4<\/span>)\n\nreduction = ((baseline - improved) \/ baseline) * <span class=\"hljs-number\">100<\/span>\n\n<span class=\"hljs-keyword\">print<\/span>(f<span class=\"hljs-string\">\"Baseline DPMO: {baseline:,.0f}\"<\/span>)\n<span class=\"hljs-keyword\">print<\/span>(f<span class=\"hljs-string\">\"Improved DPMO: {improved:,.0f}\"<\/span>)\n<span class=\"hljs-keyword\">print<\/span>(f<span class=\"hljs-string\">\"Reduction: {reduction:.1f}%\"<\/span>)<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-6\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">PHP<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">php<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">Execute:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-7\" data-shcb-language-name=\"CSS\" data-shcb-language-slug=\"css\"><span><code class=\"hljs language-css\"><span class=\"hljs-selector-tag\">python3<\/span> <span class=\"hljs-selector-tag\">quality_metrics<\/span><span class=\"hljs-selector-class\">.py<\/span><\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-7\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">CSS<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">css<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">Expected output:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-8\" data-shcb-language-name=\"CSS\" data-shcb-language-slug=\"css\"><span><code class=\"hljs language-css\"><span class=\"hljs-selector-tag\">Baseline<\/span> <span class=\"hljs-selector-tag\">DPMO<\/span>: 18,000\n<span class=\"hljs-selector-tag\">Improved<\/span> <span class=\"hljs-selector-tag\">DPMO<\/span>: 5,000\n<span class=\"hljs-selector-tag\">Reduction<\/span>: 72<span class=\"hljs-selector-class\">.2<\/span>%<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-8\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">CSS<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">css<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">The calculation is only meaningful if the opportunities and defect counting rules are consistently defined.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">6. Step 3 \u2014 Analyze the root causes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A common mistake is implementing a solution before understanding the problem.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, when tests fail frequently, teams may immediately increase retries.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Retries can conceal defects, waste execution resources, and make diagnosis harder.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Instead, classify failures and investigate their causes.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Example failure classification<\/h3>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Cause category<\/th><th>Investigation question<\/th><\/tr><tr><td>Test data<\/td><td>Is test data predictable and isolated?<\/td><\/tr><tr><td>Environment<\/td><td>Are services available and configured consistently?<\/td><\/tr><tr><td>Application defects<\/td><td>Is application behavior incorrect?<\/td><\/tr><tr><td>Automation defects<\/td><td>Are test scripts or assertions unreliable?<\/td><\/tr><tr><td>Dependencies<\/td><td>Are external services causing failures?<\/td><\/tr><tr><td>Deployment process<\/td><td>Are configuration or execution steps inconsistent?<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Useful analysis techniques include:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pareto analysis:<\/strong> Identify which categories contribute most heavily to the problem.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Five Whys:<\/strong> Repeatedly investigate why a failure occurs until a supported underlying cause is identified.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Fishbone analysis:<\/strong> Group possible causes into categories such as people, process, technology, infrastructure, and information.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Failure Mode and Effects Analysis (FMEA):<\/strong> Identify potential failure modes and prioritize investigation based on their consequences and other relevant risk factors.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Root cause analysis should be supported by evidence, not merely by the opinions of the most experienced participants.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">7. Step 4 \u2014 Improve the process<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">After identifying credible root causes, select interventions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Suppose the investigation finds that shared mutable test data causes a significant proportion of intermittent integration test failures.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Possible improvements include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Generate isolated test data for each run.<\/li>\n\n\n\n<li>Reset fixtures before execution.<\/li>\n\n\n\n<li>Remove dependencies between parallel tests.<\/li>\n\n\n\n<li>Provide deterministic service responses where appropriate.<\/li>\n\n\n\n<li>Manage environment configuration consistently.<\/li>\n\n\n\n<li>Eliminate uncontrolled shared accounts.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Implement changes incrementally where possible.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Use a controlled comparison or pilot to evaluate whether the proposed changes actually improve outcomes.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Improvement validation<\/h3>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Indicator<\/th><th>Before<\/th><th>After<\/th><th>Interpretation<\/th><\/tr><tr><td>Validation defects<\/td><td>72<\/td><td>20<\/td><td>Lower observed count<\/td><\/tr><tr><td>DPMO<\/td><td>18,000<\/td><td>5,000<\/td><td>Lower observed defect density<\/td><\/tr><tr><td>Test environment changes<\/td><td>Uncontrolled<\/td><td>Version-controlled<\/td><td>Better reproducibility<\/td><\/tr><tr><td>Test data ownership<\/td><td>Shared<\/td><td>Isolated<\/td><td>Reduced interference risk<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The values in this example are hypothetical. They illustrate the evaluation method rather than a real organizational result.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">8. Step 5 \u2014 Control the improved process<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Improvements can disappear if they are not incorporated into routine operations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A control plan should establish:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Process ownership.<\/li>\n\n\n\n<li>Standard operating practices.<\/li>\n\n\n\n<li>Measurement definitions.<\/li>\n\n\n\n<li>Monitoring responsibilities.<\/li>\n\n\n\n<li>Appropriate control limits or thresholds.<\/li>\n\n\n\n<li>Actions when performance deteriorates.<\/li>\n\n\n\n<li>Periodic process review.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">For deployment quality, this may include automated preflight checks, configuration validation, change reviews, and a defined response to abnormal failure patterns.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Statistical process control can be useful when there is sufficient stable data and the selected statistical method fits the process.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Avoid drawing conclusions from control charts that ignore changing workloads, small sample sizes, or inappropriate assumptions about the data.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">9. Useful Six Sigma metrics for software engineering<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Metric<\/th><th>Purpose<\/th><th>Important caution<\/th><\/tr><tr><td>DPMO<\/td><td>Measure defects relative to defined opportunities<\/td><td>Requires consistent opportunity definitions<\/td><\/tr><tr><td>Defect density<\/td><td>Compare defects against a defined size measure<\/td><td>Size measures must be comparable<\/td><\/tr><tr><td>Change failure rate<\/td><td>Evaluate production change reliability<\/td><td>Define what qualifies as a failure<\/td><\/tr><tr><td>First-pass yield<\/td><td>Measure work completed without rework<\/td><td>Define the unit and acceptance criteria<\/td><\/tr><tr><td>Rework percentage<\/td><td>Evaluate wasted effort<\/td><td>Requires reliable effort classification<\/td><\/tr><tr><td>Defect escape rate<\/td><td>Identify defects found after release<\/td><td>Use consistent observation windows<\/td><\/tr><tr><td>Mean recovery time<\/td><td>Evaluate recovery behavior<\/td><td>Distinguish different incident classes<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">No single metric adequately represents all aspects of software quality.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">10. Integrating Six Sigma with DevOps<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">DMAIC can fit naturally into engineering retrospectives and improvement initiatives.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Define:<\/strong> Identify recurring CI\/CD failures.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Measure:<\/strong> Establish failure frequency and rework costs.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Analyze:<\/strong> Investigate build logs and failure categories.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Improve:<\/strong> Implement targeted engineering changes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Control:<\/strong> Establish monitoring and regression prevention measures.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A structured DMAIC project is particularly helpful when repeated quick fixes have failed to solve a significant problem.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">11. Integrating Six Sigma with TMMi<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">TMMi provides a maturity improvement model, while Six Sigma offers methods for investigating measurable problems.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">At higher TMMi maturity levels, quantitative evaluation, defect prevention, and process optimization become increasingly important.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Six Sigma techniques can support those objectives, provided the methods are appropriate for the data and problem being examined.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">12. Common mistakes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Applying DPMO to arbitrary software metrics.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Defects and opportunities must be defined consistently.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Claiming Six Sigma performance from a small sample.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Process capability and statistical conclusions require appropriate evidence and assumptions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Using metrics to blame individuals.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The objective is to improve the process, not punish the people reporting defects.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Skipping measurement validation.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Incorrect or inconsistent measurement can make an improvement appear successful when it is not.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Confusing correlation with causation.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Improvement experiments should test whether the suspected intervention actually influences the result.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Six Sigma provides software organizations with a systematic method for reducing recurring defects and improving process performance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its value lies in disciplined problem definition, reliable measurement, evidence-based root cause analysis, controlled improvement, and long-term monitoring.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Used appropriately, Six Sigma complements Agile, DevOps, test maturity models, and software quality engineering.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Official resources<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/asq.org\/quality-resources\/six-sigma\">ASQ \u2014 Six Sigma<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/asq.org\/quality-resources\/dmaic\">ASQ \u2014 DMAIC methodology<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/asq.org\/quality-resources\/sixsigma\/tools\">ASQ \u2014 Six Sigma tools and techniques<\/a><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Methodology and conventional DPMO interpretation verified against the American Society for Quality. ASQ+2<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Tutorial 17 \u2014 CISQ: Automated Software Quality Measurement<\/h2>\n\n\n\n<h1 class=\"wp-block-heading\">CISQ Explained: How to Measure Software Structural Quality, Security, Reliability, Performance, and Maintainability<\/h1>\n\n\n\n<p class=\"wp-block-paragraph\">A software application can behave correctly during testing while still containing serious structural weaknesses.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, a system may pass its functional test suite but contain insecure data access paths, unmanaged resources, or architectural weaknesses that make future changes dangerous.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These problems are difficult to evaluate using functional testing alone.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The <strong>Consortium for Information &amp; Software Quality (CISQ)<\/strong> develops standardized approaches to automated software measurement, particularly software size and internal structural quality.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its work helps organizations evaluate technical risk, manage software quality, and establish measurable expectations for software delivered internally or by third-party suppliers.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">1. What is CISQ?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">CISQ is an industry consortium established to support standardized automated measurement of software quality and size.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its work addresses the problem of evaluating large software systems consistently across technologies and organizations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">CISQ is not a staged test process maturity framework like TMMi.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Instead, its measures support objective analysis of important internal software characteristics.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">CISQ&#8217;s structural quality work is represented in <strong>ISO\/IEC 5055:2021<\/strong>, which standardizes automated source code quality measures.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2. Understanding ISO\/IEC 5055<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ISO\/IEC 5055 defines automated source code quality measures across four characteristics.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Characteristic<\/th><th>Principal concern<\/th><th>Example weakness<\/th><\/tr><tr><td>Security<\/td><td>Structural weaknesses that could be exploited<\/td><td>Unprotected access to sensitive data<\/td><\/tr><tr><td>Reliability<\/td><td>Weaknesses that could affect dependable operation<\/td><td>Improper resource management<\/td><\/tr><tr><td>Performance efficiency<\/td><td>Weaknesses affecting resource efficiency or performance<\/td><td>Inefficient architectural data access<\/td><\/tr><tr><td>Maintainability<\/td><td>Weaknesses that increase modification cost and difficulty<\/td><td>Problematic dependencies and structural complexity<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The standard focuses on significant weaknesses in the construction of software.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its purpose is different from measuring whether a system satisfies a particular end-user functional requirement.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">3. How CISQ relates to ISO 25010<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ISO\/IEC 25010 defines a broad product quality model.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">ISO\/IEC 5055 provides standardized source code measures addressing four important product quality characteristics.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The two standards are complementary.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>ISO\/IEC 25010:<\/strong> Security is an important characteristic of software product quality.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>ISO\/IEC 5055:<\/strong> Relevant structural security weaknesses can be identified and counted using specified automated source code measurement rules.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Likewise, ISO 25010 describes reliability as a product quality characteristic, while ISO 5055 contributes source-level evidence about weaknesses that could undermine reliability.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Software teams should combine structural analysis with appropriate functional, security, performance, and operational testing.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. Understanding structural quality<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Structural quality refers to properties of how software is designed and implemented.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These include issues at both component and architectural levels.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Consider a web service that receives requests from users and retrieves protected account data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A sound architecture typically requires relevant authentication and authorization checks before sensitive resources can be accessed.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A structural weakness may arise when an unintended route allows a request to reach protected data without passing through the necessary controls.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A functional test suite could miss this problem if it only tests the expected application workflow.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Architecture analysis and security testing should examine such unintended paths.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">5. Implementing CISQ-oriented measurement<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Step 1: Define the measurement scope<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Identify the software systems and components under evaluation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Record:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Application and component boundaries.<\/li>\n\n\n\n<li>Programming languages.<\/li>\n\n\n\n<li>Frameworks.<\/li>\n\n\n\n<li>Relevant architectural dependencies.<\/li>\n\n\n\n<li>Third-party components.<\/li>\n\n\n\n<li>Source code availability.<\/li>\n\n\n\n<li>Applicable technology and tool limitations.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Do not assume that a scanner supports every language, component, or architectural relationship in the application.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 2: Establish structural quality objectives<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Create quality objectives aligned to business risk.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Examples include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Identify and eliminate critical security weaknesses.<\/li>\n\n\n\n<li>Reduce recurring reliability-related source code problems.<\/li>\n\n\n\n<li>Identify structural weaknesses that create performance risk.<\/li>\n\n\n\n<li>Prevent significant maintainability problems from increasing.<\/li>\n\n\n\n<li>Establish consistent software quality expectations for suppliers.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Measurement objectives should distinguish serious structural issues from minor style preferences.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Step 3: Choose appropriate analysis technology<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Different tools provide different kinds of evidence.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Tool or approach<\/th><th>Practical contribution<\/th><\/tr><tr><td>Semgrep Community Edition<\/td><td>Source code pattern analysis using configurable rules<\/td><\/tr><tr><td>SonarQube Community Build<\/td><td>Source code quality and selected security analysis<\/td><\/tr><tr><td>CodeQL<\/td><td>Query-based code security analysis for supported languages<\/td><\/tr><tr><td>Dependency analysis tools<\/td><td>Identify known vulnerabilities in third-party dependencies<\/td><\/tr><tr><td>Architecture review<\/td><td>Evaluate dependencies, trust boundaries, and design weaknesses<\/td><\/tr><tr><td>ISO 5055-capable analysis technology<\/td><td>Produce applicable standardized structural measures where supported<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Important:<\/strong> General-purpose static analysis tools do not automatically provide complete ISO\/IEC 5055 measurements.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A vendor&#8217;s specific implementation, supported rule coverage, analysis scope, and conformance claims must be verified before using a tool to produce formal ISO 5055 measures.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">6. Practical example: Start with Semgrep Community Edition<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Semgrep Community Edition is an open-source static analysis engine that can help identify selected source code weaknesses.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Install it in a Python virtual environment:<\/p>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">python3 -m venv .venv\nsource .venv\/bin\/activate\n\npython -m pip install semgrep<\/code><\/span><\/pre>\n\n\n<p class=\"wp-block-paragraph\">On Windows PowerShell:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-9\" data-shcb-language-name=\"CSS\" data-shcb-language-slug=\"css\"><span><code class=\"hljs language-css\"><span class=\"hljs-selector-class\">.venv<\/span>\\<span class=\"hljs-selector-tag\">Scripts<\/span>\\<span class=\"hljs-selector-tag\">Activate<\/span><span class=\"hljs-selector-class\">.ps1<\/span><\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-9\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">CSS<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">css<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">Scan the repository:<\/p>\n\n\n<pre class=\"wp-block-code\"><span><code class=\"hljs\">semgrep scan --config auto .<\/code><\/span><\/pre>\n\n\n<p class=\"wp-block-paragraph\">This uses Semgrep&#8217;s automatic configuration, which may retrieve applicable rules from the Semgrep registry and therefore can require network access.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Review the findings and determine:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Is the issue applicable?<\/li>\n\n\n\n<li>Can the code path be reached?<\/li>\n\n\n\n<li>What is the potential impact?<\/li>\n\n\n\n<li>Is the finding a false positive?<\/li>\n\n\n\n<li>Does the proposed remediation eliminate the weakness?<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The scan provides useful analysis evidence, but it is <strong>not an ISO 5055 conformance assessment<\/strong>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">7. Example structural security weakness<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Consider a Python application using SQLite.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An unsafe approach might construct a SQL query by concatenating untrusted input:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-10\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\">query = (\n    <span class=\"hljs-string\">\"SELECT id FROM users WHERE email = '\"<\/span>\n    + email\n    + <span class=\"hljs-string\">\"'\"<\/span>\n)<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-10\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">This construction can create SQL injection risk.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A safer approach uses parameterized queries:<\/p>\n\n\n<pre class=\"wp-block-code\" aria-describedby=\"shcb-language-11\" data-shcb-language-name=\"JavaScript\" data-shcb-language-slug=\"javascript\"><span><code class=\"hljs language-javascript\">cursor.execute(\n    <span class=\"hljs-string\">\"SELECT id FROM users WHERE email = ?\"<\/span>,\n    (email,)\n)<\/code><\/span><small class=\"shcb-language\" id=\"shcb-language-11\"><span class=\"shcb-language__label\">Code language:<\/span> <span class=\"shcb-language__name\">JavaScript<\/span> <span class=\"shcb-language__paren\">(<\/span><span class=\"shcb-language__slug\">javascript<\/span><span class=\"shcb-language__paren\">)<\/span><\/small><\/pre>\n\n\n<p class=\"wp-block-paragraph\">The database driver handles the supplied value as a parameter instead of interpreting it as SQL syntax.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This example illustrates the type of weakness static analysis can help detect. It does not mean that identifying this single weakness satisfies any particular ISO 5055 structural quality measure.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">8. Establish a structural quality baseline<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Run analysis against a defined software revision and record the results.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A practical dashboard might contain:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Dimension<\/th><th>Baseline measure<\/th><th>Improvement objective<\/th><\/tr><tr><td>Security<\/td><td>Confirmed relevant weaknesses<\/td><td>Eliminate unacceptable weaknesses<\/td><\/tr><tr><td>Reliability<\/td><td>Confirmed reliability-related weaknesses<\/td><td>Reduce recurring reliability risks<\/td><\/tr><tr><td>Performance efficiency<\/td><td>Confirmed structural performance issues<\/td><td>Address material performance risks<\/td><\/tr><tr><td>Maintainability<\/td><td>Confirmed structural maintenance issues<\/td><td>Reduce high-impact maintenance problems<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Use the actual measurement rules specified by ISO 5055 when producing standardized ISO 5055 results.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Generic static analysis finding counts, severity labels, and custom dashboards should be clearly distinguished from formal standard measures.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">9. Classify and prioritize findings<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Not every finding has equal business importance.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Consider:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Potential impact.<\/li>\n\n\n\n<li>Exploitability or triggering conditions.<\/li>\n\n\n\n<li>Component criticality.<\/li>\n\n\n\n<li>Reachability.<\/li>\n\n\n\n<li>Exposure.<\/li>\n\n\n\n<li>Likelihood of operational failure.<\/li>\n\n\n\n<li>Cost of corrective maintenance.<\/li>\n\n\n\n<li>Confidence in the analysis result.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">A significant security weakness in an externally exposed authentication service may require more urgent action than a low-impact maintainability issue in an isolated utility.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Prioritization should be based on actual risk and the organization&#8217;s agreed acceptance criteria.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">10. Implement structural quality gates<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A quality gate is a defined condition that determines whether a change or release can proceed.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Examples of policy rules include:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Security gate:<\/strong> A confirmed, unacceptable security weakness must be resolved or handled under an explicitly authorized risk-exception process.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Reliability gate:<\/strong> Critical resource-management weaknesses in important services require remediation before release.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Maintainability gate:<\/strong> New high-impact maintainability problems require engineering review.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Performance gate:<\/strong> Structural performance findings affecting critical paths must be investigated and evaluated against performance objectives.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These gates are examples of organizational policy. Their thresholds are not automatically defined by ISO 5055.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">11. Integrate structural quality into CI\/CD<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A practical workflow is:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Developers submit a source code change.<\/li>\n\n\n\n<li>Static analysis examines applicable code.<\/li>\n\n\n\n<li>Findings are classified and reviewed.<\/li>\n\n\n\n<li>Confirmed issues are linked to code changes or remediation work.<\/li>\n\n\n\n<li>Quality policies determine whether additional action is required.<\/li>\n\n\n\n<li>Approved releases retain relevant evidence.<\/li>\n\n\n\n<li>Periodic analysis examines accumulated structural risk.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Early analysis can reduce rework by identifying weaknesses close to the point where they are introduced.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Whole-system analysis remains important because some structural problems involve architectural interactions that cannot be identified from an isolated code change.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">12. Measure trends responsibly<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Useful management questions include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Are significant structural weaknesses decreasing?<\/li>\n\n\n\n<li>Are new changes introducing recurring weakness patterns?<\/li>\n\n\n\n<li>Which components carry the greatest risk?<\/li>\n\n\n\n<li>Are remediation activities improving the product?<\/li>\n\n\n\n<li>Are supplier-delivered systems meeting contractual quality expectations?<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Comparisons should account for changes in codebase size, technology, scan coverage, rule sets, and system boundaries.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">An increase in findings after enabling a more capable analyzer does not necessarily indicate that software quality has deteriorated.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It may reflect improved detection.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">13. CISQ and software sizing<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">CISQ&#8217;s work extends beyond structural quality.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Automated Function Points provide a standardized way to measure the functional size of certain software applications.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">ISO\/IEC 19515:2019 addresses Automated Function Points.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Functional sizing and structural quality measurement answer different questions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Functional sizing:<\/strong> How much functional software capability is represented?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Structural quality measurement:<\/strong> What important weaknesses exist in the internal software construction?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Both can contribute to procurement, estimation, benchmarking, and engineering management when their applicability and limitations are understood.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">14. CISQ versus traditional code quality scores<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A generic static analysis score may combine complexity, duplication, coding conventions, and selected security checks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">ISO 5055 focuses on defined structural weaknesses that have important implications for security, reliability, performance efficiency, and maintainability.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">These concepts should not be treated as interchangeable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A codebase with a high tool-generated quality rating can still contain serious architectural defects.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Likewise, an isolated low-severity rule violation does not necessarily represent significant operational risk.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">15. Common mistakes<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Assuming all static analysis tools produce ISO 5055-compliant results.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Verify support for the actual standardized measures and their scope.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Treating source code analysis as complete security testing.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Security assurance also requires appropriate architectural, configuration, integration, dynamic, and operational evaluation.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Comparing inconsistent scanning baselines.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Keep rule sets, scope, and measurement definitions comparable.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Measuring only total issue counts.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Prioritize confirmed weaknesses based on their potential consequences.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Mistake: Ignoring architectural relationships.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Important problems may exist across components and technologies rather than inside one source file.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">CISQ provides a valuable foundation for standardized automated software measurement.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">ISO\/IEC 5055 helps organizations evaluate significant structural weaknesses affecting security, reliability, performance efficiency, and maintainability.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When combined with broader product quality models, testing standards, and development process improvement practices, structural measurement can help organizations make better decisions about software risk, technical debt, and long-term maintainability.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Official resources<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><a href=\"https:\/\/www.it-cisq.org\/standards\/\">CISQ \u2014 Standards<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.it-cisq.org\/standards\/code-quality-standards\/\">CISQ \u2014 ISO 5055 structural quality measures<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.iso.org\/standard\/80623.html\">ISO\/IEC 5055 \u2014 Automated source code quality measures<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/www.iso.org\/standard\/65378.html\">ISO\/IEC 19515 \u2014 Automated Function Points<\/a><\/li>\n\n\n\n<li><a href=\"https:\/\/semgrep.dev\/products\/community-edition\/\">Semgrep Community Edition<\/a><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Verified against CISQ, ISO\/IEC 5055, ISO\/IEC 19515, and Semgrep&#8217;s official Community Edition guidance. CISQ+3<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How all seven frameworks work together<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">These frameworks should not be treated as seven competing alternatives. Each answers a different quality engineering question.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">#chatgpt-mermaid-_r_m6_{font-family:-apple-system-body,ui-sans-serif,-apple-system,system-ui,&#8221;Segoe UI&#8221;,Helvetica,&#8221;Apple Color Emoji&#8221;,Arial,sans-serif,&#8221;Segoe UI Emoji&#8221;,&#8221;Segoe UI Symbol&#8221;;font-size:16px;fill:rgb(13, 13, 13);}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#chatgpt-mermaid-_r_m6_ .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#chatgpt-mermaid-_r_m6_ .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#chatgpt-mermaid-_r_m6_ .error-icon{fill:rgb(243, 243, 243);}#chatgpt-mermaid-_r_m6_ .error-text{fill:rgb(13, 13, 13);stroke:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .edge-thickness-normal{stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .edge-thickness-thick{stroke-width:3.5px;}#chatgpt-mermaid-_r_m6_ .edge-pattern-solid{stroke-dasharray:0;}#chatgpt-mermaid-_r_m6_ .edge-thickness-invisible{stroke-width:0;fill:none;}#chatgpt-mermaid-_r_m6_ .edge-pattern-dashed{stroke-dasharray:3;}#chatgpt-mermaid-_r_m6_ .edge-pattern-dotted{stroke-dasharray:2;}#chatgpt-mermaid-_r_m6_ .marker{fill:rgb(143, 143, 143);stroke:rgb(143, 143, 143);}#chatgpt-mermaid-_r_m6_ .marker.cross{stroke:rgb(143, 143, 143);}#chatgpt-mermaid-_r_m6_ svg{font-family:-apple-system-body,ui-sans-serif,-apple-system,system-ui,&#8221;Segoe UI&#8221;,Helvetica,&#8221;Apple Color Emoji&#8221;,Arial,sans-serif,&#8221;Segoe UI Emoji&#8221;,&#8221;Segoe UI Symbol&#8221;;font-size:16px;}#chatgpt-mermaid-_r_m6_ p{margin:0;}#chatgpt-mermaid-_r_m6_ .label{font-family:-apple-system-body,ui-sans-serif,-apple-system,system-ui,&#8221;Segoe UI&#8221;,Helvetica,&#8221;Apple Color Emoji&#8221;,Arial,sans-serif,&#8221;Segoe UI Emoji&#8221;,&#8221;Segoe UI Symbol&#8221;;color:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .cluster-label text{fill:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .cluster-label span{color:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .cluster-label span p{background-color:transparent;}#chatgpt-mermaid-_r_m6_ .label text,#chatgpt-mermaid-_r_m6_ span{fill:rgb(13, 13, 13);color:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .node rect,#chatgpt-mermaid-_r_m6_ .node circle,#chatgpt-mermaid-_r_m6_ .node ellipse,#chatgpt-mermaid-_r_m6_ .node polygon,#chatgpt-mermaid-_r_m6_ .node path{fill:rgb(222, 234, 251);stroke:rgb(83, 154, 248);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .rough-node .label text,#chatgpt-mermaid-_r_m6_ .node .label text,#chatgpt-mermaid-_r_m6_ .image-shape .label,#chatgpt-mermaid-_r_m6_ .icon-shape .label{text-anchor:middle;}#chatgpt-mermaid-_r_m6_ .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .rough-node .label,#chatgpt-mermaid-_r_m6_ .node .label,#chatgpt-mermaid-_r_m6_ .image-shape .label,#chatgpt-mermaid-_r_m6_ .icon-shape .label{text-align:center;}#chatgpt-mermaid-_r_m6_ .node.clickable{cursor:pointer;}#chatgpt-mermaid-_r_m6_ .root .anchor path{fill:rgb(143, 143, 143)!important;stroke-width:0;stroke:rgb(143, 143, 143);}#chatgpt-mermaid-_r_m6_ .arrowheadPath{fill:rgb(143, 143, 143);}#chatgpt-mermaid-_r_m6_ .edgePath .path{stroke:rgb(143, 143, 143);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .flowchart-link{stroke:rgb(143, 143, 143);fill:none;}#chatgpt-mermaid-_r_m6_ .edgeLabel{background-color:rgb(252, 252, 252);text-align:center;}#chatgpt-mermaid-_r_m6_ .edgeLabel p{background-color:rgb(252, 252, 252);}#chatgpt-mermaid-_r_m6_ .edgeLabel rect{opacity:0.5;background-color:rgb(252, 252, 252);fill:rgb(252, 252, 252);}#chatgpt-mermaid-_r_m6_ .labelBkg{background-color:rgba(252, 252, 252, 0.5);}#chatgpt-mermaid-_r_m6_ .cluster rect{fill:rgb(243, 243, 243);stroke:rgba(0, 0, 0, 0.1);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .cluster text{fill:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .cluster span{color:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:-apple-system-body,ui-sans-serif,-apple-system,system-ui,&#8221;Segoe UI&#8221;,Helvetica,&#8221;Apple Color Emoji&#8221;,Arial,sans-serif,&#8221;Segoe UI Emoji&#8221;,&#8221;Segoe UI Symbol&#8221;;font-size:12px;background:rgb(243, 243, 243);border:1px solid rgba(0, 0, 0, 0.1);border-radius:2px;pointer-events:none;z-index:100;}#chatgpt-mermaid-_r_m6_ .flowchartTitleText{text-anchor:middle;font-size:18px;fill:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ rect.text{fill:none;stroke-width:0;}#chatgpt-mermaid-_r_m6_ .icon-shape,#chatgpt-mermaid-_r_m6_ .image-shape{background-color:rgb(252, 252, 252);text-align:center;}#chatgpt-mermaid-_r_m6_ .icon-shape p,#chatgpt-mermaid-_r_m6_ .image-shape p{background-color:rgb(252, 252, 252);padding:2px;}#chatgpt-mermaid-_r_m6_ .icon-shape .label rect,#chatgpt-mermaid-_r_m6_ .image-shape .label rect{opacity:0.5;background-color:rgb(252, 252, 252);fill:rgb(252, 252, 252);}#chatgpt-mermaid-_r_m6_ .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#chatgpt-mermaid-_r_m6_ .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#chatgpt-mermaid-_r_m6_ .node .neo-node{stroke:rgb(83, 154, 248);}#chatgpt-mermaid-_r_m6_ [data-look=&#8221;neo&#8221;].node rect,#chatgpt-mermaid-_r_m6_ [data-look=&#8221;neo&#8221;].cluster rect,#chatgpt-mermaid-_r_m6_ [data-look=&#8221;neo&#8221;].node polygon{stroke:url(#chatgpt-mermaid-_r_m6_-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#chatgpt-mermaid-_r_m6_ [data-look=&#8221;neo&#8221;].swimlane.cluster rect{filter:none;}#chatgpt-mermaid-_r_m6_ [data-look=&#8221;neo&#8221;].node path{stroke:url(#chatgpt-mermaid-_r_m6_-gradient);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ [data-look=&#8221;neo&#8221;].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#chatgpt-mermaid-_r_m6_ [data-look=&#8221;neo&#8221;].node .neo-line path{stroke:rgb(83, 154, 248);filter:none;}#chatgpt-mermaid-_r_m6_ [data-look=&#8221;neo&#8221;].node circle{stroke:url(#chatgpt-mermaid-_r_m6_-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#chatgpt-mermaid-_r_m6_ [data-look=&#8221;neo&#8221;].node circle .state-start{fill:#000000;}#chatgpt-mermaid-_r_m6_ [data-look=&#8221;neo&#8221;].icon-shape .icon{fill:url(#chatgpt-mermaid-_r_m6_-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#chatgpt-mermaid-_r_m6_ [data-look=&#8221;neo&#8221;].icon-shape .icon-neo path{stroke:url(#chatgpt-mermaid-_r_m6_-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#chatgpt-mermaid-_r_m6_ .node text{font-size:14px;font-weight:600;letter-spacing:normal;fill:rgb(0, 79, 153);}#chatgpt-mermaid-_r_m6_ .edgeLabels text{font-size:13px;font-weight:600;letter-spacing:-0.08px;fill:rgb(0, 79, 153);}#chatgpt-mermaid-_r_m6_ .node tspan[font-weight=&#8221;normal&#8221;],#chatgpt-mermaid-_r_m6_ .edgeLabels tspan[font-weight=&#8221;normal&#8221;]{font-weight:600;}#chatgpt-mermaid-_r_m6_ .edgeLabel .label rect{opacity:1;rx:13px;ry:13px;fill:rgb(245, 250, 255);stroke:rgb(206, 219, 229);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .node rect,#chatgpt-mermaid-_r_m6_ .node circle,#chatgpt-mermaid-_r_m6_ .node ellipse,#chatgpt-mermaid-_r_m6_ .node polygon,#chatgpt-mermaid-_r_m6_ .node path{fill:rgb(229, 243, 255);stroke:rgba(0, 0, 0, 0.1);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .node rect{rx:16px;ry:16px;}#chatgpt-mermaid-_r_m6_ .mermaid-decision .label-container{fill:rgb(245, 250, 255);stroke:rgb(206, 219, 229);stroke-dasharray:2,2;}#chatgpt-mermaid-_r_m6_ .edgePaths .flowchart-link{stroke:rgb(143, 143, 143);stroke-width:1px;stroke-linecap:round;stroke-linejoin:round;}#chatgpt-mermaid-_r_m6_ .marker{fill:rgb(143, 143, 143);stroke:rgb(143, 143, 143);}#chatgpt-mermaid-_r_m6_ :root{&#8211;mermaid-font-family:-apple-system-body,ui-sans-serif,-apple-system,system-ui,&#8221;Segoe UI&#8221;,Helvetica,&#8221;Apple Color Emoji&#8221;,Arial,sans-serif,&#8221;Segoe UI Emoji&#8221;,&#8221;Segoe UI Symbol&#8221;;}Business Needs and ProductRisksISO\/IEC 25010 &#8211; DefineQualityIEEE 730 &#8211; Govern QualityAssuranceISO\/IEC\/IEEE 29119 &#8211; Planand Perform TestingQuality Evidence and ReleaseDecisionsTMMi &#8211; ImproveOrganizational MaturityTPI NEXT &#8211; Improve SpecificTest CapabilitiesCISQ \/ ISO 5055 &#8211; MeasureStructural QualitySix Sigma &#8211; Analyze andReduce Recurring DefectsContinuous Improvement<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Conceptual integration of software quality standards and process improvement methods; this is an implementation model, not an official standards architecture.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">A practical 90-day implementation program<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This is how I would organize an initial quality engineering improvement initiative using the seven frameworks.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Phase<\/th><th>Frameworks<\/th><th>Primary work<\/th><th>Expected output<\/th><\/tr><tr><td>Days 1\u201315<\/td><td>ISO 25010, IEEE 730<\/td><td>Identify product risks, quality objectives, and assurance responsibilities<\/td><td>Quality charter and assurance baseline<\/td><\/tr><tr><td>Days 16\u201330<\/td><td>TMMi, TPI NEXT<\/td><td>Assess testing maturity, weaknesses, and improvement priorities<\/td><td>Assessment findings and improvement backlog<\/td><\/tr><tr><td>Days 31\u201345<\/td><td>ISO 29119<\/td><td>Improve testing strategy, design, documentation, and traceability<\/td><td>Consistent testing practices<\/td><\/tr><tr><td>Days 46\u201360<\/td><td>CISQ, ISO 25010<\/td><td>Introduce relevant structural, functional, and non-functional checks<\/td><td>Quality measurement baseline<\/td><\/tr><tr><td>Days 61\u201375<\/td><td>Six Sigma<\/td><td>Investigate recurring defects and test targeted improvements<\/td><td>Root cause analysis and corrective actions<\/td><\/tr><tr><td>Days 76\u201390<\/td><td>TMMi, TPI NEXT, IEEE 730<\/td><td>Evaluate results and institutionalize effective practices<\/td><td>Continuous improvement roadmap<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">This schedule establishes foundational practices. It does not represent an official certification program or guarantee that any specific maturity level will be achieved.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Which framework should an organization adopt first?<\/h2>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><th>Organization&#8217;s immediate challenge<\/th><th>Best starting point<\/th><\/tr><tr><td>Testing differs significantly between teams<\/td><td>TMMi<\/td><\/tr><tr><td>A few testing bottlenecks are causing substantial delays<\/td><td>TPI NEXT<\/td><\/tr><tr><td>Test documentation and execution lack consistency<\/td><td>ISO\/IEC\/IEEE 29119<\/td><\/tr><tr><td>Non-functional requirements and quality objectives are unclear<\/td><td>ISO\/IEC 25010<\/td><\/tr><tr><td>Engineering quality governance and accountability are weak<\/td><td>IEEE 730<\/td><\/tr><tr><td>Recurring defects persist despite repeated fixes<\/td><td>Six Sigma<\/td><\/tr><tr><td>Source code structural risks and technical debt are poorly understood<\/td><td>CISQ \/ ISO\/IEC 5055<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">For an organization starting from scratch, a practical sequence is to define product quality with ISO 25010, establish assurance ownership using IEEE 730, organize testing around ISO 29119, and use TMMi or TPI NEXT to guide process improvements. CISQ measures and Six Sigma techniques can then strengthen the evidence and continuous improvement process.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The objective is not to implement the largest possible collection of standards. It is to create a quality engineering system in which requirements are measurable, testing is effective, structural risks are understood, assurance decisions are traceable, and recurring problems lead to demonstrable improvements.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Seven in-depth technical guides covering frameworks #11\u2013#17 \u00b7 Beginner to Advanced \u00b7 Software QA, Quality Engineering, DevSecOps, and Process Improvement Software quality is not simply about executing&#8230; <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_joinchat":[],"footnotes":""},"categories":[11138],"tags":[],"class_list":["post-79532","post","type-post","status-publish","format-standard","hentry","category-best-tools"],"_links":{"self":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/79532","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/comments?post=79532"}],"version-history":[{"count":1,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/79532\/revisions"}],"predecessor-version":[{"id":79533,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/posts\/79532\/revisions\/79533"}],"wp:attachment":[{"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/media?parent=79532"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/categories?post=79532"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.devopsschool.com\/blog\/wp-json\/wp\/v2\/tags?post=79532"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}