Seven in-depth technical guides covering frameworks #11–#17 · Beginner to Advanced · Software QA, Quality Engineering, DevSecOps, and Process Improvement

Software quality is not simply about executing test cases, finding bugs, or achieving a high automation percentage. A mature quality engineering organization systematically defines quality, prevents defects, measures risks, improves testing practices, and provides objective evidence that its products are fit for purpose.

The seven frameworks in this series address different parts of that challenge.

#	Framework	Primary purpose
11	TMMi	Assess and improve organizational testing maturity
12	TPI NEXT	Identify and prioritize practical test process improvements
13	ISO/IEC/IEEE 29119	Standardize software testing processes, techniques, and documentation
14	ISO/IEC 25010	Define and evaluate software product quality
15	IEEE 730	Establish software quality assurance processes and governance
16	Six Sigma	Reduce process variation, recurring defects, and waste
17	CISQ	Measure software structural quality and technical risk

Important standards updates: TMMi released Model 2.0 in June 2026, adding updated Agile, DevOps, quality engineering, and AI guidance. IEEE 730-2026 was published in August 2026 and supersedes IEEE 730-2014. ISO/IEC 25010:2023 uses nine product quality characteristics, rather than the eight in its earlier edition. These distinctions matter when using published tutorials as implementation or assessment references. TMMi+2

The tutorials below use a fictional software product throughout, so the examples connect to a realistic engineering workflow. Implementation checklists, sample targets, and templates are practical guidance—not quotations from, or substitutes for, the full licensed standards.

Tutorial 11 — TMMi: A Complete Guide to Test Maturity Model Integration

TMMi Explained: How to Assess, Improve, and Mature Your Software Testing Process

A software development team can have hundreds of automated tests, experienced QA engineers, and sophisticated CI/CD pipelines—and still struggle with software quality.

Why? Because testing maturity is not determined by the number of tests or the tools an organization owns. It depends on whether testing activities are planned, repeatable, measurable, integrated into development, and continuously improved.

Test Maturity Model integration (TMMi) provides a structured approach to developing these capabilities.

1. What is TMMi?

TMMi is a test process improvement and assessment model maintained by the TMMi Foundation. It provides maturity levels, process areas, goals, and practices that help organizations understand how effectively they manage software testing.

TMMi is especially useful for:

Software development organizations operating multiple teams.
QA and quality engineering departments.
Enterprises introducing Agile and DevOps.
Organizations seeking consistent testing governance.
Outsourcing organizations evaluating suppliers.
Engineering teams preparing for formal test maturity assessments.

TMMi Model 2.0 incorporates contemporary Agile and DevOps practices, quality engineering concepts, and guidance related to artificial intelligence.

The model is technology-neutral. It does not mandate Selenium, Playwright, Jenkins, or a specific test management platform.

2. Understanding the five TMMi maturity levels

Level	Name	Meaning
1	Initial	Testing practices are inconsistent, reactive, or dependent on individual effort.
2	Managed	Teams establish testing policies, planning, controls, and repeatable execution.
3	Defined	Consistent organizational testing practices are established and integrated with development.
4	Measured	Testing and product quality are managed using quantitative objectives and measurement.
5	Optimization	The organization continuously improves its practices through defect prevention and process optimization.

Level 1 is the starting point and has no required process areas. Progression through the other levels depends on demonstrating that their applicable process goals have been achieved.

Level 2: Managed

At this level, testing stops being an informal activity performed immediately before release.

The six process areas in TMMi 2.0 are:

Test Policy and Strategy
Test Planning
Test Monitoring and Control
Test Design and Execution
Test Environment
Implementation and Habit

Practical example: An API development team introduces a risk-based test strategy, defines test environments, records test results, identifies accountable owners, and monitors unresolved defects before deployment.

Level 3: Defined

Testing practices become consistent across the organizational unit.

The five process areas are:

Test Organization
Test Training Program
Test Process and Integration
Non-Functional Testing
Peer Reviews

At this stage, teams share organizational testing guidance while retaining the ability to tailor their implementation to different products.

Non-functional testing—including performance, reliability, and security-related testing—becomes an organized part of engineering work rather than an occasional exercise.

Level 4: Measured

The two process areas are:

Test Measurement
Product Quality Evaluation

Organizations establish meaningful quantitative objectives and use collected evidence to evaluate testing effectiveness and product quality.

The distinction is important: merely collecting dashboards does not make a testing process quantitatively managed.

Level 5: Optimization

The three process areas are:

Defect Prevention
Quality Control
Test Process Optimization

The focus moves beyond finding defects to understanding why they occur and reducing their recurrence.

TMMi 2.0 therefore covers 16 process areas across Levels 2–5.

3. How to implement TMMi step by step

Consider a fictional connected-vehicle platform called FleetLink. The product includes a mobile application, cloud APIs, and vehicle-status synchronization services.

The engineering organization wants fewer production incidents and more predictable releases.

Step 1: Establish the assessment scope

Decide whether the initiative covers one delivery team, a business unit, or the broader engineering organization.

Document:

Systems and services in scope.
Teams and development locations.
Product criticality and customer expectations.
Applicable external requirements.
Existing test tools and test environments.
Assessment sponsor and improvement owners.

Deliverable: A test maturity improvement charter.

A small pilot is often more manageable than attempting to standardize every team simultaneously.

Step 2: Establish the testing baseline

Collect evidence of how testing currently works.

Assessment topic	Evidence to inspect	Example weakness
Test strategy	Approved strategy and risk classification	No risk-based testing
Test planning	Sprint and release test plans	Testing starts too late
Test execution	Test logs, CI results, defect reports	Inconsistent results
Test environment	Environment ownership and configuration	Unreliable environments
Test measurement	Defect and quality trends	Metrics without definitions
Governance	Responsibilities and process reviews	No accountable owner

Use interviews alongside evidence from engineering repositories, CI systems, documentation, and issue tracking tools.

Do not assign an official TMMi maturity rating from a homemade checklist.

Step 3: Define a testing policy and strategy

A useful policy answers four questions:

What quality risks matter most?
What testing is required?
Who is responsible?
What evidence supports a release decision?

For example, FleetLink may classify its API operations into critical, high, medium, and low-risk categories.

Critical operations might require additional integration testing, failure recovery tests, security analysis, and documented release approval.

Step 4: Introduce risk-based test planning

Each feature should have an identified test approach based on business impact and failure likelihood.

A simple risk-prioritization method is:

Risk score = Likelihood × Impact

For example:

Feature	Likelihood (1–5)	Impact (1–5)	Score
Vehicle-status synchronization	4	4	16
Notification preferences	3	2	6
Account authorization	3	5	15

This is an illustrative prioritization technique, not a prescribed TMMi scoring formula.

Risk classification should influence test depth, test environments, release acceptance, and review requirements.

Step 5: Improve test design and execution

Create a repeatable testing workflow:

Analyze requirements and relevant risks.
Design test cases using suitable test techniques.
Prepare representative environments and controlled test data.
Execute automated and exploratory tests.
Record results and investigate failures.
Resolve defects and perform regression testing.
Report residual risks before release.

A test is not adequately evidenced merely because a CI job displays a green status.

Retain enough information to identify the tested software version, environment, test data, execution outcome, and relevant defects.

Step 6: Establish useful quality metrics

Start with a small set of meaningful indicators.

Defect escape rate

Escaped defects / (Pre-release defects + Escaped defects) × 100

Use defects discovered within a clearly defined release and observation window.

Critical requirement coverage

Critical requirements with accepted verification evidence / Total critical requirements × 100

Test automation stability

Track inconsistent test outcomes, environment failures, and recurring false alarms.

Defect resolution time

Measure elapsed time from defect reporting to verified resolution, preferably segmented by severity.

Change failure rate

Track the proportion of production changes causing defined failures requiring remediation.

Never use a single metric to declare product quality acceptable.

Step 7: Build continuous improvement into operations

Review testing weaknesses regularly. Assign owners to improvement actions and establish measurable outcomes.

For example:

Problem: Many tests fail because test data is inconsistent.
Root cause: Teams share mutable integration test accounts.
Improvement: Generate isolated test data for each test run.
Validation: Compare test instability before and after implementation.
Sustainment: Add ownership and regular monitoring.

This moves the organization toward sustainable maturity.

4. Example TMMi implementation roadmap

Period	Activities	Deliverables
Days 1–15	Define scope, interview teams, inspect evidence	Initial gap assessment
Days 16–30	Establish policy, risk categories, ownership	Test policy and strategy
Days 31–45	Improve planning, environments, and execution evidence	Repeatable test workflow
Days 46–60	Integrate testing with CI/CD and peer reviews	Quality gates and review process
Days 61–75	Implement metrics and dashboards	Measurement baseline
Days 76–90	Verify improvements and plan next cycle	Evidence-based improvement roadmap

This is a suggested improvement schedule, not a guaranteed timeframe for reaching or certifying a maturity level.

5. Recommended supporting tools

Purpose	Example tools
Test automation	Playwright, pytest, Robot Framework
CI/CD	Jenkins, GitHub Actions, GitLab CI/CD
Performance testing	k6, Apache JMeter
Static analysis	Semgrep, SonarQube
Test reporting	Allure Report, JUnit XML
Defect tracking	GitHub Issues, OpenProject
Documentation	Git-based Markdown, internal knowledge bases

TMMi evaluates the effectiveness of testing practices, not the popularity of the tools selected.

6. Informal assessment versus formal assessment

An informal assessment helps identify improvement opportunities and provides an indicative view of testing maturity.

A formal assessment involves additional rigor, corroborating evidence, and accredited assessment practices. Formal maturity ratings and recognized certification follow the TMMi Foundation’s assessment requirements.

Organizations should pursue formal assessment when customer, contractual, supplier, or strategic requirements justify it.

7. Common implementation mistakes

Mistake: Treating automation coverage as maturity.

Automation supports testing. It does not replace governance, risk management, process integration, and defect prevention.

Mistake: Creating documentation that nobody follows.

A lightweight, consistently applied process is more useful than a large collection of abandoned procedures.

Mistake: Targeting Level 5 immediately.

Improvement should build on stable foundational practices.

Mistake: Focusing only on QA engineers.

Software quality depends on developers, product owners, operations, security engineers, and business stakeholders.

8. How to measure success

A successful TMMi improvement program should demonstrate better risk visibility, reliable test execution, more predictable quality decisions, and measurable reductions in important quality failures.

The ultimate objective is not to achieve a maturity label. It is to develop a testing capability that consistently supports business and engineering objectives.

Conclusion

TMMi provides a structured way to transform inconsistent testing activities into a managed, measurable, continuously improving quality engineering capability.

Start with objective evidence, establish Level 2 fundamentals, integrate testing into development, and gradually expand quantitative management and defect prevention.

Official resources

Verified against the TMMi Foundation’s Model 2.0 and assessment guidance. TMMi Framework model V2.0+2

Tutorial 12 — TPI NEXT: Business-Driven Test Process Improvement

TPI NEXT Tutorial: Assess Your Testing Process, Find Weaknesses, and Build an Improvement Roadmap

Every software engineering organization experiences testing bottlenecks.

Some teams struggle with unreliable environments. Others have unclear testing responsibilities, inconsistent defect reporting, insufficient test planning, or poor coordination between developers and testers.

These problems do not necessarily require a complete organizational transformation. Often, targeted improvements in a few important areas produce meaningful results.

TPI NEXT (Test Process Improvement NEXT) provides an approach for identifying these weaknesses and improving testing capabilities in a controlled, business-driven manner.

1. What is TPI NEXT?

TPI NEXT is a test process improvement model developed by Sogeti. It enables organizations to assess different aspects of testing and prioritize improvements according to business objectives.

Unlike a model that concentrates primarily on one organization-wide maturity level, TPI NEXT provides a granular view of individual testing capabilities.

An organization may have sophisticated defect management while its test environments remain poorly controlled.

TPI NEXT helps make that imbalance visible.

Its principal elements include:

Key areas
Maturity levels
Checkpoints
Improvement suggestions
Enablers
Clusters
Test maturity matrix

Business drivers such as quality, delivery speed, cost, and operational risk influence improvement priorities.

2. Understand the 16 TPI NEXT key areas

The model contains 16 key areas spanning stakeholder relationships, test management, and testing practices.

#	Key area	Main concern
1	Stakeholder commitment	Management and business support for effective testing
2	Degree of involvement	Integration of testing into development activities
3	Test strategy	Testing objectives, priorities, and risk-based approach
4	Test organization	Test responsibilities and organizational structure
5	Communication	Effective communication between stakeholders
6	Reporting	Communication of progress, results, and risks
7	Test process management	Coordinating and controlling test activities
8	Estimating and planning	Reliable test effort and resource planning
9	Metrics	Objective measurement of testing and quality
10	Defect management	Defect identification, tracking, and resolution
11	Testware management	Control of testing artifacts and their relationships
12	Methodology practice	Consistent practical application of test methods
13	Tester professionalism	Competencies and skills required for testing
14	Test case design	Appropriate methods for identifying effective test cases
15	Test tools	Selection and effective use of supporting tools
16	Test environment	Availability, suitability, and control of test environments

A key advantage is that the model highlights capabilities that are sometimes neglected when improvement programs focus exclusively on automation.

For example, better test tools will not solve a fundamental lack of stakeholder involvement.

3. The TPI NEXT maturity levels

Each key area can progress through four maturity levels.

Level	Description
Initial	Activities are informal or insufficiently controlled.
Controlled	The capability is established and performed consistently.
Efficient	The capability operates effectively with better use of resources.
Optimizing	Continuous improvement sustains and enhances the capability.

TPI NEXT uses checkpoints to assess whether the relevant practices are established.

A key area’s maturity is determined by fulfilling the applicable checkpoints, not by assigning an arbitrary numerical average.

What are checkpoints?

Checkpoints describe conditions that should exist in the testing process.

For example, an assessment may inspect whether a team has a documented testing approach, communicates material testing risks, or maintains controlled test environments.

Assessors examine evidence to determine whether those conditions are actually met.

What are clusters?

Clusters group related improvements across multiple key areas into a sensible progression.

They help prevent teams from implementing advanced practices before the supporting fundamentals exist.

What are enablers?

Enablers highlight relationships between testing and adjacent development processes.

For example, effective automated regression testing depends on suitable deployment mechanisms, test data, environment stability, and development integration.

Improving those dependencies can make testing more effective.

4. Step-by-step TPI NEXT implementation

Step 1: Identify the business problem

Do not begin by asking, “How do we achieve the highest maturity level?”

Begin with a real business concern.

Examples:

Production defects are increasing.
Releases repeatedly miss planned dates.
Testing costs are rising.
Developers wait too long for test environments.
Customers experience regressions after deployments.

For this tutorial, assume FleetLink experiences frequent delays because integration test environments are unreliable.

Step 2: Define the assessment scope

Select the relevant team, product, project, or organizational unit.

Identify the systems, development processes, responsible stakeholders, and available evidence.

Make sure the assessment covers the work that directly affects the identified business objective.

Step 3: Assess the current capabilities

Use the official TPI NEXT maturity matrix and checkpoint guidance.

A simplified diagnostic summary might look like this:

Key area	Illustrative current state	Desired improvement
Stakeholder commitment	Controlled	Faster decisions on critical risks
Test strategy	Controlled	More effective risk prioritization
Estimating and planning	Initial	Repeatable release test estimates
Metrics	Initial	Consistent definitions and reporting
Defect management	Controlled	Faster root cause identification
Test environment	Initial	Stable, reproducible environments

This is a practical summary for planning. It is not a substitute for completing the official checkpoint assessment.

Step 4: Perform evidence-based gap analysis

For each weakness, examine the symptoms, contributing factors, and business consequences.

Consider the test environment problem:

Observed symptom: Integration tests frequently fail during deployment.

Evidence: CI logs show failed connections to services that are intermittently unavailable.

Possible contributing factors:

Shared infrastructure without clear ownership.
Test data overwritten by parallel execution.
Environments not provisioned consistently.
Configuration drift.
Missing environment health checks.

Do not treat these hypotheses as confirmed causes until investigation supports them.

Step 5: Prioritize the improvements

A simple workshop prioritization method can consider:

Business impact
Urgency
Implementation effort
Dependencies
Confidence in the proposed solution

For example:

Improvement	Business impact	Effort	Priority
Make CI test environments reproducible	High	Medium	First
Isolate automated test data	High	Low	First
Introduce test result trend reporting	Medium	Low	Second
Expand UI automation coverage	Medium	High	Later
Implement advanced defect prediction	Uncertain	High	Investigate

These priorities are illustrative. The official TPI NEXT clusters and checkpoints should guide the actual improvement sequence.

Step 6: Implement targeted changes

For the FleetLink example:

Establish environment ownership.
Define standardized environment configurations.
Provision consistent test dependencies.
Introduce health checks before test execution.
Isolate test data between concurrent runs.
Record environmental failures separately from application defects.
Monitor reliability over multiple release cycles.

Step 7: Reassess and measure benefits

Compare outcomes using consistent definitions and comparable periods.

Useful measurements include:

Environment-related failure percentage

Environment-related failed runs / Total eligible runs × 100

Test cycle duration

Elapsed time between test execution beginning and completion.

Test execution predictability

Difference between estimated and actual effort or elapsed time, interpreted in context.

Defect escape rate

The proportion of discovered defects first identified after release, using a consistent observation window.

If an improvement reduces environmental failures but production regressions remain unchanged, the next improvement cycle should target another limiting capability.

5. Example improvement plan

Phase	Main activity	Outcome
Assess	Review key areas and checkpoints	Current maturity matrix
Diagnose	Analyze bottlenecks and dependencies	Root cause hypotheses
Prioritize	Apply business drivers and cluster guidance	Improvement backlog
Implement	Introduce selected improvements	Updated working practices
Measure	Evaluate outcomes	Evidence of improvement
Repeat	Reassess affected key areas	Updated improvement roadmap

TPI NEXT should operate as a continuous improvement activity rather than a one-time audit.

6. Applying TPI NEXT in Agile and DevOps

Modern teams can implement TPI NEXT practices without introducing extensive manual documentation.

Examples include:

Test strategies stored as version-controlled Markdown.
Quality requirements included in backlog refinement.
Automated tests integrated into CI/CD.
Defect trends discussed during retrospectives.
Test environment configuration managed through infrastructure as code.
Improvement activities tracked as engineering backlog items.

Sogeti also provides Agile Test Practice Improvement (Agile TPI) guidance for adapting improvement assessments to Agile practices.

Agile TPI is related to TPI NEXT but uses an adapted assessment structure, so its maturity presentation should not be confused with the original TPI NEXT model.

7. TPI NEXT versus TMMi

Dimension	TPI NEXT	TMMi
Main orientation	Targeted, business-driven test process improvement	Structured organizational test maturity
Assessment structure	Individual key areas and checkpoints	Staged maturity levels and process areas
Improvement mechanism	Business drivers, clusters, enablers	Process goals and maturity progression
Typical starting point	Specific testing bottlenecks	Organizational testing capability baseline
Useful outcome	Prioritized improvement roadmap	Defined maturity improvement path
Formal TMMi certification	Not provided by TPI NEXT	Available through the TMMi assessment framework

The models are complementary. TPI NEXT can identify focused improvements while TMMi provides a broader organizational maturity direction.

8. Common mistakes

Mistake: Improving every key area simultaneously.

Prioritize based on business impact and dependencies.

Mistake: Treating maturity as a percentage score.

The official checkpoints and maturity rules must be used when representing actual TPI NEXT maturity.

Mistake: Ignoring development processes.

Testing quality depends on requirements, architecture, code review, configuration management, deployment, and operational feedback.

Mistake: Measuring only activity.

More test cases and more execution hours do not necessarily mean greater testing effectiveness.

Conclusion

TPI NEXT offers a practical way to identify testing weaknesses, understand their relationships, and improve them in a business-driven order.

Organizations achieve the greatest value when they connect testing improvements to meaningful outcomes such as lower production risk, faster feedback, and more predictable delivery.

Official resources

Verified against Sogeti/TMAP’s TPI NEXT model and its published 16-key-area guidance. TMAP+2

Tutorial 13 — ISO/IEC/IEEE 29119: Complete Software Testing Standards Guide

ISO/IEC/IEEE 29119 Explained: Software Testing Processes, Documentation, Techniques, and Practical Implementation

Software testing becomes difficult to manage when every team follows different conventions.

One team creates detailed test plans, another relies on informal discussions, and a third maintains automated tests without clear traceability to requirements.

These differences create challenges in test governance, communication, evidence collection, and release decisions.

ISO/IEC/IEEE 29119 provides an internationally recognized family of software testing standards that addresses these problems.

It defines common concepts, test processes, documentation, and testing techniques that can be applied across different organizations, technologies, and development methodologies.

1. What is ISO/IEC/IEEE 29119?

ISO/IEC/IEEE 29119 is a standards series for software and systems testing.

Its purpose is to establish a consistent foundation for planning, designing, executing, managing, and documenting software testing.

The series supports traditional development, Agile, DevOps, and other lifecycle approaches.

It does not prescribe one testing tool or require that all test information be maintained in separate Word documents.

Organizations can use appropriately managed electronic records, version-controlled documents, test management systems, and CI/CD evidence.

2. Understand the different parts

Standard	Subject	Main purpose
29119-1:2022	General concepts	Establish common testing terminology and concepts
29119-2:2021	Test processes	Define organizational, management, and dynamic testing processes
29119-3:2021	Test documentation	Specify testing documentation templates
29119-4:2021	Test techniques	Define systematic test design techniques
29119-5:2024	Keyword-driven testing	Define a structured approach to keyword-based test specification and automation
ISO/IEC TR 29119-6:2021	Agile testing guidance	Explain application of the series in Agile projects
ISO/IEC TR 29119-11:2020	Testing AI-based systems	Provide guidance for challenges specific to AI testing

Parts 1–5 form the core of the series. The technical reports provide additional application guidance.

Part 1: General concepts

Part 1 establishes terminology and fundamental ideas used throughout the series.

These include test plans, strategies, test design, execution, defect management, and communication.

Part 2: Test processes

Part 2 defines processes covering:

Organizational test governance
Test management
Dynamic test implementation

It provides a process-based approach rather than treating testing as an isolated execution activity.

Part 3: Test documentation

Part 3 describes templates for test documentation produced by the testing processes.

Examples of useful testing records include plans, specifications, execution evidence, incident information, status reports, and completion information.

Part 4: Test techniques

Part 4 addresses systematic techniques for deriving test cases.

Instead of choosing test inputs randomly, engineers use an appropriate test model and defined techniques to produce meaningful coverage.

Part 5: Keyword-driven testing

Part 5 describes an approach in which test specifications use reusable keywords representing test operations.

This supports modular testing and automation frameworks.

3. Implement ISO 29119 step by step

Consider an API that accepts a vehicle battery charge percentage and publishes it to a companion application.

The input must be numeric and lie between 0 and 100, inclusive.

Step 1: Define the testing objectives

Before execution, identify:

Functional requirements.
Security and privacy expectations.
Reliability and performance requirements.
Applicable regulatory or contractual constraints.
Test boundaries and exclusions.
Required evidence and acceptance criteria.

For the battery-status API:

Requirement BAT-001: The service must accept valid numeric charge percentages between 0 and 100.

Requirement BAT-002: The service must reject invalid charge values.

Requirement BAT-003: Failed validation must not update the stored battery status.

These requirements create testable expectations.

Step 2: Establish a test strategy

Determine which types of testing are appropriate.

Test type	Objective
Unit testing	Validate input processing logic
Integration testing	Verify API and database interactions
API testing	Validate requests, responses, and error handling
Security testing	Verify authorization and input handling
Performance testing	Verify behavior under defined load
Regression testing	Confirm that existing behavior remains correct

Prioritize the risks that could cause serious business or technical consequences.

Step 3: Create a test plan

A lightweight test plan can be stored in the repository.

Example:

test_plan:
  id: TP-BATTERY-001
  component: battery-status-api

  objectives:
    - Validate battery percentage inputs
    - Verify API response behavior
    - Prevent invalid data persistence

  test_levels:
    - unit
    - integration
    - system

  test_environment:
    type: isolated-integration
    test_data: synthetic

  entry_criteria:
    - Requirements reviewed
    - Test environment available
    - Candidate build identified

  exit_criteria:
    - Required critical tests completed
    - No unresolved blocker defects
    - Residual risks reviewed

  evidence:
    - test-results.xml
    - defect-report
    - test-completion-summaryCode language: CSS (css)

This is a simplified practical template, not a complete reproduction of ISO/IEC/IEEE 29119-3 requirements.

For formal conformance, the applicable standard must be reviewed and its requirements satisfied or formally tailored where permitted.

Step 4: Apply test design techniques

Consider equivalence partitioning and boundary value analysis.

For the battery percentage requirement, useful partitions are:

Input class	Example	Expected result
Below lower boundary	-1	Reject
Lower boundary	0	Accept
Valid interior	50	Accept
Upper boundary	100	Accept
Above upper boundary	101	Reject
Missing value	None	Reject
Unsupported type	True	Reject

Boundary analysis is useful because errors commonly occur near the limits of accepted ranges.

Additional techniques can be selected for more complex logic.

For example, a notification rule involving severity, account permissions, and user preferences may benefit from decision-table testing.

4. Practical automation with Python and pytest

Create a project directory and install pytest:

mkdir iso29119-demo
cd iso29119-demo

python3 -m venv .venv
source .venv/bin/activate

python -m pip install pytest

On Windows PowerShell, activate the virtual environment using:

.venv\Scripts\Activate.ps1Code language: CSS (css)

Create battery.py:

def is_valid_percentage(value):
    return (
        type(value) in (int, float)
        and 0 <= value <= 100
    )Code language: JavaScript (javascript)

Create test_battery.py:

import pytest
from battery import is_valid_percentage


@pytest.mark.parametrize(
    "value,expected",
    [
        (-1, False),
        (0, True),
        (1, True),
        (50, True),
        (99, True),
        (100, True),
        (101, False),
        (None, False),
        (True, False),
    ],
)
def test_percentage_validation(value, expected):
    assert is_valid_percentage(value) is expectedCode language: PHP (php)

Run the tests:

python -m pytest -v

Generate machine-readable test evidence:

python -m pytest \
  --junitxml=test-results.xml

This example demonstrates how formal test design concepts translate into executable tests.

It does not establish compliance with the entire ISO 29119 series.

5. Establish requirements traceability

Every important requirement should have a clear relationship to its verification evidence.

Requirement	Test case	Expected behavior	Evidence
BAT-001	TC-001	Accept valid charge percentage	Unit test report
BAT-002	TC-002	Reject invalid charge percentage	Unit and API reports
BAT-003	TC-003	Invalid input does not update persistence	Integration test report

Traceability becomes especially important when teams must demonstrate testing completeness to customers or assessors.

The objective is not to create more paperwork. It is to make verification auditable and reproducible.

6. Test execution and defect management

When a test fails, distinguish between:

A genuine software defect.
An incorrect test expectation.
Invalid test data.
An environment problem.
An automation problem.
An unresolved issue requiring investigation.

A useful defect report contains:

Defect ID: BUG-104

Affected build: Identified release candidate

Requirement: BAT-002

Expected: Invalid percentage is rejected.

Actual: API accepts an out-of-range value.

Reproduction: Submit a value of 101.

Impact: Invalid data may be displayed or persisted.

Evidence: Request, response, logs, and linked test result.

The report should also identify severity, ownership, and resolution status.

7. Produce meaningful test reports

A useful test status report should answer:

What was tested?
What passed and failed?
Which planned tests remain incomplete?
Which significant defects remain unresolved?
What risks remain?
Is the available evidence sufficient for a release decision?

Avoid reporting only an overall pass percentage.

A 99% pass rate can still conceal failure of a business-critical test.

8. Apply the standard in Agile and DevOps

A lightweight implementation can map the testing process into development activities.

Development activity	Testing activity
Backlog refinement	Identify risks and testable acceptance criteria
Pull request	Execute unit tests and static checks
Integration build	Execute API and integration tests
Release candidate	Perform system and regression testing
Release review	Examine residual risks and test completion evidence
Production operation	Capture operational feedback for future testing

The test plan can be a repository artifact, test cases can be executable, and test reports can be generated automatically.

The important issue is whether the necessary process objectives and applicable requirements are satisfied—not whether the organization uses a particular document format.

9. Common mistakes

Treating ISO 29119 as a test case template standard only.

The series includes test management, processes, and techniques.

Writing tests without identifying expected behavior.

Every meaningful test needs a credible way to determine success or failure.

Using coverage percentages without analyzing risk.

Coverage helps describe what was exercised but does not by itself demonstrate adequate product quality.

Declaring conformance from tool usage alone.

Using pytest, Playwright, or a test management application does not establish ISO 29119 conformance.

Ignoring formal tailoring.

Part 1 explains that the normative parts have conformance requirements and that tailored conformance may be appropriate when the tailoring and rationale are defined and agreed as required.

Conclusion

ISO/IEC/IEEE 29119 provides a consistent foundation for software testing across different engineering environments.

Its strongest practical value comes from combining clear testing objectives, systematic test design, controlled execution, evidence management, and risk-based reporting.

Official resources

Verified against ISO’s standards catalogue, the official software testing series guidance, and the pytest documentation. Software and systems engineering+4

Tutorial 14 — ISO/IEC 25010: Software Product Quality Models

ISO/IEC 25010 Tutorial: How to Define, Measure, and Improve Software Product Quality

A software application can pass all its functional tests and still provide a poor experience.

It might be slow, difficult to maintain, inaccessible to users, vulnerable to attacks, or unreliable under demanding conditions.

These are different dimensions of software product quality, and each requires a suitable evaluation approach.

ISO/IEC 25010 provides a structured quality model that helps organizations identify and evaluate those dimensions.

1. What is ISO/IEC 25010?

ISO/IEC 25010 belongs to the SQuaRE family—Systems and software Quality Requirements and Evaluation.

The standard defines a model for the quality characteristics of ICT and software products.

Organizations can use the model when:

Establishing product requirements.
Designing systems and software.
Defining non-functional testing objectives.
Developing release acceptance criteria.
Evaluating software suppliers.
Reviewing technical quality.
Prioritizing quality engineering investment.

The standard provides a classification model, not a universal list of performance thresholds or an automatic software certification mechanism.

2. The nine software product quality characteristics

ISO/IEC 25010:2023 defines nine primary product quality characteristics.

Characteristic	What it evaluates	Example verification
Functional suitability	Whether the product provides appropriate and correct functions	Requirements-based testing
Performance efficiency	Response behavior, resource consumption, and capacity	Load and performance testing
Compatibility	Ability to coexist and interoperate with other systems	Integration and compatibility testing
Interaction capability	How effectively people or other entities can interact with the product	Usability and accessibility-related evaluations
Reliability	Ability to perform consistently and recover from failures	Fault injection and recovery testing
Security	Protection against relevant threats and unauthorized actions	Security testing and threat analysis
Maintainability	Ease of analysis, modification, testing, and reuse	Code review and maintainability analysis
Flexibility	Ability to adapt, scale, install, and operate in changing environments	Scaling and deployment testing
Safety	Ability to avoid unacceptable harm in relevant operating conditions	Hazard-based and fail-safe validation

Each characteristic contains more specific subcharacteristics.

This helps engineers turn the broad idea of quality into concrete engineering objectives.

3. Important differences from the older quality model

Many online tutorials still describe eight ISO 25010 characteristics.

That reflects ISO/IEC 25010:2011, which has been replaced.

The revised product quality model introduced important changes:

Safety became a separate quality characteristic.
Usability was replaced by Interaction capability in the product quality model.
Portability was replaced by Flexibility.
Several subcharacteristics were revised or added.
Quality-in-use is addressed separately in ISO/IEC 25019:2023.

The distinction between product quality and quality-in-use is useful.

Product quality concerns the characteristics of the product itself. Quality-in-use concerns the consequences and outcomes of using the product in a specified context.

A technically responsive interface does not automatically guarantee that users complete their tasks successfully.

4. How to implement ISO 25010

Assume FleetLink provides a mobile application and cloud services for viewing connected-vehicle information.

The product team wants to convert broad quality expectations into measurable requirements.

Step 1: Identify stakeholders and their expectations

Consider the needs of:

End users.
Product management.
Engineering teams.
Security and privacy stakeholders.
Operations teams.
Support teams.
Customers or contractual partners.

Different stakeholders may prioritize quality differently.

For example, operations teams may emphasize reliability and observability, while end users may emphasize interaction capability and responsiveness.

Step 2: Identify critical quality characteristics

Do not assume every product requires an identical evaluation depth for every characteristic.

Evaluate product purpose, architecture, operating conditions, business consequences, and potential hazards.

For FleetLink, reliability, security, performance efficiency, and interaction capability may require significant attention.

If the product includes functions with potential safety consequences, safety engineering must be addressed through suitable hazard analysis and applicable domain-specific requirements.

Step 3: Write measurable quality requirements

A weak requirement says:

“The application must be fast.”

A stronger requirement says:

“Under the agreed reference workload and environment, the API must respond within the specified latency threshold for at least 95% of successful requests.”

The requirement becomes testable when the workload, measurement point, environment, threshold, exclusions, and acceptance method are defined.

Step 4: Build a product quality matrix

The following targets are illustrative engineering examples, not ISO-mandated values.

Characteristic	Example objective	Example evidence
Functional suitability	All identified critical functions satisfy approved acceptance criteria	Functional test results
Performance efficiency	API p95 latency ≤ 300 ms under a defined workload	Load test report
Compatibility	Supported integration versions pass defined contract tests	Contract test report
Interaction capability	Critical user journeys meet agreed usability and accessibility criteria	UX evaluation and accessibility checks
Reliability	Recovery completes within the agreed service recovery objective	Recovery test results
Security	No unresolved critical findings in the agreed assessment scope	Security review and test reports
Maintainability	Critical changes meet review and maintainability criteria	Code review and static analysis
Flexibility	System satisfies approved horizontal-scaling scenarios	Scaling test report
Safety	Safety-related behavior meets approved hazard-derived requirements	Safety verification evidence

Quality objectives should be reviewed whenever the product’s architecture, operating conditions, or risks change.

5. Measuring performance efficiency

Performance efficiency involves more than response time.

Relevant measures may include:

Response latency.
Throughput.
CPU consumption.
Memory utilization.
Storage I/O.
Concurrent workload capacity.

Suppose the team defines:

A target workload of 500 requests per second.
An API p95 response time of at most 300 milliseconds.
A defined error-rate threshold.
A reproducible test environment.

Engineers can use tools such as k6 or Apache JMeter to execute the workload and collect evidence.

The workload model must reflect expected usage rather than an arbitrary collection of HTTP requests.

6. Measuring reliability

Reliability evaluation may include availability, faultlessness, fault tolerance, and recoverability.

Consider a service dependency becoming unavailable.

Useful test scenarios include:

Introduce a defined dependency failure.
Observe application behavior.
Verify that the system handles the failure as specified.
Restore the dependency.
Measure recovery time.
Confirm that relevant data remains consistent.

Realistic failure scenarios reveal weaknesses that functional happy-path tests cannot identify.

7. Measuring security

Security testing should address relevant threats, architecture, and business risks.

Practical activities include:

Authentication and authorization testing.
Secure configuration review.
Source code security analysis.
Dependency vulnerability review.
API security testing.
Security-related logging verification.
Threat modeling.
Independent testing where appropriate.

A security test result must be interpreted against the tested scope, configuration, and supported threat model.

Passing an automated scanner does not prove that software is secure.

8. Measuring maintainability

Maintainability concerns how effectively software can be understood, changed, tested, and maintained.

Useful engineering indicators include:

Change lead time: Time required to implement, review, verify, and deliver a defined change.

Complexity: Complexity measures applied to relevant components with contextual interpretation.

Code duplication: Repeated implementation patterns that create maintenance risk.

Testability: Ability to validate components and behavior without unreasonable effort.

Architecture constraints: Evidence that component dependencies respect intended boundaries.

These metrics should guide engineering decisions rather than become isolated targets that teams optimize without improving the product.

9. Measuring interaction capability

Interaction capability is particularly relevant for mobile applications, dashboards, websites, and interactive systems.

Evaluate:

Clarity of system behavior.
Ease of learning.
Prevention of user errors.
User guidance and assistance.
Support for users with diverse capabilities.
Effectiveness of interaction design.

For example, a vehicle-status application should clearly distinguish between fresh vehicle data and data that has not synchronized recently.

This prevents users from misinterpreting outdated information as real-time status.

Tools such as axe-core and Lighthouse can support selected checks, but human evaluation remains important.

10. Evaluating safety

Safety must be considered when software behavior can contribute to harm.

For example, a connected system controlling or influencing physical operations may require evidence that:

Relevant hazards have been identified.
Fault conditions are handled appropriately.
Safety constraints are enforced.
Failure behavior is appropriately controlled.
Interfaces do not introduce unacceptable hazards.

ISO/IEC 25010 supplies a quality model. It does not replace domain-specific functional safety standards such as ISO 26262 for applicable automotive safety-related systems.

Safety requirements and verification methods must be derived from the actual application and relevant safety engineering activities.

11. Connect quality objectives to DevSecOps

Quality requirements should influence development from the beginning.

Engineering stage	Quality activities
Requirements	Establish measurable quality objectives
Architecture	Evaluate tradeoffs and quality risks
Development	Apply secure coding, reviews, and component testing
Continuous integration	Execute automated tests and relevant static checks
Pre-release	Perform targeted reliability, security, and performance validation
Production	Monitor operational quality and customer impact
Improvement	Adjust requirements and verification practices using evidence

This approach moves quality from a final testing event into the software development lifecycle.

12. Common mistakes

Mistake: Using the outdated eight-characteristic model without checking the edition.

Use the applicable edition and document which version governs the project.

Mistake: Treating every quality metric as a universal threshold.

ISO 25010 does not establish one pass/fail value appropriate for all products.

Mistake: Testing only functional suitability.

Functional correctness does not guarantee reliability, security, maintainability, or other quality characteristics.

Mistake: Using static analysis as the complete quality measurement strategy.

Structural analysis should complement behavioral testing, operational evidence, and stakeholder evaluation.

Conclusion

ISO/IEC 25010 provides a powerful foundation for defining what software product quality actually means.

Its greatest value comes from translating quality characteristics into measurable requirements, selecting suitable verification methods, and continuously evaluating the resulting evidence.

Official resources

Verified against ISO’s published product quality model and quality-in-use standard. Product quality model+3

Tutorial 15 — IEEE 730: Software Quality Assurance Processes

IEEE 730 Tutorial: How to Build an Effective Software Quality Assurance Process and SQA Plan

Software testing is essential, but testing alone cannot guarantee software quality.

A test team may identify defects while the wider engineering organization struggles with inconsistent requirements, inadequate reviews, uncontrolled changes, supplier risks, or incomplete release evidence.

Software Quality Assurance (SQA) addresses these broader concerns.

IEEE 730 provides a framework for establishing and managing software quality assurance processes throughout the software lifecycle.

1. What is IEEE 730?

IEEE 730 is a standard for software quality assurance processes.

The current edition is IEEE 730-2026, titled IEEE Standard for Software Quality Assurance Processes. It supersedes IEEE 730-2014.

The standard addresses initiating, planning, controlling, and executing software quality assurance activities. Its scope extends beyond software testing to quality governance and assurance across software development and maintenance.

The updated standard is harmonized with relevant software lifecycle and engineering standards, including ISO/IEC/IEEE 12207 and IEEE 2675.

When referring to the current publication, use the designation IEEE 730-2026, rather than assuming that the designation ISO/IEC/IEEE 730 identifies the current IEEE publication.

2. Understand QA, QC, and testing

These terms are related but not interchangeable.

Discipline	Main concern	Example
Quality Assurance	Establishing confidence that appropriate quality processes are implemented and followed	Process assessment and assurance reviews
Quality Control	Evaluating whether products meet specified criteria	Product inspection and acceptance evaluation
Software Testing	Evaluating software behavior and discovering defects or risks	API, integration, and regression testing

Software testing can contribute to quality control, while SQA evaluates the wider process and evidence supporting the delivery of quality software.

For example, SQA may verify that changes receive appropriate review, critical requirements have verification evidence, and release exceptions are formally considered.

3. What is a Software Quality Assurance Plan?

A Software Quality Assurance Plan (SQAP) describes how quality assurance responsibilities and activities will be organized and performed for a project or organizational context.

A practical SQAP should make the assurance approach understandable, repeatable, and auditable.

The plan should answer:

What quality objectives apply?
Which lifecycle processes are in scope?
Who is accountable for assurance activities?
Which standards and procedures apply?
What evidence must be maintained?
How are nonconformities handled?
What happens when quality expectations are not met?
How will the assurance process be evaluated and improved?

The exact plan structure and required contents for a claim of conformance must be checked against the applicable IEEE 730 edition.

4. Step-by-step implementation

Step 1: Establish software quality objectives

Start with business, contractual, regulatory, and product requirements.

For the fictional FleetLink platform, example objectives might include:

Reliable delivery of vehicle-status information.
Protection of customer and vehicle-related data.
Predictable release quality.
Effective management of software changes.
Traceable verification evidence.
Timely management of significant quality issues.

Quality objectives should be actionable and connected to measurable evidence.

Step 2: Define the scope of assurance

Identify relevant lifecycle activities.

These may include:

Requirements development and review.
Software architecture and design.
Implementation.
Testing and verification.
Configuration and change management.
Software acquisition and supplier management.
Deployment and release.
Maintenance and incident management.

Quality assurance should not begin only after development is complete.

Step 3: Define roles and responsibilities

A simplified responsibility model could be:

Activity	Accountable role	Supporting roles
Quality objectives	Engineering or product leadership	SQA, security, stakeholders
Code review process	Engineering lead	Developers
Test strategy	Quality engineering lead	Developers, product owners
Security assurance	Security owner	Development and operations
Release decision	Authorized release owner	SQA, engineering, operations
Quality process assessment	SQA owner	Independent reviewers as appropriate
Corrective action tracking	Assigned process owner	Relevant engineering teams

The organization should ensure that assurance findings can be reported and escalated without inappropriate conflicts of interest.

A small organization may use designated reviewers and structured escalation rather than a large dedicated QA department.

Step 4: Prepare the SQA plan

The following is an illustrative plan outline.

Software Quality Assurance Plan

1. Purpose and scope
2. Product and system context
3. Quality objectives
4. Applicable standards and procedures
5. Roles and responsibilities
6. Lifecycle assurance activities
7. Requirements and design reviews
8. Verification and validation evidence
9. Configuration and change management
10. Security, privacy, and reliability assurance
11. Supplier and third-party quality controls
12. Nonconformity and corrective action process
13. Quality measurement and reporting
14. Release quality review
15. Records and evidence retention
16. Assurance process evaluation and improvement

This example is an implementation starting point, not an official IEEE 730 template or exhaustive compliance checklist.

Step 5: Establish quality checkpoints

Integrate assurance activities into the development lifecycle.

During requirements development

Verify that critical quality expectations are identified, reviewed, and testable.

During architecture and design

Review important security, reliability, performance, and maintainability risks.

During implementation

Verify the application of relevant development practices, source code review, and configuration management.

During testing

Examine the adequacy of test coverage, environments, outcomes, and unresolved risks.

Before release

Review acceptance evidence, significant defects, operational readiness, and approved exceptions.

After deployment

Evaluate production incidents and feedback for corrective and preventive actions.

Step 6: Establish a nonconformity management process

A nonconformity is a failure to satisfy an applicable requirement, process expectation, or agreed criterion.

For example:

A production deployment is completed without a required security review.

The corrective process should identify:

The applicable requirement.
What was not satisfied.
The impact and current risk.
Necessary containment.
The root cause.
Corrective action.
Verification of effectiveness.
Final closure evidence.

A process nonconformity is not always the same thing as a software defect.

Both may require tracking, but they concern different aspects of quality.

5. Implement quality assurance in CI/CD

Software assurance can be supported through automated evidence.

A simplified workflow is:

Source Change
    |
    v
Code Review
    |
    v
Static Analysis and Security Checks
    |
    v
Unit and Integration Testing
    |
    v
Build and Artifact Verification
    |
    v
Release Readiness Assessment
    |
    v
Authorized Deployment
    |
    v
Production Monitoring
    |
    v
Corrective and Preventive ImprovementCode language: PHP (php)

CI/CD automation improves consistency and evidence collection.

However, automated tools cannot independently determine whether every applicable product, contractual, or assurance requirement has been satisfied.

6. Example quality assurance checklist

Assurance question	Evidence	Responsible party
Are critical requirements reviewed?	Review records	Product and engineering
Are relevant architecture risks assessed?	Design review and threat model	Architecture and security
Are code changes controlled?	Pull requests and approvals	Engineering
Are required tests executed?	Test results	Quality engineering
Are significant defects managed?	Defect records	Engineering and QA
Are release exceptions approved?	Exception and risk acceptance records	Authorized decision maker
Is configuration traceable?	Build and release metadata	Release engineering
Are production incidents reviewed?	Incident and corrective action records	Operations and engineering

This is a practical assurance checklist rather than a standards conformance assessment.

7. Recommended metrics

Quality assurance metrics may include:

Completion of applicable reviews.
Open and overdue nonconformities.
Corrective action closure time.
Repeated process nonconformities.
Verification evidence completeness.
Release exceptions and their risk levels.
Production incidents related to process weaknesses.
Supplier quality issues.

Avoid using process compliance scores as a substitute for product outcomes.

A well-documented process can still produce software that needs improvement.

8. A practical 30-day adoption plan

Week 1: Assess the current state

Identify assurance responsibilities, applicable requirements, existing procedures, and major gaps.

Week 2: Establish the assurance plan

Define scope, roles, review activities, evidence requirements, and nonconformity handling.

Week 3: Integrate assurance into engineering

Introduce review checkpoints, evidence collection, quality reporting, and release risk decisions.

Week 4: Evaluate effectiveness

Review a sample of actual changes and releases. Identify whether the new process detects important weaknesses and produces useful evidence.

The goal is to establish a functioning baseline, not to claim full standards conformance after one month.

9. Common mistakes

Mistake: Treating SQA as another name for software testing.

Testing is one of several activities contributing to quality assurance.

Mistake: Assigning quality responsibility exclusively to a QA team.

Quality outcomes require active participation across engineering and the business.

Mistake: Maintaining an SQA plan without operational evidence.

A procedure is valuable only when it influences actual practices.

Mistake: Creating excessive manual approval gates.

Controls should be proportionate to risk and integrated into existing delivery workflows where possible.

Mistake: Assuming IEEE 730 certification from an internal checklist.

Compliance claims require examination of the actual standard and relevant assessment or contractual mechanisms.

Conclusion

IEEE 730 provides a foundation for establishing systematic software quality assurance across the engineering lifecycle.

An effective SQA program combines clear responsibilities, documented objectives, risk-based reviews, reliable evidence, nonconformity management, and continuous improvement.

The result should be greater confidence in software delivery—not additional bureaucracy without measurable value.

Official resource

IEEE Standards Association — IEEE 730-2026

Verified against the current IEEE 730-2026 publication and its supersession record. IEEE 730-2026+2

Tutorial 16 — Six Sigma: Defect Reduction and Process Improvement

Six Sigma for Software Engineering: A Practical Guide to DMAIC, Defect Reduction, and Continuous Improvement

Software engineering teams frequently encounter recurring quality problems.

An application may repeatedly fail deployment checks. Integration tests may be unreliable. Production incidents may follow similar patterns, and teams may spend significant time fixing issues that could have been prevented.

Solving individual incidents is necessary, but repeatedly fixing the same categories of problems is expensive.

Six Sigma provides a disciplined approach to understanding process variation, identifying root causes, and implementing measurable improvements.

Although originally developed in industrial quality management, its principles can be applied to software engineering, testing, DevOps, and operational processes.

1. What is Six Sigma?

Six Sigma is a data-driven quality improvement methodology focused on reducing defects, errors, and process variation.

It emphasizes:

Clearly defined problems.
Reliable measurement.
Root cause analysis.
Evidence-based changes.
Verification of improvement.
Sustaining results over time.

A commonly cited Six Sigma performance convention is 3.4 defects per million opportunities (DPMO) at the six-sigma level under the conventional long-term sigma-shift model.

This is not a universal software quality target or a guarantee that a product will be safe or defect-free.

Six Sigma is most useful when an organization has a recurring, measurable process problem that warrants systematic investigation.

2. Lean versus Six Sigma

Lean and Six Sigma are closely related but emphasize different improvement concerns.

Approach	Main emphasis	Software engineering example
Lean	Reduce waste and improve flow	Eliminate unnecessary waiting during releases
Six Sigma	Reduce variation and recurring defects	Reduce inconsistent deployment outcomes
Lean Six Sigma	Improve flow while reducing defects and variation	Improve deployment speed and reliability together

These approaches can be combined with Agile, DevOps, TMMi, and other engineering process improvement practices.

3. Understand DMAIC

DMAIC is the primary structured improvement cycle for existing processes.

The five phases are:

Define
Measure
Analyze
Improve
Control

Each phase has a distinct purpose and expected outcome.

4. Step 1 — Define the problem

Begin with a measurable problem statement.

Weak statement:

“Our deployments are unreliable.”

Stronger statement:

“A significant proportion of production deployments require remediation because the deployment process experiences recurring configuration and integration failures.”

Define:

The process being examined.
The measurable problem.
The affected stakeholders.
The business consequences.
The improvement objective.
The investigation boundaries.

Example project charter

Field	Example
Project	Deployment Reliability Improvement
Problem	Recurring deployment validation failures
Process	Build, test, and deployment workflow
Business impact	Delayed releases and engineering rework
Objective	Reduce preventable deployment defects
Sponsor	Engineering leadership
Process owner	Release engineering
Success measure	Fewer comparable deployment failures

The objective should be specific enough to test after implementing changes.

5. Step 2 — Measure the baseline

Six Sigma requires trustworthy data.

For a software deployment improvement project, useful measures include:

Deployment attempts.
Failed deployments.
Deployment-related defects.
Recovery time.
Rework effort.
Causes of failed validation checks.
Change failure rate.
Deployment duration.

Measurement definitions must remain consistent across the baseline and improvement periods.

Understanding DPMO

Defects per million opportunities is calculated as:

[
DPMO=\frac{\text{Number of defects}}{\text{Units}\times\text{Opportunities per unit}}\times1{,}000{,}000
]

An opportunity is a clearly defined chance for a particular defect to occur.

In software engineering, this definition requires care because opportunities are not always obvious or independent.

Worked example

Suppose an engineering team examines 1,000 build executions.

Each build contains four clearly defined validation opportunities.

During the baseline period, the team records 72 qualifying validation defects.

Units: 1,000 builds
Opportunities per unit: 4
Total opportunities: 4,000
Defects: 72

[
DPMO=\frac{72}{4000}\times1{,}000{,}000=18{,}000
]

Now suppose a later, comparable period contains 20 qualifying defects across the same number of opportunities.

[
DPMO=\frac{20}{4000}\times1{,}000{,}000=5{,}000
]

The illustrative DPMO reduction is approximately 72.2%.

This suggests improvement, but a sound analysis should also consider workload comparability, changing system complexity, measurement accuracy, and whether the change is statistically distinguishable from ordinary variation.

Calculate DPMO with Python

Create quality_metrics.py:

def calculate_dpmo(defects, units, opportunities_per_unit):
    if units <= 0 or opportunities_per_unit <= 0:
        raise ValueError("Units and opportunities must be positive")

    if defects < 0:
        raise ValueError("Defect count cannot be negative")

    total_opportunities = units * opportunities_per_unit

    return (defects / total_opportunities) * 1_000_000


baseline = calculate_dpmo(72, 1000, 4)
improved = calculate_dpmo(20, 1000, 4)

reduction = ((baseline - improved) / baseline) * 100

print(f"Baseline DPMO: {baseline:,.0f}")
print(f"Improved DPMO: {improved:,.0f}")
print(f"Reduction: {reduction:.1f}%")Code language: PHP (php)

Execute:

python3 quality_metrics.pyCode language: CSS (css)

Expected output:

Baseline DPMO: 18,000
Improved DPMO: 5,000
Reduction: 72.2%Code language: CSS (css)

The calculation is only meaningful if the opportunities and defect counting rules are consistently defined.

6. Step 3 — Analyze the root causes

A common mistake is implementing a solution before understanding the problem.

For example, when tests fail frequently, teams may immediately increase retries.

Retries can conceal defects, waste execution resources, and make diagnosis harder.

Instead, classify failures and investigate their causes.

Example failure classification

Cause category	Investigation question
Test data	Is test data predictable and isolated?
Environment	Are services available and configured consistently?
Application defects	Is application behavior incorrect?
Automation defects	Are test scripts or assertions unreliable?
Dependencies	Are external services causing failures?
Deployment process	Are configuration or execution steps inconsistent?

Useful analysis techniques include:

Pareto analysis: Identify which categories contribute most heavily to the problem.

Five Whys: Repeatedly investigate why a failure occurs until a supported underlying cause is identified.

Fishbone analysis: Group possible causes into categories such as people, process, technology, infrastructure, and information.

Failure Mode and Effects Analysis (FMEA): Identify potential failure modes and prioritize investigation based on their consequences and other relevant risk factors.

Root cause analysis should be supported by evidence, not merely by the opinions of the most experienced participants.

7. Step 4 — Improve the process

After identifying credible root causes, select interventions.

Suppose the investigation finds that shared mutable test data causes a significant proportion of intermittent integration test failures.

Possible improvements include:

Generate isolated test data for each run.
Reset fixtures before execution.
Remove dependencies between parallel tests.
Provide deterministic service responses where appropriate.
Manage environment configuration consistently.
Eliminate uncontrolled shared accounts.

Implement changes incrementally where possible.

Use a controlled comparison or pilot to evaluate whether the proposed changes actually improve outcomes.

Improvement validation

Indicator	Before	After	Interpretation
Validation defects	72	20	Lower observed count
DPMO	18,000	5,000	Lower observed defect density
Test environment changes	Uncontrolled	Version-controlled	Better reproducibility
Test data ownership	Shared	Isolated	Reduced interference risk

The values in this example are hypothetical. They illustrate the evaluation method rather than a real organizational result.

8. Step 5 — Control the improved process

Improvements can disappear if they are not incorporated into routine operations.

A control plan should establish:

Process ownership.
Standard operating practices.
Measurement definitions.
Monitoring responsibilities.
Appropriate control limits or thresholds.
Actions when performance deteriorates.
Periodic process review.

For deployment quality, this may include automated preflight checks, configuration validation, change reviews, and a defined response to abnormal failure patterns.

Statistical process control can be useful when there is sufficient stable data and the selected statistical method fits the process.

Avoid drawing conclusions from control charts that ignore changing workloads, small sample sizes, or inappropriate assumptions about the data.

9. Useful Six Sigma metrics for software engineering

Metric	Purpose	Important caution
DPMO	Measure defects relative to defined opportunities	Requires consistent opportunity definitions
Defect density	Compare defects against a defined size measure	Size measures must be comparable
Change failure rate	Evaluate production change reliability	Define what qualifies as a failure
First-pass yield	Measure work completed without rework	Define the unit and acceptance criteria
Rework percentage	Evaluate wasted effort	Requires reliable effort classification
Defect escape rate	Identify defects found after release	Use consistent observation windows
Mean recovery time	Evaluate recovery behavior	Distinguish different incident classes

No single metric adequately represents all aspects of software quality.

10. Integrating Six Sigma with DevOps

DMAIC can fit naturally into engineering retrospectives and improvement initiatives.

Define: Identify recurring CI/CD failures.

Measure: Establish failure frequency and rework costs.

Analyze: Investigate build logs and failure categories.

Improve: Implement targeted engineering changes.

Control: Establish monitoring and regression prevention measures.

A structured DMAIC project is particularly helpful when repeated quick fixes have failed to solve a significant problem.

11. Integrating Six Sigma with TMMi

TMMi provides a maturity improvement model, while Six Sigma offers methods for investigating measurable problems.

At higher TMMi maturity levels, quantitative evaluation, defect prevention, and process optimization become increasingly important.

Six Sigma techniques can support those objectives, provided the methods are appropriate for the data and problem being examined.

12. Common mistakes

Mistake: Applying DPMO to arbitrary software metrics.

Defects and opportunities must be defined consistently.

Mistake: Claiming Six Sigma performance from a small sample.

Process capability and statistical conclusions require appropriate evidence and assumptions.

Mistake: Using metrics to blame individuals.

The objective is to improve the process, not punish the people reporting defects.

Mistake: Skipping measurement validation.

Incorrect or inconsistent measurement can make an improvement appear successful when it is not.

Mistake: Confusing correlation with causation.

Improvement experiments should test whether the suspected intervention actually influences the result.

Conclusion

Six Sigma provides software organizations with a systematic method for reducing recurring defects and improving process performance.

Its value lies in disciplined problem definition, reliable measurement, evidence-based root cause analysis, controlled improvement, and long-term monitoring.

Used appropriately, Six Sigma complements Agile, DevOps, test maturity models, and software quality engineering.

Official resources

Methodology and conventional DPMO interpretation verified against the American Society for Quality. ASQ+2

Tutorial 17 — CISQ: Automated Software Quality Measurement

CISQ Explained: How to Measure Software Structural Quality, Security, Reliability, Performance, and Maintainability

A software application can behave correctly during testing while still containing serious structural weaknesses.

For example, a system may pass its functional test suite but contain insecure data access paths, unmanaged resources, or architectural weaknesses that make future changes dangerous.

These problems are difficult to evaluate using functional testing alone.

The Consortium for Information & Software Quality (CISQ) develops standardized approaches to automated software measurement, particularly software size and internal structural quality.

Its work helps organizations evaluate technical risk, manage software quality, and establish measurable expectations for software delivered internally or by third-party suppliers.

1. What is CISQ?

CISQ is an industry consortium established to support standardized automated measurement of software quality and size.

Its work addresses the problem of evaluating large software systems consistently across technologies and organizations.

CISQ is not a staged test process maturity framework like TMMi.

Instead, its measures support objective analysis of important internal software characteristics.

CISQ’s structural quality work is represented in ISO/IEC 5055:2021, which standardizes automated source code quality measures.

2. Understanding ISO/IEC 5055

ISO/IEC 5055 defines automated source code quality measures across four characteristics.

Characteristic	Principal concern	Example weakness
Security	Structural weaknesses that could be exploited	Unprotected access to sensitive data
Reliability	Weaknesses that could affect dependable operation	Improper resource management
Performance efficiency	Weaknesses affecting resource efficiency or performance	Inefficient architectural data access
Maintainability	Weaknesses that increase modification cost and difficulty	Problematic dependencies and structural complexity

The standard focuses on significant weaknesses in the construction of software.

Its purpose is different from measuring whether a system satisfies a particular end-user functional requirement.

3. How CISQ relates to ISO 25010

ISO/IEC 25010 defines a broad product quality model.

ISO/IEC 5055 provides standardized source code measures addressing four important product quality characteristics.

The two standards are complementary.

For example:

ISO/IEC 25010: Security is an important characteristic of software product quality.

ISO/IEC 5055: Relevant structural security weaknesses can be identified and counted using specified automated source code measurement rules.

Likewise, ISO 25010 describes reliability as a product quality characteristic, while ISO 5055 contributes source-level evidence about weaknesses that could undermine reliability.

Software teams should combine structural analysis with appropriate functional, security, performance, and operational testing.

4. Understanding structural quality

Structural quality refers to properties of how software is designed and implemented.

These include issues at both component and architectural levels.

Consider a web service that receives requests from users and retrieves protected account data.

A sound architecture typically requires relevant authentication and authorization checks before sensitive resources can be accessed.

A structural weakness may arise when an unintended route allows a request to reach protected data without passing through the necessary controls.

A functional test suite could miss this problem if it only tests the expected application workflow.

Architecture analysis and security testing should examine such unintended paths.

5. Implementing CISQ-oriented measurement

Step 1: Define the measurement scope

Identify the software systems and components under evaluation.

Record:

Application and component boundaries.
Programming languages.
Frameworks.
Relevant architectural dependencies.
Third-party components.
Source code availability.
Applicable technology and tool limitations.

Do not assume that a scanner supports every language, component, or architectural relationship in the application.

Step 2: Establish structural quality objectives

Create quality objectives aligned to business risk.

Examples include:

Identify and eliminate critical security weaknesses.
Reduce recurring reliability-related source code problems.
Identify structural weaknesses that create performance risk.
Prevent significant maintainability problems from increasing.
Establish consistent software quality expectations for suppliers.

Measurement objectives should distinguish serious structural issues from minor style preferences.

Step 3: Choose appropriate analysis technology

Different tools provide different kinds of evidence.

Tool or approach	Practical contribution
Semgrep Community Edition	Source code pattern analysis using configurable rules
SonarQube Community Build	Source code quality and selected security analysis
CodeQL	Query-based code security analysis for supported languages
Dependency analysis tools	Identify known vulnerabilities in third-party dependencies
Architecture review	Evaluate dependencies, trust boundaries, and design weaknesses
ISO 5055-capable analysis technology	Produce applicable standardized structural measures where supported

Important: General-purpose static analysis tools do not automatically provide complete ISO/IEC 5055 measurements.

A vendor’s specific implementation, supported rule coverage, analysis scope, and conformance claims must be verified before using a tool to produce formal ISO 5055 measures.

6. Practical example: Start with Semgrep Community Edition

Semgrep Community Edition is an open-source static analysis engine that can help identify selected source code weaknesses.

Install it in a Python virtual environment:

python3 -m venv .venv
source .venv/bin/activate

python -m pip install semgrep

On Windows PowerShell:

.venv\Scripts\Activate.ps1Code language: CSS (css)

Scan the repository:

semgrep scan --config auto .

This uses Semgrep’s automatic configuration, which may retrieve applicable rules from the Semgrep registry and therefore can require network access.

Review the findings and determine:

Is the issue applicable?
Can the code path be reached?
What is the potential impact?
Is the finding a false positive?
Does the proposed remediation eliminate the weakness?

The scan provides useful analysis evidence, but it is not an ISO 5055 conformance assessment.

7. Example structural security weakness

Consider a Python application using SQLite.

An unsafe approach might construct a SQL query by concatenating untrusted input:

query = (
    "SELECT id FROM users WHERE email = '"
    + email
    + "'"
)Code language: JavaScript (javascript)

This construction can create SQL injection risk.

A safer approach uses parameterized queries:

cursor.execute(
    "SELECT id FROM users WHERE email = ?",
    (email,)
)Code language: JavaScript (javascript)

The database driver handles the supplied value as a parameter instead of interpreting it as SQL syntax.

This example illustrates the type of weakness static analysis can help detect. It does not mean that identifying this single weakness satisfies any particular ISO 5055 structural quality measure.

8. Establish a structural quality baseline

Run analysis against a defined software revision and record the results.

A practical dashboard might contain:

Dimension	Baseline measure	Improvement objective
Security	Confirmed relevant weaknesses	Eliminate unacceptable weaknesses
Reliability	Confirmed reliability-related weaknesses	Reduce recurring reliability risks
Performance efficiency	Confirmed structural performance issues	Address material performance risks
Maintainability	Confirmed structural maintenance issues	Reduce high-impact maintenance problems

Use the actual measurement rules specified by ISO 5055 when producing standardized ISO 5055 results.

Generic static analysis finding counts, severity labels, and custom dashboards should be clearly distinguished from formal standard measures.

9. Classify and prioritize findings

Not every finding has equal business importance.

Consider:

Potential impact.
Exploitability or triggering conditions.
Component criticality.
Reachability.
Exposure.
Likelihood of operational failure.
Cost of corrective maintenance.
Confidence in the analysis result.

A significant security weakness in an externally exposed authentication service may require more urgent action than a low-impact maintainability issue in an isolated utility.

Prioritization should be based on actual risk and the organization’s agreed acceptance criteria.

10. Implement structural quality gates

A quality gate is a defined condition that determines whether a change or release can proceed.

Examples of policy rules include:

Security gate: A confirmed, unacceptable security weakness must be resolved or handled under an explicitly authorized risk-exception process.

Reliability gate: Critical resource-management weaknesses in important services require remediation before release.

Maintainability gate: New high-impact maintainability problems require engineering review.

Performance gate: Structural performance findings affecting critical paths must be investigated and evaluated against performance objectives.

These gates are examples of organizational policy. Their thresholds are not automatically defined by ISO 5055.

11. Integrate structural quality into CI/CD

A practical workflow is:

Developers submit a source code change.
Static analysis examines applicable code.
Findings are classified and reviewed.
Confirmed issues are linked to code changes or remediation work.
Quality policies determine whether additional action is required.
Approved releases retain relevant evidence.
Periodic analysis examines accumulated structural risk.

Early analysis can reduce rework by identifying weaknesses close to the point where they are introduced.

Whole-system analysis remains important because some structural problems involve architectural interactions that cannot be identified from an isolated code change.

12. Measure trends responsibly

Useful management questions include:

Are significant structural weaknesses decreasing?
Are new changes introducing recurring weakness patterns?
Which components carry the greatest risk?
Are remediation activities improving the product?
Are supplier-delivered systems meeting contractual quality expectations?

Comparisons should account for changes in codebase size, technology, scan coverage, rule sets, and system boundaries.

An increase in findings after enabling a more capable analyzer does not necessarily indicate that software quality has deteriorated.

It may reflect improved detection.

13. CISQ and software sizing

CISQ’s work extends beyond structural quality.

Automated Function Points provide a standardized way to measure the functional size of certain software applications.

ISO/IEC 19515:2019 addresses Automated Function Points.

Functional sizing and structural quality measurement answer different questions.

Functional sizing: How much functional software capability is represented?

Structural quality measurement: What important weaknesses exist in the internal software construction?

Both can contribute to procurement, estimation, benchmarking, and engineering management when their applicability and limitations are understood.

14. CISQ versus traditional code quality scores

A generic static analysis score may combine complexity, duplication, coding conventions, and selected security checks.

ISO 5055 focuses on defined structural weaknesses that have important implications for security, reliability, performance efficiency, and maintainability.

These concepts should not be treated as interchangeable.

A codebase with a high tool-generated quality rating can still contain serious architectural defects.

Likewise, an isolated low-severity rule violation does not necessarily represent significant operational risk.

15. Common mistakes

Mistake: Assuming all static analysis tools produce ISO 5055-compliant results.

Verify support for the actual standardized measures and their scope.

Mistake: Treating source code analysis as complete security testing.

Security assurance also requires appropriate architectural, configuration, integration, dynamic, and operational evaluation.

Mistake: Comparing inconsistent scanning baselines.

Keep rule sets, scope, and measurement definitions comparable.

Mistake: Measuring only total issue counts.

Prioritize confirmed weaknesses based on their potential consequences.

Mistake: Ignoring architectural relationships.

Important problems may exist across components and technologies rather than inside one source file.

Conclusion

CISQ provides a valuable foundation for standardized automated software measurement.

ISO/IEC 5055 helps organizations evaluate significant structural weaknesses affecting security, reliability, performance efficiency, and maintainability.

When combined with broader product quality models, testing standards, and development process improvement practices, structural measurement can help organizations make better decisions about software risk, technical debt, and long-term maintainability.

Official resources

Verified against CISQ, ISO/IEC 5055, ISO/IEC 19515, and Semgrep’s official Community Edition guidance. CISQ+3

How all seven frameworks work together

These frameworks should not be treated as seven competing alternatives. Each answers a different quality engineering question.

#chatgpt-mermaid-_r_m6_{font-family:-apple-system-body,ui-sans-serif,-apple-system,system-ui,”Segoe UI”,Helvetica,”Apple Color Emoji”,Arial,sans-serif,”Segoe UI Emoji”,”Segoe UI Symbol”;font-size:16px;fill:rgb(13, 13, 13);}@keyframes edge-animation-frame{from{stroke-dashoffset:0;}}@keyframes dash{to{stroke-dashoffset:0;}}#chatgpt-mermaid-_r_m6_ .edge-animation-slow{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 50s linear infinite;stroke-linecap:round;}#chatgpt-mermaid-_r_m6_ .edge-animation-fast{stroke-dasharray:9,5!important;stroke-dashoffset:900;animation:dash 20s linear infinite;stroke-linecap:round;}#chatgpt-mermaid-_r_m6_ .error-icon{fill:rgb(243, 243, 243);}#chatgpt-mermaid-_r_m6_ .error-text{fill:rgb(13, 13, 13);stroke:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .edge-thickness-normal{stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .edge-thickness-thick{stroke-width:3.5px;}#chatgpt-mermaid-_r_m6_ .edge-pattern-solid{stroke-dasharray:0;}#chatgpt-mermaid-_r_m6_ .edge-thickness-invisible{stroke-width:0;fill:none;}#chatgpt-mermaid-_r_m6_ .edge-pattern-dashed{stroke-dasharray:3;}#chatgpt-mermaid-_r_m6_ .edge-pattern-dotted{stroke-dasharray:2;}#chatgpt-mermaid-_r_m6_ .marker{fill:rgb(143, 143, 143);stroke:rgb(143, 143, 143);}#chatgpt-mermaid-_r_m6_ .marker.cross{stroke:rgb(143, 143, 143);}#chatgpt-mermaid-_r_m6_ svg{font-family:-apple-system-body,ui-sans-serif,-apple-system,system-ui,”Segoe UI”,Helvetica,”Apple Color Emoji”,Arial,sans-serif,”Segoe UI Emoji”,”Segoe UI Symbol”;font-size:16px;}#chatgpt-mermaid-_r_m6_ p{margin:0;}#chatgpt-mermaid-_r_m6_ .label{font-family:-apple-system-body,ui-sans-serif,-apple-system,system-ui,”Segoe UI”,Helvetica,”Apple Color Emoji”,Arial,sans-serif,”Segoe UI Emoji”,”Segoe UI Symbol”;color:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .cluster-label text{fill:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .cluster-label span{color:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .cluster-label span p{background-color:transparent;}#chatgpt-mermaid-_r_m6_ .label text,#chatgpt-mermaid-_r_m6_ span{fill:rgb(13, 13, 13);color:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .node rect,#chatgpt-mermaid-_r_m6_ .node circle,#chatgpt-mermaid-_r_m6_ .node ellipse,#chatgpt-mermaid-_r_m6_ .node polygon,#chatgpt-mermaid-_r_m6_ .node path{fill:rgb(222, 234, 251);stroke:rgb(83, 154, 248);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .rough-node .label text,#chatgpt-mermaid-_r_m6_ .node .label text,#chatgpt-mermaid-_r_m6_ .image-shape .label,#chatgpt-mermaid-_r_m6_ .icon-shape .label{text-anchor:middle;}#chatgpt-mermaid-_r_m6_ .node .katex path{fill:#000;stroke:#000;stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .rough-node .label,#chatgpt-mermaid-_r_m6_ .node .label,#chatgpt-mermaid-_r_m6_ .image-shape .label,#chatgpt-mermaid-_r_m6_ .icon-shape .label{text-align:center;}#chatgpt-mermaid-_r_m6_ .node.clickable{cursor:pointer;}#chatgpt-mermaid-_r_m6_ .root .anchor path{fill:rgb(143, 143, 143)!important;stroke-width:0;stroke:rgb(143, 143, 143);}#chatgpt-mermaid-_r_m6_ .arrowheadPath{fill:rgb(143, 143, 143);}#chatgpt-mermaid-_r_m6_ .edgePath .path{stroke:rgb(143, 143, 143);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .flowchart-link{stroke:rgb(143, 143, 143);fill:none;}#chatgpt-mermaid-_r_m6_ .edgeLabel{background-color:rgb(252, 252, 252);text-align:center;}#chatgpt-mermaid-_r_m6_ .edgeLabel p{background-color:rgb(252, 252, 252);}#chatgpt-mermaid-_r_m6_ .edgeLabel rect{opacity:0.5;background-color:rgb(252, 252, 252);fill:rgb(252, 252, 252);}#chatgpt-mermaid-_r_m6_ .labelBkg{background-color:rgba(252, 252, 252, 0.5);}#chatgpt-mermaid-_r_m6_ .cluster rect{fill:rgb(243, 243, 243);stroke:rgba(0, 0, 0, 0.1);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .cluster text{fill:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ .cluster span{color:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:-apple-system-body,ui-sans-serif,-apple-system,system-ui,”Segoe UI”,Helvetica,”Apple Color Emoji”,Arial,sans-serif,”Segoe UI Emoji”,”Segoe UI Symbol”;font-size:12px;background:rgb(243, 243, 243);border:1px solid rgba(0, 0, 0, 0.1);border-radius:2px;pointer-events:none;z-index:100;}#chatgpt-mermaid-_r_m6_ .flowchartTitleText{text-anchor:middle;font-size:18px;fill:rgb(13, 13, 13);}#chatgpt-mermaid-_r_m6_ rect.text{fill:none;stroke-width:0;}#chatgpt-mermaid-_r_m6_ .icon-shape,#chatgpt-mermaid-_r_m6_ .image-shape{background-color:rgb(252, 252, 252);text-align:center;}#chatgpt-mermaid-_r_m6_ .icon-shape p,#chatgpt-mermaid-_r_m6_ .image-shape p{background-color:rgb(252, 252, 252);padding:2px;}#chatgpt-mermaid-_r_m6_ .icon-shape .label rect,#chatgpt-mermaid-_r_m6_ .image-shape .label rect{opacity:0.5;background-color:rgb(252, 252, 252);fill:rgb(252, 252, 252);}#chatgpt-mermaid-_r_m6_ .label-icon{display:inline-block;height:1em;overflow:visible;vertical-align:-0.125em;}#chatgpt-mermaid-_r_m6_ .node .label-icon path{fill:currentColor;stroke:revert;stroke-width:revert;}#chatgpt-mermaid-_r_m6_ .node .neo-node{stroke:rgb(83, 154, 248);}#chatgpt-mermaid-_r_m6_ [data-look=”neo”].node rect,#chatgpt-mermaid-_r_m6_ [data-look=”neo”].cluster rect,#chatgpt-mermaid-_r_m6_ [data-look=”neo”].node polygon{stroke:url(#chatgpt-mermaid-_r_m6_-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#chatgpt-mermaid-_r_m6_ [data-look=”neo”].swimlane.cluster rect{filter:none;}#chatgpt-mermaid-_r_m6_ [data-look=”neo”].node path{stroke:url(#chatgpt-mermaid-_r_m6_-gradient);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ [data-look=”neo”].node .outer-path{filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#chatgpt-mermaid-_r_m6_ [data-look=”neo”].node .neo-line path{stroke:rgb(83, 154, 248);filter:none;}#chatgpt-mermaid-_r_m6_ [data-look=”neo”].node circle{stroke:url(#chatgpt-mermaid-_r_m6_-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#chatgpt-mermaid-_r_m6_ [data-look=”neo”].node circle .state-start{fill:#000000;}#chatgpt-mermaid-_r_m6_ [data-look=”neo”].icon-shape .icon{fill:url(#chatgpt-mermaid-_r_m6_-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#chatgpt-mermaid-_r_m6_ [data-look=”neo”].icon-shape .icon-neo path{stroke:url(#chatgpt-mermaid-_r_m6_-gradient);filter:drop-shadow( 1px 2px 2px rgba(185,185,185,1));}#chatgpt-mermaid-_r_m6_ .node text{font-size:14px;font-weight:600;letter-spacing:normal;fill:rgb(0, 79, 153);}#chatgpt-mermaid-_r_m6_ .edgeLabels text{font-size:13px;font-weight:600;letter-spacing:-0.08px;fill:rgb(0, 79, 153);}#chatgpt-mermaid-_r_m6_ .node tspan[font-weight=”normal”],#chatgpt-mermaid-_r_m6_ .edgeLabels tspan[font-weight=”normal”]{font-weight:600;}#chatgpt-mermaid-_r_m6_ .edgeLabel .label rect{opacity:1;rx:13px;ry:13px;fill:rgb(245, 250, 255);stroke:rgb(206, 219, 229);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .node rect,#chatgpt-mermaid-_r_m6_ .node circle,#chatgpt-mermaid-_r_m6_ .node ellipse,#chatgpt-mermaid-_r_m6_ .node polygon,#chatgpt-mermaid-_r_m6_ .node path{fill:rgb(229, 243, 255);stroke:rgba(0, 0, 0, 0.1);stroke-width:1px;}#chatgpt-mermaid-_r_m6_ .node rect{rx:16px;ry:16px;}#chatgpt-mermaid-_r_m6_ .mermaid-decision .label-container{fill:rgb(245, 250, 255);stroke:rgb(206, 219, 229);stroke-dasharray:2,2;}#chatgpt-mermaid-_r_m6_ .edgePaths .flowchart-link{stroke:rgb(143, 143, 143);stroke-width:1px;stroke-linecap:round;stroke-linejoin:round;}#chatgpt-mermaid-_r_m6_ .marker{fill:rgb(143, 143, 143);stroke:rgb(143, 143, 143);}#chatgpt-mermaid-_r_m6_ :root{–mermaid-font-family:-apple-system-body,ui-sans-serif,-apple-system,system-ui,”Segoe UI”,Helvetica,”Apple Color Emoji”,Arial,sans-serif,”Segoe UI Emoji”,”Segoe UI Symbol”;}Business Needs and ProductRisksISO/IEC 25010 – DefineQualityIEEE 730 – Govern QualityAssuranceISO/IEC/IEEE 29119 – Planand Perform TestingQuality Evidence and ReleaseDecisionsTMMi – ImproveOrganizational MaturityTPI NEXT – Improve SpecificTest CapabilitiesCISQ / ISO 5055 – MeasureStructural QualitySix Sigma – Analyze andReduce Recurring DefectsContinuous Improvement

Conceptual integration of software quality standards and process improvement methods; this is an implementation model, not an official standards architecture.

A practical 90-day implementation program

This is how I would organize an initial quality engineering improvement initiative using the seven frameworks.

Phase	Frameworks	Primary work	Expected output
Days 1–15	ISO 25010, IEEE 730	Identify product risks, quality objectives, and assurance responsibilities	Quality charter and assurance baseline
Days 16–30	TMMi, TPI NEXT	Assess testing maturity, weaknesses, and improvement priorities	Assessment findings and improvement backlog
Days 31–45	ISO 29119	Improve testing strategy, design, documentation, and traceability	Consistent testing practices
Days 46–60	CISQ, ISO 25010	Introduce relevant structural, functional, and non-functional checks	Quality measurement baseline
Days 61–75	Six Sigma	Investigate recurring defects and test targeted improvements	Root cause analysis and corrective actions
Days 76–90	TMMi, TPI NEXT, IEEE 730	Evaluate results and institutionalize effective practices	Continuous improvement roadmap

This schedule establishes foundational practices. It does not represent an official certification program or guarantee that any specific maturity level will be achieved.

Which framework should an organization adopt first?

Organization’s immediate challenge	Best starting point
Testing differs significantly between teams	TMMi
A few testing bottlenecks are causing substantial delays	TPI NEXT
Test documentation and execution lack consistency	ISO/IEC/IEEE 29119
Non-functional requirements and quality objectives are unclear	ISO/IEC 25010
Engineering quality governance and accountability are weak	IEEE 730
Recurring defects persist despite repeated fixes	Six Sigma
Source code structural risks and technical debt are poorly understood	CISQ / ISO/IEC 5055

For an organization starting from scratch, a practical sequence is to define product quality with ISO 25010, establish assurance ownership using IEEE 730, organize testing around ISO 29119, and use TMMi or TPI NEXT to guide process improvements. CISQ measures and Six Sigma techniques can then strengthen the evidence and continuous improvement process.

The objective is not to implement the largest possible collection of standards. It is to create a quality engineering system in which requirements are measurable, testing is effective, structural risks are understood, assurance decisions are traceable, and recurring problems lead to demonstrable improvements.

Rajesh Kumar

I’m Rajesh Kumar, a DevOps, SRE, DevSecOps, Cloud, and Platform Engineering expert passionate about sharing practical knowledge, real-world experiences, and industry best practices. I have worked at Cotocus and regularly write about technology, travel, investing, health, product reviews, and digital marketing through my various platforms.

I publish technical articles at DevOps School, travel stories at Holiday Landmark, stock market insights at Stocks Mantra, health and fitness guidance at My Medic Plus, product reviews at TrueReviewNow, and SEO and digital marketing strategies at Wizbrand.

Find Trusted Cardiac Hospitals

Compare heart hospitals by city and services — all in one place.

Explore Hospitals

Find the Best Cosmetic Hospitals

Tutorial 11 — TMMi: A Complete Guide to Test Maturity Model Integration

TMMi Explained: How to Assess, Improve, and Mature Your Software Testing Process

1. What is TMMi?

2. Understanding the five TMMi maturity levels

Level 2: Managed

Level 3: Defined

Level 4: Measured

Level 5: Optimization

3. How to implement TMMi step by step

Step 1: Establish the assessment scope

Step 2: Establish the testing baseline

Step 3: Define a testing policy and strategy

Step 4: Introduce risk-based test planning

Step 5: Improve test design and execution

Step 6: Establish useful quality metrics

Step 7: Build continuous improvement into operations

4. Example TMMi implementation roadmap

5. Recommended supporting tools

6. Informal assessment versus formal assessment

7. Common implementation mistakes

8. How to measure success

Conclusion

Tutorial 12 — TPI NEXT: Business-Driven Test Process Improvement

TPI NEXT Tutorial: Assess Your Testing Process, Find Weaknesses, and Build an Improvement Roadmap

1. What is TPI NEXT?

2. Understand the 16 TPI NEXT key areas

3. The TPI NEXT maturity levels

What are checkpoints?

What are clusters?

What are enablers?

4. Step-by-step TPI NEXT implementation

Step 1: Identify the business problem

Step 2: Define the assessment scope

Step 3: Assess the current capabilities

Step 4: Perform evidence-based gap analysis

Step 5: Prioritize the improvements

Step 6: Implement targeted changes

Step 7: Reassess and measure benefits

5. Example improvement plan

6. Applying TPI NEXT in Agile and DevOps

7. TPI NEXT versus TMMi

8. Common mistakes

Conclusion

Tutorial 13 — ISO/IEC/IEEE 29119: Complete Software Testing Standards Guide

ISO/IEC/IEEE 29119 Explained: Software Testing Processes, Documentation, Techniques, and Practical Implementation

1. What is ISO/IEC/IEEE 29119?

2. Understand the different parts

Part 1: General concepts

Part 2: Test processes

Part 3: Test documentation

Part 4: Test techniques

Part 5: Keyword-driven testing

3. Implement ISO 29119 step by step

Step 1: Define the testing objectives

Step 2: Establish a test strategy

Step 3: Create a test plan

Step 4: Apply test design techniques

4. Practical automation with Python and pytest

5. Establish requirements traceability

6. Test execution and defect management

7. Produce meaningful test reports

8. Apply the standard in Agile and DevOps

9. Common mistakes

Conclusion

Tutorial 14 — ISO/IEC 25010: Software Product Quality Models

ISO/IEC 25010 Tutorial: How to Define, Measure, and Improve Software Product Quality

1. What is ISO/IEC 25010?

2. The nine software product quality characteristics

3. Important differences from the older quality model

4. How to implement ISO 25010

Step 1: Identify stakeholders and their expectations

Step 2: Identify critical quality characteristics

Step 3: Write measurable quality requirements

Step 4: Build a product quality matrix

5. Measuring performance efficiency

6. Measuring reliability

7. Measuring security

8. Measuring maintainability

9. Measuring interaction capability