QA Roles and Levels in Continuous Delivery Model | Software Testing

0 – Manual compilation, no unit tests, manual testing by QA
Before we all recoil in horror it’s worth recalling the pure horror of how things used to be with an enormous gulf between developers and QA. Thankfully this approach is lost in the mist of time for almost everyone but if has not, you have my deepest sympathies.

1 – Automated compilation, some unit tests by dev, manual testing by QA
We start to see the process become more agile. Build scripts have made their appearance. Life is a bit easier for developers with the introduction of Continuous Integration but not much different to QA who are left out in the cold.

2 – Automated compilation, high standard unit tests by dev, automation of manual testing by QA
Build frameworks that allow failing a build process that successfully compiled but did not pass the unit tests are now used. QA are using tools which theoretically allow them to sign off individual user stories while still being able to quickly regression test the entire application to ensure that no new errors have been introduced. However, the separation of dev and QA ensures that almost every change by developers results in failing QA tests. The only way to get all the tests to pass is to either stop development or to fork the codebase so that the QA’s can work on stable code. Bugs are fed back into the development code branch and promoted to the QA branch. Congratulations for reinventing waterfall and for ensuring that the ratio between developers and QA remains 1:1.

3 – Automated compilation, high standard unit tests by dev, integration tests by dev, automated testing by QA using better tools
Some of the QA workload is taken over by developers who automate obvious integration points with the rest of the system. Some more load is taken off the QA who can re-use tests with tools like Fitnesse but the phrase “dev complete” is still heard. Developers may provide QA with test utilities and abstractions like Stubs and Fakes to make testing easier and more deterministic. It’s almost inevitable that every team’s story board will have 3 or 4 times as many stories waiting for QA than are either “In Development” or “Undergoing QA”. You can do Continuous Delivery at this stage but everyone wonders why it’s so painful and costs so much in time and effort.

What do we really need?

For us to ship reliable quality code, we have to change not just the tooling but how the delivery organisation is structured. When manual testing was the norm, a separate QA organisation existed because it was more efficient and more effective to have non-developers test the code. With the levels of automation now available a separate QA organisation is an anachronism which should no longer exist. So what do we really need?

4 – Automated compilation, high standard unit tests by dev, integration tests by dev, agile testing by dev, sign-off by QA
The role of the QA has mutated to that of QABA (aka ‘a bloody good BA’) – domain experts that represent the business in the delivery team but who are also responsible for creating the acceptance criteria for user stories where the acceptance criteria are expressed as scenarios that can be easily converted into actual test code. Developers write the application code and the code that tests it, including creating any tooling. The QABA can then sign off the story on completion without having to go back to the business. Business still see new features and capabilities at weekly demos and show’n’tells but are rarely involved with the delivery team on a regular basis. If you are B2C, even better is where new code is released into production without being signed off but hidden behind feature throttles that stop the new functionality from being seen outside of the company network. That way the entire company gets to do UAT on new functionality before it hits the customers.
A fundamental pillar of continuous delivery is that all* your tests must be automated. To achieve this the QA organisation should be in the business of writing the test scenarios that the code needs to be evaluated against and for signing off that the code does this. Test code should be a first class citizen of the application and should be written by people who’s primary job is writing code – the developers. I will say it again – QA should not be in the business of writing test code.
Developers are responsible for quality and should act like it. Sometimes that means taking responsibility from the QA organisation that should never should have been given to them. Quality is too important to leave to QA. Developers need to take full responsibility for the quality of their code and they should be in the firing line if something is broken. The role of the QA is to keep the developers on the straight and narrow and the most effective way of doing this is to get them to apply their confrontational mindset to the code via the acceptance criteria used to sign off the new functionality. Those of you familiar with BDD are probably nodding your heads right now but I’m not sure that BDD is the answer. It’s the right approach from the perspective of test case creation but the tooling is currently a zero sum game – the effort you save by having natural language test descriptions run is equivalent to the amount of effort you have to expend in extending your tooling to support your test cases in all bar the most trivial cases.
If you take a look at a company that prioritizes the ability to ship code, e.g. Facebook, you will see the developer taking far more responsibility for QA than is seen in the rest of the industry. It’s time for the rest of us to catch up.

Rajesh Kumar

I’m Rajesh Kumar, a DevOps, SRE, DevSecOps, Cloud, and Platform Engineering expert passionate about sharing practical knowledge, real-world experiences, and industry best practices. I have worked at Cotocus and regularly write about technology, travel, investing, health, product reviews, and digital marketing through my various platforms.

I publish technical articles at DevOps School, travel stories at Holiday Landmark, stock market insights at Stocks Mantra, health and fitness guidance at My Medic Plus, product reviews at TrueReviewNow, and SEO and digital marketing strategies at Wizbrand.