What We Lost When QA Teams Disappeared

In many companies over the last 15 years, QA teams have diminished or even disappeared.

The tech industry decided we do not need special QA teams because everyone should care about quality. As companies aimed to be more flexible and moved away from old step-by-step methods, they stopped using QA as the final check before releases and instead encouraged the whole team to work together.

I think this concept became popular around 2009. The best resource on this is Lisa Crispin and Janet Gregory’s book, Agile Testing: A Practical Guide for Testers and Agile Teams (2009).

Thinking about quality from the beginning is great. It is like handling security or privacy: you have to start as soon as you define what the product should do. Adding quality, security, or privacy at the end is expensive, takes longer, and delays benefits for users.

Developers doing more testing

I want to focus on the impact this has on developers, especially regarding testing practices.

Focusing on quality early led to more automated testing. Developers began testing their own work, mostly using automated tools. This is good. To achieve high product quality, we need to start with quality from the beginning, and everyone involved should pay attention to it.

Developers are good at writing tools and automating tasks, so more testing tools and better automation environments were created. As a result, more code is now covered by tests.

Do we have the right tests in place?

Having close to 100% code coverage with automated tests might sound like a great metric. It is useful because any line of code not executed could cause a failure in production. However, these metrics mostly refer to line coverage, which can be achieved in different ways, sometimes without testing important cases for certain users. This shows a team can have good test coverage but still miss important quality or functional targets.

Developers and testing skills

Most schools and programs that teach software development do not have a proper curriculum for testing. Testing is usually taught as writing unit tests or learning a framework, not as a skill. It is mostly seen as a way to evaluate code, not the holistic product or its value from a quality perspective.

In practice, developers pick up testing skills as they go. If you were in a good culture led by someone who knew testing philosophy and practices, you probably have decent or great testing skills. If not, you may still write tests just to show the code works as written. This is better than having no tests, but these tests might miss important or critical cases for the business and end-users.

Reflect on your own testing habits. Are there areas where your skills might be lacking? Are your tests effectively capturing the needs of your end-users? By regularly evaluating your testing strategies, you can identify gaps in your knowledge and take steps to improve.

Here is an example. Let’s look at the following piece of code:

# https://github.com/basecamp/once-campfire/blob/main/app/models/user.rb#L26
class User < ApplicationRecord
 # t.string "name", null: false

 def initials
   name.scan(/\b\w/).join
 end
end

This code is supposed to generate initials from the user’s name. What tests would you write for it?

• If you focus on achieving 100% code coverage of Users#initials, a single test is enough.
• You could write two tests by adding a negative one that might throw an exception.

Both cases overlook an important point: Who are the users of our application, and which countries are they from? Depending on the country, this method might often return empty. For example, if the name is José Ángel, this would return only J. If the name is 佐藤 陽翔, it will return empty. Maybe this is correct, but my point is that we need test design to uncover for whom we are writing these tests, and then write the tests accordingly.

Knowing who the end-users are and their most common or important use cases is a crucial part of a tester’s approach.

What is currently missing is test design. We know how to implement and execute tests. We can organize and optimize test code, but asking whether these are the right tests is missing from conversations.

With QA vanished, testing best practices, knowledge and wisdom also vanished

Here is the problem today: In companies where the testing team is gone, there is no focus on testing as a skill. Whether the tests are good or not is random. It just becomes about numbers: if you write enough tests, you might cover important situations.

I do not like this approach because it is not systematic. We need to increase developers’ knowledge about testing, especially as developers use LLMs. If you cannot properly ask for a testing concept, you probably will not get it from an LLM. You may get many tests written by an LLM that you have to maintain and execute, but without understanding test design and what to ask for, you might end up with bloat.

The solution: learn test design

The solution is to learn about test design, test design techniques, and discuss how to approach testing in the current context.

Here are some starting points in case you want to learn more about test design: 

• Understand more about the entire test process and where test design fits in.
• Clarify the role of testing: it is about risk, priorities, product context, and project constraints. Accept that testing has tradeoffs, and when designing test cases you have to consider this.
• Get comfortable with test levels and test types. Choosing what to test depends on where and what you are planning to test.
• How to set test objectives and traceability to requirements, use cases, code, or important artifacts
• Test Design Techniques: mathematically backed ways to cover important areas of your code or requirements systematically. The core of test design is to know these techniques and when to apply them.
• Prioritization and risk: ranking test objectives and possible test cases based on risk, and choosing to apply a test design technique to generate more or fewer test cases, is crucial.
• Read about preconditions, postconditions, and the difference between them and assertions. This will help organize your tests even before writing the first line of code.