Designing Quality into the System

You cannot OBSERVE Quality into the system. We have done a continued disservice to our testing organizations by holding THEM responsible to determine the quality of the systems that we develop post development, rather than holding the development teams responsible for quality and correctness at the time of development.

If you are waiting to determine if you have quality in the system until after development, it is too late, and you have failed to create quality in the system. Our systems aren’t some quasi-Heisenberg Environment that will somehow change because we decide to observe them, so we must engineer quality as an asepct of our development processes, not our testing process.

How do we design quality into the system?

• We can Design it with Strong, Emergent Architectures that are projected as code and built through automation
• We can use SOLID Development Patterns that are unit tested, LINTed and Checked at the Pull request through Automation
• We can ensure good Acceptance criteria that gets projected into good automated Acceptance testing and run through automation regularly.
• We can use these patterns to Prove or Disprove correctness of function and the expression of our desired “–ilities”, which we’ll define shortly.

The Scrum Team

We know that a scrum team is:

• Self-organizing
• Cross-functional
• Comprised of 3-9 members (recommended)
• Without titles, other than “Developer” or "Team Member"
• Without sub-teams (e.g. testing team, BA Teams, etc.)
• Responsible for forecasting and planning the work to correctly transform PBIs into increments of releasable functionality every Sprint

They are Responsible for the “how”.

Because our team is cross-functional, they are responsible for HOW the work is done. This includes how we are going to develop it, test it, automate its delivery, and operate it in production. These are the questions the development team must ask itself as it refines each backlog item. Any gap in the answers indicates a lack of cross-functional ability on the team and/or a lack of understanding of the definition of done.

Collective Ownership

The developers collectively own several aspects of the development process, including:

• Sizing
• Sprint Backlog
• Code
• Writing Tests
• Failing tests
• Failing builds
• Definition of Done
• Failure
• Success

In my post about The Definition of Ready, we determined that the development team collectively owns the implementation of the answer to the question “How will we test this?” AND the fixing of broken tests and code pursuant to that answer.

Agile Testing Manifesto

To build a more complex answer to the question, let's review the Agile manifesto, which is a set of value expressions to guide decisions within any agile process. If we extend that concept to our testing strategy, then we get something like the Agile Testing Manifesto.

The Agile Testing Manifesto represents a set of value expressions regarding our approach to testing as an agile team. It emphasizes:

• Testing Throughout over Testing at the End
• Preventing Bugs over Finding Bugs
• Testing Understanding over Checking Functionality
• Building the Best System over Breaking the System
• Team responsibility for Quality over Tester Responsibility

What Next?

We have our questions, a strong team of developers, and a set of values. So what? What is the role of QA in this new paradigm? Before we can answer that, we need to better define the “Q” in Quality and put it in context with the Agile Team.

Redefining Quality in the Context of Agile

In agile, QA doesn’t go away, but the idea absolutely needs to be refined to fit our model. Starting with redefining “Quality” in the context of Agile. First of all, it’s important to point out that “Quality” is difficult to define and even more difficult to measure, because it will have a different definition and set of measures for each person that you ask.

Before we begin testing, it’s imperative for the organization to define what “Quality” means to the organization and specifically to the product or platform that is being developed. This definition can (and probably should) be inspected for adaptation each sprint through retrospection as it will impact our definitions of ready and done. Quality has many faces (different people define it differently, for their purposes) and is difficult to measure. It can be quantified through the expression of what, in scrum, we call the “–ilities”.

Non-Functional Requirements (NFRs)

First, I recognize that 1.) Not all of Non-Functional Requirement (NFR) words end in “-ility” and 2.) This is far from a complete list. Both the the Product Owner and the development team need to be asked “Which Quality Attributes Do you value and how will they be measured?", and the combination of answers helps us understand the shape of our strategy.

Testing Provides Proof of Quality

Testing does NOT need to be done, nor should it be frequently done, by a human. As a matter of fact, most human-centric testing is inefficient and a gross misuse of development resources. Humans work best when testing edge cases, doing discovery testing. If humans are bad at testing, then what is good? Automation. Humans write the automation that is run quickly, efficiently AND FREQUENTLY. Repeatable, refactorable, and fast, automated testing of all forms is the foundation of a good testing strategy.

So, in general, our testing must:

• Make sure our code functions as designed (functional testing)
• Make sure the code performs under production-like loads (performance testing)
• Make sure the –ilities are fulfilled (non-functional testing)

Agile and Scrum are based on inspection and adaptation. Automated testing of all kinds provides a steady stream of continuous inspection that is usable by both the product owner and the development team to adapt the development to better describe the product goals. The smaller the changes and the faster the feedback cycles, the more accurate and the higher the potential quality of the increment delivered at the end of each iteration.

Agile Testing Quadrants

Different types of testing have different goals and motivations in the overall strategy, and not ALL types of testing are present in all facets of development. The intent, then, is to understand each feature or story enough to understand the question from the definition of ready “How are we going to test this?” such that it fulfills our strategy and our expression of quality.

Creating Coverage that Matters

The Goal is to create enough Testing Coverage of our system to ensure both high quality, and, more importantly, confidence in our ability to refactor, change and adjust based on our most recent observations. Some areas will have more or less total coverage, be subject to more stringent security and performance, or be less likely to come under automated UI testing. But each of those are choices that can only be reliably made in the presence of an understood strategy that is developed, expressed, and implemented by the team.

Who does What?

In the past, “Quality Assurance” or “QA” was typically defined as a team of people dedicated to manually executing test processes to check the correctness of the system when it was asserted to be “Dev Complete”. In this new Agile Paradigm, then, what is the role of the “QA” team? Agile and scrum assert a unified team of cross-functional people - no silos. No Hand offs (which are just mini-waterfalls), but a unified, collaborative approach to delivery. If that's the case, then we have developers who are testing focused, working with functional developers to insure correct testing strategy implementations across the architecture. This participation as development team members encourages good refinement and estiamtion of all of the work necessary to complete a backlog item, pair development throughout the process,and insures our definiiton of done includes appropriate testing at all levels.

QA Specialist Focused

• Test Strategy Development
• Acceptance Test Development
• Automated UI Testing
• Test Harness Specification
• Exploratory Testing

Everyone

• Unit Testing All New Code
• Integration Testing at the Boundaries of the Application Layers
• Performance Testing
• Acceptance Test DSL Creation
• Acceptance Test Code Implementation
• Test Harness Development

On the teams we now have embedded testing professionals which allows for:

• No testing Crunch
• No Handovers
• Localized Testing Expertise
• Developers work on Test Automation Together