Over the past two decades, quality assurance in software engineering has seen a considerable revolution. Agile ideologies and its ensuing methodologies forced an immediate shift from the once phase-gated practices that mimicked manufacturing quality control to the more recent ideas within the DevOps movement that encourage continuous testing and an increased individual responsibility for quality. 

As software development continues to evolve, the ultimate desire to scale efforts across enterprise organizations is growing, resulting in several documented and proficient methodologies for scaling Agile practices to the enterprise. This Refcard will walk through patterns and anti-patterns in quality assurance within the Lean- and Agile-forward implementations of one such methodology, the Scaled Agile Framework, herein referred to as SAFe.

This is a preview of the Quality Assurance Patterns and Anti-Patterns Refcard. To read the entire Refcard, please download the PDF from the link above.

Quality assurance, in its purest form, is a process of inspection and adaptation. This means creating and maintaining quality in the products, processes, and teams responsible for building software by gathering data around what’s actually happening, and then, if deemed necessary, making changes in an attempt to improve quality. It is of no surprise that several quotes from W. Edwards Deming, who is often referred to as "the father of quality improvement," can be found throughout many SAFe materials. 

This connection to Deming’s work is most openly noted by the inclusion of the phrase "built-in quality," as adapted from Deming’s landmark Out of the Crisis, as the leading principle for understanding what quality means within SAFe. Deming and his work with Toyota strongly connect Lean thinking to Deming’s management and quality philosophies, which, for brevity, can be referred to as the quality assurance mindset "Lean Quality Assurance". So, what benefits can teams expect to gain from adopting the Lean Quality Assurance mindset proposed within SAFe?

This is a preview of the Quality Assurance Patterns and Anti-Patterns Refcard. To read the entire Refcard, please download the PDF from the link above.

Pattern #1: Defining Quality Based on Context 

Quality, especially in the context of software development, is subjective. This makes it incredibly difficult to discuss, and even more difficult to attempt to manage at scale. Gerald Weinberg stated it best in How Software is Built, defining quality as "value to some person". 

Understanding the human element to quality opens the door to effectively communicating about quality, not just assuming that teams know the implications of "quality", "excellence", or "insert management-word-here". So, as it relates to quality assurance, how can teams assure the appropriate language and shared context?

Pattern #2 Built-in Quality and the Value Stream 

 The language of “built-in quality” tends to spark a connection with those generally seen as “builders”, and less so with those whose connection to quality is less direct than the production of a clear customer deliverable.  

Unfortunately, this couldn’t be further from what built-in quality should impart within a product development context. The concept of building in quality stems from Harold F. Dodge, spread further through W. Edwards Deming’s Out of the Crisis, stating that “quality can not be inspected into a product or service; it must be built into it”. 

To provide context, Deming was discussing the process of mass inspection within a manufacturing facility as an ineffective means of improving quality. Earlier within the book, Deming also states that instead of inspection, an “improvement of production process” should be the aim, providing a definition of building in quality as “improvement of the production process”. 

Pattern #3: Improvement of Mindset and Tooling

Developers love their tools. They also know what happens when they misuse tools – chaos, distrust, and abandonment. The mindset of tool danger is so prevalent that it even served to create the first line of the Manifesto for Agile Software Development: “individuals and interactions over processes and tools”.  

Unfortunately, this tool-heavy approach is also prevalent in implementations of quality assurance within scaled settings as well. While it may be exciting to attempt to roll out a Behavior-Driven Development tool to enhance alignment or focus obsessively over creating the perfect automated system test suite, emphasis on these efforts without paying equal attention to developing the necessary Lean and quality-first mindsets amongst an organization can ultimately limit the effectiveness of the tooling. 

Specifically, within the context of SAFe, organizational leaders are encouraged to take a hands-on approach to quality, again relying on the principle of built-in quality to make any necessary improvements, including enhanced tooling. This aids in building the appropriate reverence and respect for the quality of the product, and, ultimately, the customer in the journey to delivering value.

Pattern #4: Shift Left — and Right

Software products have become more complex as new developments continue in the fields of artificial intelligence and cloud computing. Along with this, complexity has grown the mindset of “shifting left”, which aims to shorten the standard timeline and tests to get feedback more quickly and combat complexity. This is a key component for a Lean implementation and is also defined here as an essential benefit of a Lean quality assurance model.

However, along with shifting left, there should also be a desire to shift right and utilize production systems and data to further validate quality and locate opportunities for improvement. This concept is often described as continuous testing, which encourages additional testing during the deployment, operation, and monitoring of an application. 

In a scaled setting where products and teams tend to be large, this can be incredibly difficult, but the richness of the data potentially left unexplored is too valuable to be ignored in a comprehensive quality assurance strategy. 

This is a preview of the Quality Assurance Patterns and Anti-Patterns Refcard. To read the entire Refcard, please download the PDF from the link above.

In this refcard, four quality-first patterns and anti-patterns were introduced, along with the benefits that they can provide to an organization while attempting to continuously and incrementally deliver value in a paradigm of Lean Quality Assurance.  

Lean Quality Assurance in a scaled environment presents new challenges and opportunities for quality-minded individuals to evangelize and reinforce the importance of quality to their organization. Ultimately, these Lean practices can help to build a culture where accountability and responsibility for quality are shared from leadership all the way down to the individuals on a team.

This is a preview of the Quality Assurance Patterns and Anti-Patterns Refcard. To read the entire Refcard, please download the PDF from the link above.