Integrating Software Supply Chains and DevOps: Tips for Effectively Reconciling Supply Chain Management and DevOps

Editor's Note: The following is an article written for and published in DZone's 2024 Trend Report, The Modern DevOps Lifecycle: Shifting CI/CD and Application Architectures.

Software supply chains (SSCs) have become a prevalent topic in the software development world, and for good reason. As software development has matured, so has our understanding of the dependencies that can affect the security and the legal standing of our products. We only have to hear names like Log4Shell to remember how crippling a single vulnerability can be.

SSCs are a blend of development and operations, and as we start to take SSCs more seriously, it is important that we strike an effective balance of responsibilities in our existing DevOps culture. In this article, we will take a look at how DevOps plays a critical role in SSC management (SSCM) and how we can effectively manage our SSCs with our existing DevOps structures.

The Software Supply Chain 

A supply chain is a network of resources that are required to procure a product. In software, this means all the software artifacts that our product depends on and all the artifacts we publish, including:

• Binaries
• Configurations
• Scripts
• Licenses

For SSCs, there are generally three parts:

1. Upstream – the dependencies our product relies on
2. Build system –  the infrastructure used to build our product
3. Downstream – the artifacts we publish

This conceptual SSC is illustrated below in Figure 1:

Figure 1. SSCs comprise upstream artifacts, a build system, and downstream artifacts

 Image source: Software Supply Chain Security, DZone Refcard

The first step in managing an SSC is to answer some fundamental questions, including:

• What artifacts, including transitive ones, does our product use?
• What artifacts does our product publish?
• Is our build infrastructure secure?
• Do we verify that our dependencies are trustworthy?
• Do we provide verifiably secure forms of trust for our downstream artifacts?

Answering these questions is not a simple task, and it quickly becomes evident that both development and operations play a critical role in managing a supply chain.

Integrating DevOps and Software Supply Chain Management 

Managing an SSC can be a difficult task since there are a lot of pieces to consider. Responsibility for these puzzle pieces are also spread across different groups. For example, if a dependency is vulnerable, it will likely fall on development to upgrade and mitigate the dependency, but changing our deployment infrastructure needs to verify the trustworthiness of our downstream artifacts will likely fall on operations.

We must be cautious against creating the silos that our DevOps culture has succeeded in lowering, but at the same time, we must recognize that development and operations are not interchangeable. To effectively manage an SSC, we must leverage the strengths of each team and apply them to where they are needed, while still maintaining visibility and trust between them.

Striking this balance requires three parts:

1. Visibility
2. Input
3. Accountability

Visibility 

Both development and operations must have access to the supply chain configuration. 

As the SSC for a product grows, both development and operations will need to see the chain. In the past, the build and deployment configuration would reside in a completely different repository, and possibly require a completely different set of tools, from the code itself. In order to effectively manage an SSC, we want our SSC configurations to reside in the same area as our code. That way, both development and operations have access to the configurations, and just as importantly, they can both see the changes that the other makes to the configuration.

Some tips include:

• Keeping repositories tidy and lean so that configurations and scripts are easy to find
• Following established conventions so that everyone knows where to find scripts and configuration in any repository
• Ensuring Software Bills of Materials (SBOMs) are published where development and operations can see them

Input 

Both development and operations must be able to see and approve changes. 

Stemming from this shared visibility, it is critical that both teams have input into changes that will affect the SSC. It becomes clear early on in SSCM that both development and operations have common interests, but sometimes they have competing ideas on how to achieve those goals. 

For example, if our product has a vulnerability in a downstream artifact that makes our deployment vulnerable, it may not be feasible to simply remove that artifact from deployment since it may contain code that is critical for our product to operate correctly. Instead, development and operations need to work together (and both provide input) to see if the vulnerability can be fixed by development or be remediated by operations.

Some tips include:

• Adding both development and operations personnel as approvers on pull requests (PRs)
• Fostering a close working relationship between development and operations personnel so that each can see from the other's perspective
• Ensuring that development and operations are continually in sync ("on the same page")

Accountability 

Both development and operations must both be accountable for the SSC. 

Lastly, accountability by both development and operations is essential for trust to grow and for DevOps to succeed. A simple, "What do you guys think?" can go a long way in building and maintaining the trust and relationship between the development and operations teams. It is equally important that both teams also take shared responsibility for the SSC outcomes. 

For example, if a vulnerability makes it into production, development should be asking, "How did we let that vulnerability slip into our code and get deployed?" And operations should be asking, "How did we let a vulnerable artifact make it into a production environment?" This shared accountability ensures that both teams have the mindset of "What can we do to secure our SSC?" rather than pointing the finger at each other.

Some tips include:

• Fostering an environment of personal responsibility
• Ensuring that development and operations are both praised for successes in the SSC
• Expecting both development and operations to handle SSC issues that come in (not just one or the other)

DevOps Tips for Effective Software Supply Chains 

Visibility, input, and accountability are the cornerstones of effective SSCM, especially in a DevOps culture, but sometimes these concepts can be too abstract. Below are three specific tips that both development and operations can use to implement these concepts on a daily basis.

1. Place Code and Configuration in the Same Repository

A key, although sometimes overlooked, aspect of visibility is adding configuration to a place that development and operations both have access to. The ideal location is the repository that contains the code. Not only does this ensure that both development and operations can see the changes made that will affect the SSC for our product, but it also ensures that both teams do not have to go "hunting" for the configurations in another location or use another set of tools. The same tools that development and operations already use to develop and deploy our product are the same ones that are used to manage the SSC.

2. Utilize Pull Requests Whenever Possible

There are usually two ways to make a change to a repository: commit directly to the repository or create a PR and merge the PR. Creating a PR has the benefit of adding approvers who can approve or reject the PR. This can be a very useful tool when one team is making a change and requests the input of the other. For example, if the development team wants to add a new testing stage in the build pipeline, it is important that the operations team provides their input. This not only creates buy-in by both teams, but it also builds trust. Furthermore, it creates accountability by both teams since the development team created the change and the operations team approved it (or vice versa).

3. Automate as Much as Possible

Manual steps can inadvertently create artificial barriers between teams. For example, if there is a manual step in the deployment process that only operations staff know how to perform, the visibility of the development team is greatly reduced. Instead, we should automate as many of the development and operations steps as possible. This not only increases the agility of the team but also documents the procedure and allows both development and operations to request changes and scrutinize the process.

Conclusion 

SSCM has become a crucial part of software development, and for good reason. An ineffective management process can lead to an ineffective supply chain, which can result in significant financial and reputational damage to both a product and a company. Enacting visibility, input, and accountability for both the development and operations teams not only ensures that our SSCs are secure, but it also provides a crucial opportunity to strengthen our DevOps culture and create trust between our development and operations teams.

This is an excerpt from DZone's 2024 Trend Report,
The Modern DevOps Lifecycle: Shifting CI/CD and Application Architectures.

For more:

Read the Report