Compliance in the age of DevOps.
When I was still practicing law, the engineering team at my former company used a waterfall development methodology. One of their software development lifecycle (SDLC) phases was a long, manual, mandatory review of all the open-source licenses used in the application. The goal was to ensure open-source packages were compliant with their open-source licensing policies. After adopting Agile and DevOps, the engineering team replaced that phase with an automatic process, and the legal reviews that used to take days or weeks are now done in minutes. I moved on, too – from writing contracts to writing code.
With Agile and DevOps, the focus has shifted from control to speed. The idea is to deliver software to users earlier, get feedback earlier, and iterate on that feedback to make the software better, faster. The faster a team can deploy releases, the higher their software delivery performance.
Teams achieve high delivery performance by removing bottlenecks – increasing developer autonomy, automating workflows, and codifying and hosting everything on decentralized source controls like Git (also called “GitOps”). But autonomy and automation result in more decentralization, and therefore complexity; you have more code, more repositories, and more responsibilities for developers. Coordinating and ensuring alignment among developers becomes very difficult without controls.
But implementing excessive control is not advisable either, as it conflicts with the cultural foundations of Agile and DevOps (removing bottlenecks, remember?) and as a result negatively affects software delivery speed. This is a tricky problem when you’re looking to mandate development practices like traceability.
What is traceability?
“Traceability” is the ability to describe and follow the life of a requirement in both a forwards and backwards direction – from requirements-to-code, and vice versa. When executed effectively, traceability allows you to track work items across the development lifecycle to where requirements are implemented in the code, playing a crucial role in your organization’s control framework.
Why you need traceability
As the co-creator and head of product at Datree, I interviewed dozens of development teams to learn about their workflows and understand their challenges. I learned that there are three main reasons development organizations should implement traceability.
1. Achieve compliance
The most obvious reason for implementing traceability has to do with regulations. This is typically true for companies operating in financial services, healthcare, and government sectors. The ability to trace code to requirements is a prerequisite criterion among popular compliance frameworks – for example, the SOC2 process integrity requirement can’t be fulfilled without a full chain of evidence for every code change in the application source code.
2. Reduce software defects (bugs)
There is a statistically significant relationship between the completeness of the captured traceability information and the defect rate of the developed source code. More complete traceability decreases the expected defect rate in the developed software. The strong impact of traceability on the defect rate suggests that traceability is of great practical value for any kind of software development project, even if it’s not mandated by a standard or regulation.
3. Increase speed and code maintainability
A study in the Empirical Software Engineering journal shows that one group of developers who implemented traceability performed a given task 24 percent faster and created 50 percent more correct solutions on average, suggesting that traceability not only saves effort but can profoundly improve software maintenance quality. Well-maintained code can help developers make changes in code written by other teams and reuse other people’s code, and also helps decrease technical debt. The recently released State of DevOps report: 2019 also claims that “organizations that elevate code maintainability provide real advantages to their engineers.”
This first reason is a mandatory one if your organization is required to comply with regulations. But even if compliance isn’t a requirement, the processes enabled through requirement-to-code traceability can materially improve your team performance by lowering the number of defects in production and increasing development velocity.
Implementing traceability in the agile and DevOps era
Keep in mind that compliance frameworks also existed in the pre-DevOps era, meaning compliance standards were crafted with the waterfall method in mind. When you implement traceability as an “organization-level” requirement, you must do so with care. Don’t fall into the trap of applying the waterfall way of compliance, which creates bottlenecks and affects speed.
By now you may be convinced of the “what” and “why” of traceability, but I also want to address the trickier part: how to ensure that the practice of traceability is adopted by development teams across your organization, without the excessive control of a phase-based approach.
Organizations often respond to problems with software releases by introducing additional processes (phases) to the SDLC. They then struggle to enforce them effectively. In my experience as a developer and researching development workflows, I saw too many organizations implement compliance steps in the wrong way. For example, communicate practices to follow without introducing workflow adjustments, and effectively relying on developers to remember to follow steps or, worse, creating a heavyweight change management process to gain control at the expense of speed and autonomy. Those methods are increasing the mental burden for the team and they are Agile and DevOps anti-patterns!
So what’s the solution? Here’s how to “hack” the DevOps principles (fast feedback loop) to meet requirements for traceability, so you can eat your cake and have it too.
Step 1: Write a lightweight change process guideline
A guideline document helps developers gain a clear understanding of what is required of them when they submit code for review and approval. Keep it short and simple, and don’t neglect the “why” – it is always easier to introduce new workflow steps if the developers understand the importance of following organization-level practices as defined. In the document, explain how traceability benefits the team – link to those studies mentioned above – and provide both good and bad examples.
Step 2: Implement traceability in a “developer-friendly” way
Good – integrations: Developer autonomy also means giving developers the freedom to choose which tools to use. Don’t force a specific solution on the engineering team – instead, implement traceability by creating integrations with their existing tools and workflows.
Better – automation: Implement automatic checks to detect and notify developers if their code is not traceable. Practices such as continuous testing and continuous integration (CI) employ such automatic checks. If you are using CI or automated testing, you’ll want to add a traceability check.
Excellent – shift-left: Give the developer feedback as early as possible if his or her code is not traceable. High-performing engineering teams are doing this as part of the CI process, but excellent engineering teams add indication at an even earlier stage, when the code is pushed to the repository.
What would this look like in the real world? If you’re using Jira, you’ll want to ensure that every code change mentions a Jira ticket and that developers have an option to follow this practice seamlessly, through integrations with source control and the Jira instance. Then, add automated checks as part of your CI process to verify that each code change mentions a Jira ticket number, and set it to warn the developers if ticket numbers are missing when they are pushing code to the repository.
DevOps is all about removing bottlenecks to enable faster software delivery, which is why developer autonomy and automation are the new norms for modern software teams. But increased speed means increased complexity and difficulty in implementing standards and best practices like traceability. Traceability is especially important for compliance purposes, but when implemented properly, studies have shown it actually helps increase software delivery performance through faster releases and better code maintainability. The key to ensuring traceability adoption lies in integration with existing tools and automated checks as part of the existing workflow.