Expectations from continuous integration (CI) and continuous delivery (CD) tools differ between teams and organizations. Small projects require relatively simple CI/CD solutions, where the simplicity of configuration (with a handful of defaults) is the key to success. However, mature, enterprise teams have more intricate processes, with hundreds of thousands of tests and advanced release pipelines. Configuration-as-code is a natural fit for such teams. But you need a sophisticated tool to support it.
Even when thinking about small to medium-sized teams, if the number of team members grows into the thousands, we’re dealing with a new problem of coordination and proper organisation, which affects the CI world as well.
In Bamboo 6, we introduced support for configuration-as-code, with Java as the language of choice for defining build plans. Fast-forward to today, and customers can use a configuration-as-code approach to deployments, use YAML configs (if you’re not into the whole Java thing, which is what I’ll focus on here), store their coded configs in repositories, and control permissions on those repos.
So now the question is: how can teams and build admins best take advantage of all that power?
I have a few tips and suggestions for you. Most are aimed at every-day users who create plans and deployments. But don’t close this tab yet, Bamboo administrators – you’ll find some useful information too.
1. Learn by exporting existing Bamboo plans and deployments to Specs
If you already have an instance of Bamboo with plans and deployments, exporting will speed up your learning process as well as make your migration experience seamless. It’s a perfect starting point if you don’t know much about Bamboo Specs or configuration-as-code in general.
Check the documentation on how to export your existing Bamboo configs to Specs. In the code generated during export, you’ll see how the structure of Bamboo is represented by various dedicated Java classes and method calls.
If you don’t have any Bamboo plans yet, and you’re only beginning your journey with Bamboo, the export feature may still come in handy! First of all, you may try to create a plan via the UI and export it to have a working codebase. Additionally, it can be used to help you figure out how to configure particular areas of Bamboo via code. If you know where to go in the user interface to apply changes, but have no idea how to reflect those changes in the Specs world, try applying them in the browser, and then export your configuration.
Please note, however, that any exported content will by no means look like what you want in the final shape of your Specs. It’s just something to start with. The generated specification will work so it can be immediately committed to a VCS repository to use with Repository Stored Specs. From there, it’s a matter of code improvements and refactoring.
2. Test your Specs!
One of the strengths of the Bamboo Java Specs is that offline validation is available for free. Local validation allows you to write more meaningful unit tests for your content. Simply building your plan or deployment project executes most of the checks. In addition, testing your configuration brings in possibilities like team-wide or company-wide consistency. Tests may also help you to validate compatibility with organisational policies and frameworks.
There is, however, something that you need to be aware of. Because the local validation is happening offline, some Bamboo constraints will not be verified. It’s not 100% guaranteed that plans passing such tests will be afterward accepted by Bamboo. Nevertheless, many typical errors and programmers’ mistakes can be detected early.
Here is some code sample to demonstrate the very basic check for your plan spec:
import com.atlassian.bamboo.specs.api.builders.plan.Plan;
import com.atlassian.bamboo.specs.api.util.EntityPropertiesBuilders;
import org.junit.Test;
public class MyPlanTest {
@Test
public void testMyPlan() {
final Plan myPlan = ...;
// if validation fails, an exception will be thrown
EntityPropertiesBuilders.build(myPlan);
}
}
|
3. If possible, keep Bamboo Specs and your build code together*
* Terms and conditions may apply 
This is a big decision to make: where to store Bamboo Specs. My advice is to keep your code and CI definition together, as these are closely coupled. Consider Bamboo Specs to be a higher-level Makefile for your code. Your sources, as well as information “how to build it”, “how to properly test it”, and even “how to deploy it” should be colocated.
That said, there are cases where you’d want to separate your Bamboo Specs from your source base. To give some examples, one repository containing Bamboo Specs may operate on multiple source code repositories, and none of these may technically “own” the build pipeline definition. Or, it’s just more efficient to maintain permissions for a single VCS repository.
Ultimately, you should aim to couple sources and Specs, but don’t force it. Complex build pipelines are usually quite unique, and thus it’s hard to say which option is objectively better.
4. Progressively extract common build configuration to shared components
While migrating your build infrastructure to Bamboo Specs, you’ll find more and more configuration similarities across your build plans and deployments. Shared behaviour does not need to be repeated and redundancies can be removed. That’s one of many benefits of defining your plans using code! This approach allows you to extract shared logic into helper/utility classes, and use programming patterns like factories or factory methods.
Sharing configuration chunks across plans reduces the cost of code maintainability. It also makes it easier to add new content to your build specifications in the future. Both factors are critical for scalable build infrastructure.
To give an example: the Bamboo development team discovered a very common pattern of testing plugins against Bamboo core. The tasks which were executed for each plugin looked almost identical. That’s when we’ve decided to create helpers and utilities for them. Currently, defining a plugin to be built and tested against Bamboo requires adding merely a few lines of code, with a few configuration options available. A bug discovered for one plugin will automatically be fixed for all of them. General CI configuration changes? No problem! Applied everywhere.
In plans building and testing Bamboo we share logic for tasks, projects, plans, capabilities, variables, artifacts, notifications, and many more!
Below is an example of a helper class and method. It relies on an enum to ensure that only known, available versions of Node.js are used by our builds:
/**
* Node.js task shortcuts.
*/
public class NodeTasks {
/**
* Node.js executables configured on our agents.
*/
public enum NodeExecutable {
NODE_8("Node.js 8"),
NODE_6("Node.js 6"),
NODE_4("Node.js 4");
private final String executableLabel;
NodeExecutable(String executableLabel) {
this.executableLabel = executableLabel;
}
}
public static NpmTask npmTask(String description,
String command,
NodeExecutable nodeExecutable) {
return new NpmTask()
.description(description)
.nodeExecutable(nodeExecutable.executableLabel)
.command(command);
}
...
}
|
5. Maximize the power of Java
This can’t be stressed enough. After you start to “programmatically” control your CI, it will become difficult to imagine how could anyone live without it. Writing a script task? Define it as a resource, and make sure it’s available on the classpath when the code is executed. This will give you, inter alia, the syntax highlighting and validation from your IDE. Want to ensure every plan ever created by anyone is always tested? Use Java reflections for generic testing.
Below is an example of our utility class for defining script tasks, with script body taken from Java resources:
/**
* Script task shortcuts.
*/
public class ScriptTasks {
/**
* Create a script task with inline body from a classpath available resource.
*/
public static ScriptTask scriptTaskFromResource(Class<?> acquiringClass,
String resourceName,
String description) throws IOException {
// search for the resource on the classpath under various paths
final URL resource = ObjectUtils.firstNonNull(
acquiringClass.getResource(resourceName),
acquiringClass.getResource("/" + resourceName),
acquiringClass.getResource(acquiringClass.getSimpleName() + "/" + resourceName));
if (resource == null) {
throw new NullPointerException("Script body not found for " + resourceName);
}
try (Scanner scanner = new Scanner(resource.openStream(), StandardCharsets.UTF_8.name())) {
final String scriptBody = scanner.useDelimiter("\\A").next();
return new ScriptTask()
.description(description)
.inlineBody(scriptBody);
}
}
...
}
|
6. Share parts of your configuration amongst teams
If you are familiar with the DevOps culture, you may have already encountered dedicated teams responsible for maintaining build and release infrastructure for a group of development teams. For example, a shared release pipeline may exist which should be used across the company. Regardless of the reason (maintainability, legal aspects, you name it), Bamboo Java Specs can help you out.
If you can use tools like Maven or Gradle. All it takes is to have dependencies on artifacts maintained by other teams. These artifacts can then produce or alter your Bamboo Java Specs in any way. Plenty of possibilities include sharing helper classes for adding requirements, tasks, jobs, stages, variables… even code for setting up entire plans and deployments can be common.
In addition, let’s not forget about permissions. Shared content may help you properly configure the permission scheme for your CI. Common utilities may grant correct permissions to specific users, groups, and roles so each team doesn’t have to be concerned about Bamboo security.
7. Progressively build a framework for your Specs as you scale
For many teams, this step might feel unnecessary and may appear as a symptom of over-engineering, and that’s fully understandable. With the approach of small modules built and maintained independently, your CI configuration may never exceed the size to require an additional layer over the provided toolset.
But imagine you are building and testing a large product. What would you do if you needed to test hundreds of dependencies and configuration options? Various operating systems, database management systems, versions of libraries… Maintaining a multi-dimensional build matrix may give you headaches.
The more plans you have, and the more time it takes to organize your specs into a logically consistent whole, the more a need emerges for a new abstraction layer. That’s what I refer to as “framework”. A little bit of inversion of control, some additional processing logic, whatever suits you best. Such an approach, in my opinion, takes the configuration-as-code aspect of your CI to a new step. Paradoxically, you may find it easier to control your CI with “inversion of control” in place.
To give you an example: the Bamboo dev team, when defining plans which build and test Bamboo, came up with a concept of a “job group”. Job groups… well, logically group jobs which are related to each other (for example, a group of jobs which build and test plugins bundled with Bamboo). Job groups are defined on an abstract level and are not linked to any “Plan” entity directly.
At a certain point, the framework kicks in, which combines job groups into plans. Triggers, repositories, and notification settings are validated and merged to form a complete plan definition. Job-groups defined in this way can be passed between plans according to our needs more freely. At a certain point this allowed us to merge many (way too many…) plans together and simplify an overly-complex CI configuration.
Configuration-as-code made it much simpler to move building blocks around, and that’s why I encourage you to experiment in this area if you feel like it.
8. Use repository-stored Specs for fine-grained audit log and delegated access control
If you configure repository-stored Bamboo Specs, you will discover many available maintainability improvements for your CI setup. *
For improved traceability, set up repository-stored Specs as your team’s primary interaction with Bamboo. You can narrow down possible ways users change Bamboo plans, by disallowing UI access for them, leaving Bamboo Specs as the only option. Regardless of how harsh this sounds, it immediately simplifies many things, including Bamboo permission management. Accessibility may be controlled on the repository level (i.e., only users who are allowed to commit changes to the Specs repository can effectively make changes to Bamboo), while in Bamboo itself only a handful of permission schemes need to be configured (e.g., which repositories have access to which Bamboo components).
The above change alone will give you the benefit of an alternative, clear audit log – via the commit history of your repository. Additionally, with repository-stored Specs in place, you may configure your VCS host to only allow changes via approved pull requests, for even more control over Bamboo content. Never again worry about undetected, unwanted modifications to your CI/CD pipeline!
* Going this route means you’ll lose access to the learning mechanism suggested in “Tip 1”, above. So I recommend being reasonable with limitations.
Configuration-as-code for the win
If you have never tried configuring your CI/CD via source code, it may feel a little awkward at first. It’s a matter of enough exploration and familiarity to begin optimizing your CI configuration for maintainability and scalability. Hopefully the tips I’ve provided will help you get you up and running faster. If you do have experience with configuration-as-code, I do hope that the tips here will bring you at least a few new ideas for improving your CI pipeline and Specs code base.
The Bamboo dev team has learned a lot in the entire process of developing Bamboo Java Specs. We’ve been constantly migrating our own build pipeline to Specs, which gave us a great opportunity to learn and improve (with pretty fast feedback loop!). That’s why I believe that the lessons we’ve learned will help you on your journey with Bamboo.
. . .
If managing your continuous delivery pipeline with configuration-as-code sounds good, wait ’til you see everything else Bamboo has to offer.