Announcing smarter caches in Bitbucket Pipelines

Caches in Bitbucket Pipelines allows build dependencies and outputs to be stored and reused across many pipeline runs. By reusing work previously done in an initial build, subsequent builds can save time by reducing the need for fetching dependencies or performing expensive computations.

Until today, Bitbucket Pipelines caches have operated much like caches that use time to live (TTL) settings, commonly seen in examples like CDNs and browser caches. Once a pipelines cache is created, it lives for 7 days or until the cache is manually deleted. For many software projects with frequent builds and dependency updates, this type of cache expiry can be suboptimal. Dependencies updated during the course of those 7 days are not stored in the cache so future builds still need to spend time redownloading any new dependencies.

Introducing cache keys

Today we are announcing the option to add cache keys to your Pipelines cache definitions. Cache keys provide a way to uniquely identify versions of a cache based on a set of files in your repository. The typical use case would be to define a cache key based on files that define a project’s dependencies. When dependencies are updated, the hash of the key files also updates and Pipelines will be able to generate a unique cache version for subsequent builds. As multiple cache versions are able to be retained, future builds using either the old or new dependency set will have a unique cache version to reuse.

Here’s an example showing how to define cache keys for a gradle project:

pipelines:
  default:
    - step:
        caches:
          - gradle
        script:
          - ./gradlew assemble

definitions:
  caches:
    gradle:
      key:
        files:
          - build.gradle
      path: ~/.gradle/caches

There can be multiple files listed under the files key. These can refer to any file within your repository. Glob patterns can also be used here to discover multiple key files matching a pattern. This could be useful in a multi-module project for example.

key:
  files:
    - build.gradle
    - "**/build.gradle"

path defines the directory that will be cached and this can include paths outside the repository directory.

Additional documentation on smarter caches with cache keys can be found on our caches documentation page.

Note: cache key support is also available for our customers who are using self-hosted runners as well.