Third-party secrets are now supported in Bitbucket Pipelines

Bitbucket Pipelines is excited to announce support for CI/CD third-party secrets – a powerful new feature that enables you to seamlessly retrieve secrets directly from your preferred secret store. This enhancement ensures your workflows remain efficient and secure. Now, you can set up your own integration and customize the utilization of your secrets to suit your needs.

Moving beyond workspace and repository variables

Bitbucket’s third-party secrets integration allows for secure injection of variables into a build step’s execution environment. Unlike user-defined workspace and repository variables, these are never stored within the Pipelines platform – they are accessed from your third-party vault only when a properly configured pipeline needs them during a run. Logs automatically mask these variables to prevent exposure. Furthermore, on self-hosted runners, the variables remain confined to the customers own network which reduces the risk of external access.

With this integration, you can customize the use of your variables; for instance, it is possible to configure your integration so that each repository in your workspace is only able to access a specific set of secrets. This capability significantly enhances control and security across your projects.

How does the third-party secrets integration work

When integrating your secret provider with Bitbucket Pipelines, you need to create an integration that serves as a bridge between your specific secret store and the runner. This integration is responsible for verifying the request coming from Pipelines, and mapping your secrets into the response to send back to the build. While the configuration process varies slightly for cloud runners compared to self-hosted runners, the overall flow is the same as what is shown in the diagram below.

1. Step scheduling with OIDC token: When a step requiring secrets is scheduled, Bitbucket Pipelines generates an OIDC token and provides it to the runner as part of the step execution.
2. Runner requests secrets: The runner sends a GET request to the secret’s integration endpoint including the OIDC token in the Authorization header.
3. OIDC validation: The secrets integration service validates the OpenID Connect (OIDC) token by retrieving the JSON Web Key Set (JWKS) and issuer through Bitbucket Pipelines endpoints to verify its authenticity.
4. Fetching and returning secrets: Upon successful validation, the secrets integration retrieves the necessary secrets from the secret store.
5. Filtering the secrets: This process involves reviewing the token’s claims and securely returning the secrets to the runner. At this point, the secret provider has the capability to customize the secrets returned based on the claims if the user chooses to do so.
6. Completion of the step: The runner uses the retrieved secrets to execute the pipeline step.

Building your secrets integration

The provider is a service that must serve an endpoint where the pipelines will make POST requests. The response format is a flat JSON key-value map, where both keys and values are strings.

Pipelines will send an OIDC JWT token in the Authorization header, which, when unencrypted, contains several claims that the provider can utilize to determine which secrets to return.

We have developed a demo secret provider integration that you can reference for guidance when creating your own integration.

Configuring your pipeline to use secrets integration

Configuring your pipeline to utilize secrets from your third-party secrets integration is easy. Follow the steps outlined below to establish the integration endpoint for your self-hosted or cloud runners, and then configure your pipeline to utilize the secrets from your designated provider.

Self Hosted

To configure a runner, use the SECRET_PROVIDER_URI variable to set the endpoint for retrieving third-party secret variables and then start the runner. The following examples show how to configure some of the self-hosted runners with the endpoint for the secret: http://secret.provider/endpoint.

Docker

Set SECRET_PROVIDER_URI environment variable when starting the container:

docker container run ... -e SECRET_PROVIDER_URI="http://secret.provider/endpoint" ...

Shell

Add –secretProviderUri property when calling start.sh script:

./start.sh ... --secretProviderUri "http://secret.provider/endpoint" ...

Powershell

Add -secretProviderUri property when calling start.ps1 script:

.\start ... -secretProviderUri "http://secret.provider/endpoint" ..

Cloud

In the Bitbucket workspace you’d like to use with this provider, enter the URI of your secret provider that will serve the secrets into the Secret provider endpoint field, and select the Add button.

Enable OIDC in your pipeline

In order to be authorized for connection with the secrets integration, you must configure your pipeline to utilize OIDC for the steps where you wish to use the third-party secrets, as in the example below.

   pipelines:
   default:
   - step:
     name: Default
     oidc: true               # Required to get the token to pass to the Secret Provider

For more information on how to integrate with your preferred secret provider, visit Variables and secrets documentation or reach out to us on the Bitbucket Community.