Docker Builder on VM
Docker Builder VM¶
"Docker Builder" tasks are a way to build and publish Docker Images to Docker Registries of your choice using a VM as build environment.
In essence, a
docker_builder is basically a
task that is executed in a VM with pre-installed Docker.
docker_builder can be defined the same way as a
Leveraging features such as Task Dependencies, Conditional Execution and Encrypted Variables with a Docker Builder can help building relatively complex pipelines. It can also be used to execute builds which need special privileges.
In the example below, a
docker_builder will be only executed on a tag creation, once both
tasks have finished successfully:
test_task: ... lint_task: ... docker_builder: only_if: $CIRRUS_TAG != '' depends_on: - test - lint env: DOCKER_USERNAME: ENCRYPTED[...] DOCKER_PASSWORD: ENCRYPTED[...] build_script: docker build --tag myrepo/foo:$CIRRUS_TAG . login_script: docker login --username $DOCKER_USERNAME --password $DOCKER_PASSWORD push_script: docker push myrepo/foo:$CIRRUS_TAG
For more examples please check how we use Docker Builder to build and publish Cirrus CI's Docker Images for Android.
Docker Builder VM has QEMU pre-installed and is able to execute multi-arch builds via
Add the following
setup_script to enable
buildx and then use
docker buildx build instead of the regular
docker_builder: setup_script: - docker buildx create --name multibuilder - docker buildx use multibuilder - docker buildx inspect --bootstrap build_script: docker buildx build --platform linux/amd64,linux/arm64 --tag myrepo/foo:$CIRRUS_TAG .
For your convenience, a Docker Builder VM has some common packages pre-installed:
- AWS CLI
- Docker Compose
- Ruby with Bundler
Under the hood¶
Under the hood a simple integration with Google Compute Engine
is used and basically
docker_builder is a syntactic sugar for the following
You can check Packer templates of the VM image in
Docker has the
--cache-from flag which allows using a previously built image as a cache source. This way only changed
layers will be rebuilt which can drastically improve performance of the
build_script. Here is a snippet that uses
# pull an image if available docker pull myrepo/foo:latest || true docker build --cache-from myrepo/foo:latest \ --tag myrepo/foo:$CIRRUS_TAG \ --tag myrepo/foo:latest .
Dockerfile as a CI environment¶
With Docker Builder there is no need to build and push custom containers so they can be used as an environment to run CI tasks in.
Cirrus CI can do it for you! Just declare a path to a
Dockerfile with the
dockerfile field for your
declarations in your
.cirrus.yml like this:
Cirrus CI will build a container and cache the resulting image based on
Dockerfile’s content. On the next build,
Cirrus CI will check if a container was already built, and if so, Cirrus CI will instantly start a CI task using the cached image.
Under the hood, for every
Dockerfile that is needed to be built, Cirrus CI will create a Docker Builder task as a dependency.
You will see such
build_docker_image_HASH tasks in the UI.
Danger of using
Cirrus only includes files directly added or copied into a container image in the cache key. But Cirrus is not recursively waking contents of folders that are being included into the image. This means that for a public repository a potential bad actor can create a PR with malicious scripts included into a container, wait for it to be cached and then reset the PR, so it looks harmless.
Please try to only
COPY files by full path, e.g.:
Using with private GKE clusters
gke_container you first need to create a VM with Docker installed within your GCP project.
This image will be used to perform building of Docker images for caching. Once this image is available, for example, by
MY_DOCKER_VM name, you can use it like this:
gke_container: dockerfile: .ci/Dockerfile builder_image_name: MY_DOCKER_VM cluster_name: cirrus-ci-cluster zone: us-central1-a namespace: default
Please make sure your buidler image has
gcloud configured as a credential helper.
If your builder image is stored in another project you can also specify it by using
By default, Cirrus CI assumes builder image is stored within the same project as the GKE cluster.
Using with private EKS clusters
eks_container you need three things:
- Either create an AMI with Docker installed or use one like ECS-optimized AMIa. For example,
- Create a role which has
AmazonEC2ContainerRegistryFullAccesspolicy. For example,
cirrus-cacherepository in your Elastic Container registry and make sure user that
aws_credentialsare associated with has
ecr:DescribeImagesaccess to it.
Once all of the above requirement are met you can configure
eks_container like this:
eks_container: region: us-east-2 cluster_name: my-company-arm-cluster dockerfile: .ci/Dockerfile builder_image: MY_DOCKER_AMI builder_role: cirrus-builder # role for builder instance profile builder_instance_type: c7g.xlarge # should match the architecture below builder_subnet_id: ... # optional, default subnet from your default VPC is used by default architecture: arm64 # default is amd64
This will make Cirrus CI to check whether
cirrus-cache repository in
us-east-2 region contains a precached image
.ci/Dockerfile of this repository.
Docker builders also support building Windows Docker containers - use the