Cirrus CI pioneered an idea of directly using compute services instead of requiring users to manage their own infrastructure, configuring servers for running CI jobs, performing upgrades, etc. Instead, Cirrus CI just uses APIs of cloud providers to create virtual machines or containers on demand. This fundamental design difference has multiple benefits comparing to more traditional CIs:
- Ephemeral environment. Each Cirrus CI task starts in a fresh VM or a container without any state left by previous tasks.
- Infrastructure as code. All VM versions and container tags are specified in
.cirrus.ymlconfiguration file in your Git repository. For any revision in the past Cirrus tasks can be identically reproduced at any point in time in the future using the exact versions of VMs or container tags specified in
.cirrus.ymlat the particular revision. Just imagine how difficult it is to do a security release for a 6 months old version if your CI environment independently changes.
- Predictability and cost efficiency. Cirrus CI uses elasticity of modern clouds and creates VMs and containers on demand only when they are needed for executing Cirrus tasks and deletes them right after. Immediately scale from 0 to hundreds or thousands of parallel Cirrus tasks without a need to over provision infrastructure or constantly monitor if your team has reached maximum parallelism of your current CI plan.
What is a Persistent Worker¶
For some use cases the traditional CI setup is still useful. However, not everything is available in the cloud. For example, Apple releases new ARM-based products and there is no virtualization yet available for the new hardware. Another use case is to test the hardware itself, since not everyone is working on websites and mobile apps after all! For such use cases it makes sense to go with a traditional CI setup: install some binary on the hardware which will constantly pull for new tasks and will execute them one after another.
This is precisely what Persistent Workers for Cirrus CI are: a simple way to run Cirrus tasks beyond cloud!
First, create a persistent workers pool for your personal account or a GitHub organization (
Once a persistent worker is created, copy registration token of the pool and follow Cirrus CLI guide to configure a host that will be a persistent worker.
Once configured, target task execution on a worker by using
persistent_worker instance and matching by workers' labels:
labels filed if you want to target any worker:
By default, Cirrus CI limits task concurrency to 1 task per each worker. To schedule more tasks on a given worker, configure it's
Once done, the worker will be considered resource-aware and will be able to execute concurrently either:
- one resource-less task (a task without
- multiple resourceful tasks (a task with
resources:field) as long worker has resources available for these tasks
labels matching still takes place for both resource-less and resource-aware tasks.
So, considering a worker with the following configuration:
It will be able to concurrently execute two of these tasks:
And two of these:
By default, a persistent worker spawns all the tasks on the same host machine it's being run.
However, using the
isolation field, a persistent worker can utilize a VM or a container engine to increase the separation between tasks and to unlock the ability to use different operating systems.
To use this isolation type, install the Parallels Desktop on the persistent worker's host machine and create a base VM that will be later cloned for each task.
This base VM needs to:
- be either in a stopped or suspended state
- provide SSH access on port 22
Here's an example of a configuration that will run the task inside of a fresh macOS virtual machine created from the
big-sur-base base VM:
Once the VM spins up, persistent worker will connect to the VM's IP-address over SSH using
password credentials and run the latest agent version targeted for the
Here's an example that runs a task in a separate container with a couple directories from the host machine being accessible: