SourceCraft CI/CD workers

A worker is a virtual machine, physical server, or serverless container used to run CI/CD workflows.

All cubes within a single task run on the same worker.

The following worker types are supported:

• Cloud workers
• Serverless workers
• Self-hosted workers

Cloud workers

Cloud workers are cloud-based virtual machines provided by SourceCraft. They are the default resource for CI/CD workflows.

Cloud worker computing resources have the following configuration: 4 vCPUs, 8 GB RAM.

Serverless workers

Serverless workers are cloud-based serverless containers provided by SourceCraft. Unlike cloud workers, serverless workers spin up almost instantly as they do not wait for a VM to boot.

The following limitations apply:

• Only Yandex Container Registry is supported for Docker cubes.
• You cannot run docker or docker-compose within a cube.
• One task may include up to nine unique custom Docker images.

To run a CI/CD process on a serverless worker, specify runs_on: serverless in the workflow (workflow) or task (task) parameters.

If runs_on is not specified in task, it will default to the parameter from workflow:runs_on.

Example of running the whole workflow on a serverless worker

workflows:
  workflow-in-serverless-mode:
    runs_on: serverless
  
    tasks:
      - name: task1-in-serverless-mode
        cubes:
          - name: hello
            script:
              - echo "hello from serverless"

      - name: task2-in-serverless-mode
        cubes:
          - name: hello
            script:
              - echo "hello from serverless again"

Example of running an individual task on a serverless worker

workflows:
  workflow1:
  
    tasks:
      - name: task1-in-compute-mode
        cubes:
          - name: hello
            script:
              - echo "hello from compute"

      - name: task2-in-serverless-mode
        runs_on: serverless
        cubes:
          - name: hello
            script:
              - echo "hello from serverless"

Self-hosted workers

Self-hosted workers are users’ personal servers, both virtual and physical, on which CI/CD processes run. These processes will have access to the user server environment.

Restrictions on the amount of computing resources do not apply to self-hosted workers, and they also do not consume the total CI/CD runtime quota.

Custom workers can run tasks directly on the server or in a Kubernetes cluster. In Kubernetes mode, each task runs in a separate pod from a pool provisioned in advance.

For more information, see these pages:

• Setting up a self-hosted worker for SourceCraft
• Setting up the execution of SourceCraft CI/CD jobs in a Kubernetes cluster
• Deploying a self-hosted SourceCraft worker on a Yandex Compute Cloud VM

To run a CI/CD process on a custom self-hosted worker, specify runs_on: self-hosted in the workflow (workflow) or task (task) parameters.

If runs_on is not specified in task, the parameter from workflow:runs_on will be used by default.

Example of running the whole workflow on a self-hosted worker

workflows:
  my-awesome-workflow:
    runs_on: self-hosted
    
    tasks:
      - name: self-hosted-task
        cubes:
          - name: hello
            script:
              - echo "hello from self-hosted"

      - name: self-hosted-go-builder-task
        runs_on: [self-hosted, go-builder]
        cubes:
          - name: hello
            script:
              - echo "hello from self-hosted go builder"

In this case, tasks from my-awesome-workflow will run on any of your self-hosted workers by default unless their runs_on parameter is redefined. At the same time, self-hosted-go-builder-task will run only on the self-hosted worker assigned the go-builder tag during initialization.

Learn more about worker tags.