Configuring CI/CD in a SourceCraft repository

SourceCraft has an in-built CI/CD workflow management mechanism.

The CI/CD configuration is set up for a particular repository and stored in a file named .sourcecraft/ci.yaml.

General format of the .sourcecraft/ci.yaml configuration file:

on:
  pull_request:
    - workflows: [<list_of_workflows>]
      filter:
        source_branches: [<list_of_source_branches>]
        target_branches: [<list_of_target_branches>]
        paths: [<list_of_paths>]

  push:
    - workflows: [<list_of_workflows>]
      filter:
        branches: [<list_of_branches>]
        paths: [<list_of_paths>]
    - workflows: [<list_of_workflows>]
      filter:
        tags: [<list_of_tags>]

workflows:
  <workflow_name>:

    tasks:
      - name: <task_name>
        
        cubes:
          - name: <cube_name>
            image: <Docker_image_path>
            script:
              - <executed_script>
...

Tip

The .sourcecraft/ci.yaml file supports file structure output as well as search for declarations and element usages. For more information, see Navigating and searching by code in SourceCraft.

You can either use nested structure for configuration elements or mark them up as separate sections.

Example of CI/CD configuration marked up as separate sections
on:
  pull_request:
    - workflows: my-test-workflow
      filter:
        source_branches: ["**", "!test**"]
        target_branches: "main"

workflows:
  my-test-workflow:
    tasks:
      - my-test-task

tasks:
  - name: my-test-task
    cubes:
      - name: my-test-cube
        image: docker.io/library/node
        script:
          - echo Hello, world!

The configuration file supports secrets. For details, see Using the value of a secret in CI/CD.

Warning

Support for storing configurations for CI/CD, approval rules, and branch policies in a single .src.ci.yaml file at the repository root will soon be discontinued. Use the separate .sourcecraft/ci.yaml, .sourcecraft/review.yaml, and .sourcecraft/branches.yaml files.

See the templates repository in SourceCraft for the full specification of the .sourcecraft/ci.yaml file.

CI/CD workflows are run by trigger events. Such events may include pushing changes to a remote repository branch or creating a pull request.

You can configure different workflows for different events. You can also configure triggers for specific branches or paths in the repository.

For a description of the configuration elements, see the CI/CD reference.

You can configure CI/CD when creating a new repository or in an existing one.

Configuring CI/CD when creating a repository

  1. Open the SourceCraft home page.

  2. In the left-hand panel, click Create repository.

  3. In the window that opens, set the repository parameters.

  4. Under Configuration, select Add a CI/CD template.

  5. Select the CI/CD configuration template from the YAML drop-down list:

    • Docker
    • HelloWorld
    • Java
    • NodeJS
    • Python

    For the contents of the templates, see Examples.

  6. Click Create repository.

    The root of the created repository will contain the .sourcecraft/ci.yaml file with the CI/CD configuration template.

Configuring CI/CD in an existing repository

  1. Clone the repository:

    1. Install Git.

    2. Open the SourceCraft home page.

    3. On the Home tab, under Your craftspace, navigate to Repositories and select your repository.

    4. In the top-right corner, click Clone.

    5. Depending on your connection method, copy the link for cloning the repository.

    6. In the terminal, run this command:

      git clone <link_for_cloning_repository>
      
    7. Go to your cloned repository:

      cd <repository_name>
      
  2. Generate a CI/CD .sourcecraft/ci.yaml configuration file as described in the CI/CD reference or use the templates in Examples.

  3. Add the CI/CD configuration file to the git index, commit, and push the changes to the main branch:

    git add .sourcecraft/ci.yaml
    git commit -m "Added CI/CD configuration"
    git push -u origin main
    

Checking CI/CD workflow execution

  1. Depending on the settings specified in .sourcecraft/ci.yaml, run a trigger event.

  2. Check CI/CD workflow execution:

    1. Under Code on the repository page, go to CI/CD.
    2. Select a running workflow.
    3. The page that opens will display all workflow tasks, cubes (task steps), as well as statuses and execution results.

Examples

# This basic workflow 
# will help you get started with SourceCraft CI/CD

# Configuring a trigger event to execute a workflow
on:
  # Executing a workflow triggered by sending changes to the remote repository's main branch
  # or by creating a pull request for the main branch
  pull_request:
    - workflows: [sample-workflow]
      filter:
        source_branches: ["**", "!test**"]
        target_branches: "main"

  push:
    - workflows: [sample-workflow]
      filter:
        branches: ["main"]

workflows:
  sample-workflow:
    tasks:
      - name: sample-task
      
        cubes:
          # Starting a set of commands 
          - name: sample-cube1
            image: docker.io/library/node
            script:
              - echo Hello, world!
           
          - name: sample-cube2
            script:
              - echo Add other cubes to build,
              - echo test, and deploy your project.
on:
  push:
    - workflows: build-workflow 
      filter: 
        branches: ["main"]

workflows:
  build-workflow:
    tasks:
      - build-task

tasks:
  - name: build-task
    env:
        IMAGE_URI: dockerhub_account/hello-world
    cubes:
      - name: docker-build
        script:
          - docker build . --file Dockerfile --tag $IMAGE_URI:$(date +%s)
      # Optionally, send a Docker image to a registry
      # Provide to the cube the credentials for authentication in the registry,
      # e.g., DockerHub or Yandex Container Registry
      # - name: push-dockerhub
      #   env:
      #     USER: username
      #     PAT: personal_access_token
      #   script:
      #     - echo $PAT | docker login --username $USER --password-stdin
      #     - docker push $IMAGE_URI

To test the CI\CD process, send a commit with Dockerfile to the main branch:

FROM alpine
CMD ["echo", "Hello World!!"]
on:
  push:
    - workflows: [sample-workflow]
      filter:
        branches: ["main"]

workflows:
  sample-workflow:
    tasks:
      - name: sample-task
        cubes:
          - name: sample-cube1
            image: docker.io/library/python
            script:
              - python -m pip install --upgrade pip
              - if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

Add data specific to testing Python applications to the .gitignore file.

.gitignore example
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
#   For a library or package, you might want to ignore these files since the code is
#   intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# UV
#   Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
#   This is especially recommended for binary packages to ensure reproducibility, and is more
#   commonly ignored for libraries.
#uv.lock

# poetry
#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
#   This is especially recommended for binary packages to ensure reproducibility, and is more
#   commonly ignored for libraries.
#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
#   in version control.
#   https://pdm.fming.dev/latest/usage/project/#working-with-version-control
.pdm.toml
.pdm-python
.pdm-build/

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
#  and can be added to the global gitignore or merged into this file. For a more nuclear
#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/

# PyPI configuration file
.pypirc

To test the CI\CD process, send a commit with a main.py file to the main branch:

print('Hello SourceCraft')
print('Hello SourceCraft')

for i in range(10):
    print(i)

# input 
hi = input()
print(hi)
on:
  push:
    - workflows: build-workflow 
      filter: 
        branches: ["main"]

workflows:
  build-workflow:
    tasks:
      - build-task

tasks:
  - name: build-task
    cubes:
      - name: build-npm
        image: docker.io/library/node
        script:
          - npm ci
          - npm run build --if-present
          - npm test

To test the CI\CD process, send a commit with the following files to the main branch:

  • index.js:

    //index.js
    
    function helloNpm() {
        return "hello SourceCraft CI/CD"
    }
    
    module.exports = helloNpm
    
  • package.json:

    {
        "name": "sourcecraft-samples-nodejs",
        "version": "1.0.0",
        "description": "Sample for SourceCraft CI/CD",
        "license": "MIT",
        "author": "SourceCraft Team",
        "type": "commonjs",
        "main": "index.js",
        "scripts": {
          "test": "echo \"test passed\""
        }
    }
    
  • package-lock.json:

    {
        "name": "sourcecraft-samples-nodejs",
        "version": "1.0.0",
        "lockfileVersion": 3,
        "requires": true,
        "packages": {
          "": {
            "name": "sourcecraft-samples-nodejs",
            "version": "1.0.0",
            "license": "MIT"
          }
        }
    }
    
# This workflow will build a Java project using Maven
on:
  push:
    - workflows: publish-package-workflow
      filter:
        branches: ["main"]

workflows:
  publish-package-workflow:
    settings:
      max_cube_duration: 10m
    tasks:
      - build-publish-task

tasks:
  - name: build-publish-task
    cubes:
      - name: setup-jdk-maven
        script:
          - sudo apt install openjdk-21-jdk -y
          - sudo apt install maven -y
          - java --version
          - mvn --version
      - name: test
        script:
          - echo 'test'
      - name: package
        script:
          - mvn package
        # Optionally, export the artifact named
        # `sourcecraft-sample-maven-1.0-SNAPSHOT.jar` from the target directory.
        # You can download this artifact from the cube in the CI/CD section
        #artifacts: 
        #  paths:
        #    - target/sourcecraft-sample-maven-1.0-SNAPSHOT.jar

Add data specific to testing Java applications to the .gitignore file.

.gitignore example
##############################
## Java
##############################
.mtj.tmp/
*.class
*.jar
*.war
*.ear
*.nar
hs_err_pid*
replay_pid*

##############################
## Maven
##############################
target/
pom.xml.tag
pom.xml.releaseBackup
pom.xml.versionsBackup
pom.xml.next
pom.xml.bak
release.properties
dependency-reduced-pom.xml
buildNumber.properties
.mvn/timing.properties
.mvn/wrapper/maven-wrapper.jar

##############################
## Gradle
##############################
bin/
build/
.gradle
.gradletasknamecache
gradle-app.setting
!gradle-wrapper.jar

##############################
## IntelliJ
##############################
out/
.idea/
.idea_modules/
*.iml
*.ipr
*.iws

##############################
## Eclipse
##############################
.settings/
bin/
tmp/
.metadata
.classpath
.project
*.tmp
*.bak
*.swp
*~.nib
local.properties
.loadpath
.factorypath

##############################
## NetBeans
##############################
nbproject/private/
build/
nbbuild/
dist/
nbdist/
nbactions.xml
nb-configuration.xml

##############################
## Visual Studio Code
##############################
.vscode/
.code-workspace

##############################
## OS X
##############################
.DS_Store

##############################
## Miscellaneous
##############################
*.log

To test the CI\CD process, send a commit with the following files to the main branch:

  • pom.xml:

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
      <modelVersion>4.0.0</modelVersion>
    
      <groupId>com.mycompany.app</groupId>
      <artifactId>sourcecraft-sample-maven</artifactId>
      <version>1.0-SNAPSHOT</version>
    
      <name>bondarevsky-maven</name>
      <!-- FIXME change it to the project's website -->
      <url>http://www.example.com</url>
    
      <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.release>17</maven.compiler.release>
      </properties>
    
      <dependencyManagement>
        <dependencies>
          <dependency>
            <groupId>org.junit</groupId>
            <artifactId>junit-bom</artifactId>
            <version>5.11.0</version>
            <type>pom</type>
            <scope>import</scope>
          </dependency>
        </dependencies>
      </dependencyManagement>
    
      <dependencies>
        <dependency>
          <groupId>org.junit.jupiter</groupId>
          <artifactId>junit-jupiter-api</artifactId>
          <scope>test</scope>
        </dependency>
        <!-- Optionally: parameterized tests support -->
        <dependency>
          <groupId>org.junit.jupiter</groupId>
          <artifactId>junit-jupiter-params</artifactId>
          <scope>test</scope>
        </dependency>
      </dependencies>
    
      <build>
        <pluginManagement><!-- lock down plugins versions to avoid using Maven defaults (may be moved to parent pom) -->
          <plugins>
            <!-- clean lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#clean_Lifecycle -->
            <plugin>
              <artifactId>maven-clean-plugin</artifactId>
              <version>3.4.0</version>
            </plugin>
            <!-- default lifecycle, jar packaging: see https://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging -->
            <plugin>
              <artifactId>maven-resources-plugin</artifactId>
              <version>3.3.1</version>
            </plugin>
            <plugin>
              <artifactId>maven-compiler-plugin</artifactId>
              <version>3.13.0</version>
            </plugin>
            <plugin>
              <artifactId>maven-surefire-plugin</artifactId>
              <version>3.3.0</version>
            </plugin>
            <plugin>
              <artifactId>maven-jar-plugin</artifactId>
              <version>3.4.2</version>
            </plugin>
            <plugin>
              <artifactId>maven-install-plugin</artifactId>
              <version>3.1.2</version>
            </plugin>
            <plugin>
              <artifactId>maven-deploy-plugin</artifactId>
              <version>3.1.2</version>
            </plugin>
            <!-- site lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#site_Lifecycle -->
            <plugin>
              <artifactId>maven-site-plugin</artifactId>
              <version>3.12.1</version>
            </plugin>
            <plugin>
              <artifactId>maven-project-info-reports-plugin</artifactId>
              <version>3.6.1</version>
            </plugin>
          </plugins>
        </pluginManagement>
      </build>
    </project>
    
  • src/test/java/com/mycompany/app/AppTest.java:

    package com.mycompany.app;
    
    import static org.junit.jupiter.api.Assertions.assertTrue;
    
    import org.junit.jupiter.api.Test;
    
    /**
     * Unit test for simple App.
     */
    public class AppTest {
    
        /**
         * Rigorous Test :-)
         */
        @Test
        public void shouldAnswerWithTrue() {
            assertTrue(true);
        }
    }
    
  • src/main/java/com/mycompany/app/App.java:

    package com.mycompany.app;
    
    /**
     * Hello world!
     */
    public class App {
        public static void main(String[] args) {
            System.out.println("Hello World!");
        }
    }
    

See also