Workflows#

Workflows enable the organized execution of commands and scripts. After execution, a report is generated, containing the status and embedded documentation of the workflow and scripts in a self-contained format. To run the workflow produced by cijoe --example, use the following command:

# Run the workflow
cijoe cijoe-example-core.default/cijoe-workflow.yaml \
 --config cijoe-example-core.default/cijoe-config.toml

The command will execute all the steps in the workflow. To run a subset of steps, you can specify the step name(s) as arguments to the cijoe tool, similar to how targets are specified in a Makefile:

# Run a subset of the steps in the workflow
cijoe cijoe-example-core.default/cijoe-workflow.yaml \
 --config cijoe-example-core.default/cijoe-config.toml \
 inline_commands

With the above, only the step named builtin_script will be executed. This becomes even more useful when utilizing cijoe bash completions.

There are a couple of workflow-specific options. See the Configs section for reference.

Content Overview#

Let’s take a look at what the workflow file produced by cijoe --example core.default looks like:

---
doc: |
  This is a workflow file, it serves as an example on how to run commands and
  scripts, the structure intentionally mimics that of GitHUB actions, however,
  the keys you see here are all there is.

  Running commands, as you can see below, looks just like running commands in a
  GitHUB Workflow

  * Add the 'run' key with a value of multi-line string

  Using scripts, it is similar to that of a GitHUB action

  * Add the 'uses' key with the name of the script
    - packaged scripts have a namespaced name e.g. "my_pkg.my_script"
    - non-packaged do not e.g. "my_script"
  * Add the 'with' key providing arguments to the script

  The commands and the scripts are passed an instance of cijoe which they can
  use to call run() / get() / put(), with an output-directory matching the
  current step. This is it, end of story.

steps:
- name: inline_commands
  run: |
    cat /proc/cpuinfo
    hostname

- name: script_with_args
  uses: core.cmdrunner
  with:
    commands:
    - cat /proc/cpuinfo
    - hostname

- name: example_script
  uses: core.example_script_default
  with:
    repeat: 2

#
# Uncomment the lines below to include the non-packaged script
# 'cijoe-script.py', from the directory where you ran 'cijoe --example'
#
#- name: custom_script
#  uses: cijoe-script

At a first glance, then it might feel a bit similar to GitHub Actions workflow, but dramatically simpler since:

  • There are no logic operators

  • There is simple variable substitution using

    • Values from configuration file

    • Values from environment variables on initiator

  • Minimal amount of “magic” keys

    • doc: Describe what the workflow does using multi-line plain-text

    • steps: Ordered list of scripts, to inline-commands, to run

Descriptions of the content is provided in the following subsections.

Steps#

Although cijoe aims to be simple, with minimal “magic” and a low learning curve, there is some yaml-magic involved in the workflow steps. A step can take one of two forms: either as Inline Commands or as Steps with Scripts.

Both forms require that a step must have a name. This allows subsets of steps to be executed via the cijoe command-line tool. When naming steps, follow these conventions:

  • Letters: a-z

  • Numbers: 0-9

  • Special characters: - and _

  • Must be lowercase

  • Must not start with a number

In short, use the typical lowercase identifier convention.

Inline Commands#

A step with inline commands take the form:

- name: inline_commands
  run: |
    cat /proc/cpuinfo
    hostname

Each line in a multi-line string is executed. It is implemented as a call to cijoe.run(command). Thus, the above notation for inline commands turn into execution of functions in the cijoe Python module:

cijoe.run("cat /proc/cpuinfo")
cijoe.run("hostname")

Note

This is implemented in cijoe as “syntactic-sugar” for running the built-in script core.cmdrunner. Thus, have a look at Steps with Scripts to see what this unfolds as.

Steps with Scripts#

When a step runs a script, then you give it a name and you tell it

- name: script_with_args
  uses: core.cmdrunner
  with:
    commands:
    - cat /proc/cpuinfo
    - hostname

Take note of the “magic” keys:

uses

Name of the script to run, without the .py extension. Packaged scripts include a prefix, such as core. or linux.. As in the example above where the script cmdrunner.py from the core package is used (core.cmdrunner).

with

Everything under this key is passed to the script’s entry function: main(args, cijoe) in the args argument.

Linting#

When you write a workflow yourself it can be nice to check whether it is valid without running it. You can do so by running:

# Check format of workflow and verify existance of the scripts used
cijoe --integrity-check \
	--config cijoe-example-core.default/cijoe-config.toml \
	cijoe-example-core.default/cijoe-workflow.yaml