Creating Packages¶
Here you will find directions on creating a cijoe package for your project.
cijoe packages abide by the naming convention cijoe-pkg-<project_name>,
additionally, a certain directory-layout is expected, along with helper-scripts
and utilities aka boilerplate.
To get the boiler-plate right, then the cijoe package
cijoe-pkg-example is provided; use it as a template for your own
package-creation, see the following details on how to do just that, along with
Conventions and Style information.
If you are not looking to create package, then skip that section and read the remaining sections for an overview of what a package contains and the requirements for each entity.
Package Template¶
The following will go through how to utilize the example to create a package
for your project. Start by downloading and unpacking the zip-file, if
you have wget and unzip then run:
wget https://github.com/refenv/cijoe-pkg-example/archive/refs/heads/main.zip
unzip main.zip
ls cijoe-pkg-example-main
Once retrieved then modify the following files:
setup.py
Makefile
README.rst
Go through the files, replacing all instances of “example” with the name of
your project, make certain you modify setup.py appropriately, providing the
correct author information, license, version, email, and all the other Python
package properties.
Tip
To easily match your cijoe package version to your project, then
use the same version number. For example, if you were creating a cijoe
package for fio version 3.14, then set version=3.14.0 in
setup.py. Then use the patch-version of the version number to
indicate changes to the cijoe package itself.
Tip
pip distinguishes between release and pre-release versions. A
pre-release is created by adding .devN version number where N is a
number, e.g. version=3.14.dev1.
By doing so then pip install --user cijoe-pkg-fio will install version
3.14, and pip install --user --pre cijoe-pkg-fio will install
version 3.14.dev1.
Once you have done so, then run the selftest to see that things are plumbed up correctly:
make
make selftest-view
Note
This will build and install your package as a user-local package. The
installation method of pip install --user is the preferred means of
installing cijoe and cijoe packages, thus the Makefile does so
by default.
Repositories for cijoe and cijoe packages live on GitHUB, it thus makes use of GitHUB workflows to automatically run the above-mentioned selftest and deploy to PyPi.
Conventions and Style¶
The portion of a cijoe package which consists of Bash-scripts, namely, Testcases, Hooks, and Modules, and Reference Environment(s), should aim at following Google Shell Style Guide, with the following additions and exceptions.
File-names¶
To avoid overwriting files from other packages, then follow the rule/convention below.
- Prefix filenames with the package name followed by an underscope. This applies to reference environments, modules, hooks, testplans, testsuites, and testplans.
E.g. testcases in the cijoe-pkg-example are all prefixed with example_.
Resulting in names such as example_01_minimal.sh,
example_02_output_annotation.sh, example_03_filetransfer.sh etc.
Module-separator¶
The Google Shell Style Guide defines :: as a package/module separator.
This is an exceptionally poor choice of separator for Bash-scripts as it breaks
Bash-auto-completion. Thus following the exception to the style guide:
- Use
.as module-separator.
E.g. for a module named example create functions as in the following:
example.foo() {
}
example.bar() {
}
By using . we allow for proper Bash-auto-completion and readability making
Bash scripts look more like Python and less like C++ and PHP.
Note
The :: was used by cijoe, as it was adheering to the Google
Shell Style Guide, up until version version v0.2.0. From that version
and onwards the module-separator is .. However, to avoid breaking
existing package modules, hooks, and testcases, then a “legacy” module was
added. At some point this will also be removed.
Reference Environment(s)¶
The main instrumental entity in cijoe is the Environment Definition. To assist in the creation of these, then a cijoe package provide examples aka reference environments. These serve as the name suggests, as a reference for the user of a given cijoe package when creating an Environment Definition using a what a given package provides.
This includes the variables required by Testcases, Hooks, and Modules. Reference environments double as light documentation of your package, by providing helpful comments on-top of the exported variable declarations.
Note
Reference environments are installed into the $CIJ_ENVS
directory, jump to the Interactive Shell, to see how to inspect the variables
defining cijoe. Since all cijoe packages install modules into the
same directory, this is why Conventions and Style are needed.
Example¶
An example is provided with the cijoe-pkg-example it looks like this:
Modules¶
Modules are Bash-scripts providing functions for common tasks. With cijoe the following core Bash modules are included:
- ssh, wrapping ssh for remote command execution, file-transfer etc.
- test, conventions for testcases pass/fail status etc.
- cij, output annotation, command execution mapped to ssh when defined in the environment and locally when explicitly configured to
A Bash module must have a name and a matching namespace, e.g. the following
module named example is defined in the file named example.sh, the
functions defined in the module must be prefixed with example.func_name.
A Bash module must have an environment-verification function and it
must be named <module_name>.env.
Each other function defined in the Module should call this function as the first thing it does.
Tip
Only create a module once you find that your
Testcases are repeating the same commands, and
maintaining differences is error-prone. The env comes in handy here as
it ensures that the required variables are well-defined in the
Environment Definition. The env-verification might be the only function
you need in your module.
Note
Modules are installed into the $CIJ_MODULES directory, jump to
the Interactive Shell, to see how to inspect the variables defining
cijoe. Since all cijoe packages install modules into the same
directory, this is why Conventions and Style are needed.
Example¶
An example is provided with the cijoe-pkg-example it looks like this:
For additional examples look at x, y, and z.
Hooks¶
Hooks are Bash-scripts which are intended to be executed in a manner
similar to setup/teardown in a unittest framework. The
Testplans control which entities the hooks are executed
before/after, such as individual Testcases, entire
Testsuites, and/or everything in defined in
Testplans.
Hooks come in handy for different use-cases:
- Probing the test-target for information, collecting HW information, operating system Kernel version, tool and library version etc. All the information that potentially have an impact on the thing you are doing.
- Measuring test-target side-effects, e.g. how much have been read/written to a storage device, by retrieving Smart Logs before and after running IO-intensive Testcases
- Manipulating the test-target, such as loading/unloading drivers, mounting/unmounting file-systems, resetting operating system page-caches etc.
The above are just to name a few, for an example see the following section.
Note
Hooks are installed into the $CIJ_HOOKS directory, jump to the
Interactive Shell, to see how to inspect the variables defining cijoe.
Since all cijoe packages install hooks into the same directory, this is
why Conventions and Style are needed.
Example¶
Defining the before/after portion is handled by naming the hook
accordingly. That is, for a hook named example_probe, create the files:
example_probe_enter.sh and example_probe_exit.sh as in the following
example.
Testcases¶
Testcases are Bash-scripts which utilize cijoe to invoke commands on a test-target and check whether they provide the expected return codes when doing so.
- Invoke commands by running
cij.cmd - Transfer files using
cij.push/cij.pull - Indicate success via
test.pass/test.fail
Note
Testcases are installed into the $CIJ_TESTCASES directory, jump
to the Interactive Shell, to see how to inspect the variables defining
cijoe. Since all cijoe packages install testcases into the same
directory, this is why Conventions and Style are needed.
Testsuites¶
A testsuites is a collection of Testcases. By convention
then each file ending with .suite in the testsuites directory is a
testsuite. A .suite file is a text-file consisting of a list of
testcase filenames separated by a newline.
For a testsuite to be valid then it must contain only names of files that
exists in the $CIJ_TESTCASES directory. The introspection tool
cij_tlint checks this, and it is executed when you run the selftest.
Testsuites are used in Testplans to avoid repetition when declaring the Testcases to run.
Note
Testsuites are installed into the $CIJ_TESTSUITES directory, jump
to the Interactive Shell, to see how to inspect the variables defining
cijoe. Since all cijoe packages install testsuites into the same
directory, this is why Conventions and Style are needed.
Example¶
Here is an example:
Testplans¶
A testplan describes what to run. Specifically, which Testcases/Testsuites to run, along with Hooks and the option to provide default-values for environment variables otherwise defined in the environment definition.
Note
Testplans are installed into the $CIJ_TESTPLANS directory, jump
to the Interactive Shell, to see how to inspect the variables defining
cijoe. Since all cijoe packages install testplans into the same
directory, this is why Conventions and Style are needed.
Testfiles¶
Testfiles are auxilary files that a cijoe package can provide to the user. These should be small in terms of size and primarily text-based formats. Things such as configuration-files, examples of data-formats, inputs to tools, performance requirements for known devices, device properties to verify and check for.
Note
Testfiles are installed into the $CIJ_TESTFILES directory, jump
to the Interactive Shell, to see how to inspect the variables defining
cijoe. Since all cijoe packages install testplans into the same
directory, this is why Conventions and Style are needed.