core.testrunner#

testrunner#

This is the CIJOE testrunner, it is implemented as a pytest-plugin. By doing so, then there is little to nothing to learn, assuming, that you have used pytest before.

The “special” thing is the pytest-plugin, it provides pytest command-line arguments: -c / --config and -o / --output. These behave just like the cijoe counter-part. Within, your tests, then a cijoe fixture is provided. It will give the tests access to a cijoe-instance with the given config. Quite neat :)

Thus, arbitrary testing of third-party test-suites, tools benchmarks etc. is possible with something as simple as:

def test_foo(cijoe):

    err, state = cijoe.run("execute-some-testsuite")
    assert not err

Requires the following pytest plugins for correct behaviour:

  • cijoe, fixtures providing ‘cijoe’ object and “–config” and “–output” pytest-arguments to instantiate cijoe.

  • report-log, dump testnode-status as JSON, this is consumed by ‘core.report’ to produce an overview of testcases and link them with the cijoe-captured output and auxiliary files.

Invocation of pytest is done in one of the following two ways, and controlled by args.run_local, with boolean value True / False. The default is True.

  • args.run_local: True

    This is the most common, invoking pytest locally, which in turn will be using the same config as all other cijoe-scripts. To clarify, cijoe will execute ‘pytest’ in the same environment/system where the cijoe cli was executed.

  • args.run_local: False

    This is a special-case, where a collection of pytests uses cijoe, but only the configuration, the that the pytest verify is Python code / statements / expressions, not CIJOE command executions cijoe.run(). In order ot run these remotely, then the code must be available, and then it does the following:

    • Create a copy of the currently used cijoe-config and remove the transport section if any is there

    • Transfer cijoe-config-copy to remote

    • Invoke pytest remotely using cijoe-config-copy

    • Download the testreport.log from remote

    Why this? This allows for executing pytests on a remote system which does not use cijoe.run(). Such as tests implemented in Python.

cijoe.core.scripts.testrunner.main(args, cijoe)#

Invoke the pytest + cijoe-plugin test-runner

cijoe.core.scripts.testrunner.merge_tests_facts(cijoe, path: Path)#

Merge the facts at ‘path’, where the tests’ own cijoe saved them, into the run’s. They started as the run’s, so only what the tests wrote differs; a fact the tests deleted stays in the run’s.

cijoe.core.scripts.testrunner.pytest_cmdline(args, config_path, output_path, reportlog_path)#

Contruct pytest command-line arguments given args and paths

cijoe.core.scripts.testrunner.pytest_local(args, cijoe)#

Run pytest locally, that is, execute on the same system on which cijoe-cli was executed. The cijoe config provided via ‘args.config’ is forwarded to the pytest-cijoe-plugin unmodified.

cijoe.core.scripts.testrunner.pytest_remote(args, cijoe)#

Run pytest on remote, that is, transfer config, execute pytest on the remote and transfer the testreport.log from remote to local

CLI arguments#

  • --run_local {true,false}

    Whether ‘pytest’ should be executed in same environment as ‘cijoe’ (default: True)

  • --random_order {true,false}

    Whether the tests should be run in random order. This is generally recommended, as it helps reduce inter-test dependencies and assumptions about the environment’s state (default: True)

  • --args ARGS

    Additional arguments passed verbatim to ‘pytest’. (default: None)