testsuite.driver

Submodules

Attributes

ResultQueue

Classes

`ResultQueueItem`	Information to integrate a test result in a testsuite report.
`TestDriver`	Testsuite Driver.
`BasicTestDriver`	Testsuite Driver.

Package Contents

class testsuite.driver.ResultQueueItem

Information to integrate a test result in a testsuite report.

Test drivers create test results. They travel from there to the testsuite final report through various queues. This class gathers all the information needed for the various stages of this pipeline.

test_name: str: Name of the test that created this test result.

result: e3.testsuite.result.TestResultSummary: Summary for this test result.

filename: str: Name of the file that contains test result data.

traceback: List[str]

Stack trace for this result’s push time.

This stack trace corresponds to the code that led the TestResult instance to be included in the testsuite report.

gaia_results: e3.testsuite.report.gaia.GAIAResultFiles | None

GAIA files for this result.

This is None if no GAIA report is requested.

testsuite.driver.ResultQueue

class testsuite.driver.TestDriver(env: e3.env.Env, test_env: Dict[str, Any])

Bases: object

Testsuite Driver.

All drivers declared in a testsuite should inherit from this class

Fore: Any

Style: Any

test_env_filename: str

env: e3.env.Env

testsuite_options: argparse.Namespace

test_env: Dict[str, Any]

test_name: str

result: e3.testsuite.result.TestResult

result_queue: ResultQueue = []: Queue of test results that this driver plans to integrate to the testsuite report.

execution_started: bool = False: Whether the execution of at least one fragment for this test driver has started.

pending_fragments: set[str]: Set of UIDs for fragments whose execution has not yet completed for this test driver.

push_result(result: e3.testsuite.result.TestResult | None = None) → None

Push a result to the testsuite.

This method should be called to push results to the testsuite report.

Parameters:: result – A TestResult object to push. If None push the current test result.

add_fragment(dag: e3.collection.dag.DAG, name: str, fun: e3.testsuite.fragment.FragmentCallback | None = None, after: List[str] | None = None) → None

Add a test fragment.

This function is a helper to define test workflows that do not introduce dependencies to other tests. For more complex operation use directly add_vertex method from the dag. See add_test method.

Parameters:

dag – DAG containing test fragments.
name – Name of the fragment.
fun – Callable that takes two positional arguments: a mapping from fragment names to return values for already executed fragments, and a slot ID. If None looks for a method inside this class called name.
after – List of fragment names that should be executed before this one.

working_dir(*args: str) → str: Build a filename in the test working directory.

test_dir(*args: str) → str: Build a filename in the testcase directory.

abstractmethod add_test(dag: e3.collection.dag.DAG) → None

Create the test workflow.

Amend a DAG with the test fragments that should be executed along with their dependencies. See BasicTestDriver for an example of workflow.

property working_dir_cleanup_enabled: bool

Return whether test drivers should cleanup their working directory.

Unless this returns False, test drivers should delete their working directory when the test has completed, so that temporaries for the whole testsuite are removed incrementally. This is necessary to avoid creating huge temporary directories when executing big testsuites.

class testsuite.driver.BasicTestDriver(env: e3.env.Env, test_env: Dict[str, Any])

Bases: TestDriver

Testsuite Driver.

All drivers declared in a testsuite should inherit from this class

add_test(dag: e3.collection.dag.DAG) → None

Create a standard test workflow.

set up -> run -> analyze -> tear_down in which set up and tear_down are optional.

set_up(prev: Dict[str, Any], slot: int) → None: Execute operations before executing a test.

tear_up(prev: Dict[str, Any], slot: int) → None: Backwards-compatible name for the “set_up” method.

tear_down(prev: Dict[str, Any], slot: int) → None: Execute operations once a test is finished.

abstractmethod run(prev: Dict[str, Any], slot: int) → None: Execute a test.

abstractmethod analyze(prev: Dict[str, Any], slot: int) → None: Compute the test result.