Alcotezt: An Alcotest Compatibility Wrapper for Tezt¶

Alcotezt is a compatibility wrapper for tests originally written in Alcotest (now deprecated), enabling running them with Tezt.

Alcotezts (i.e. tests wrapped with Alcotezt) are programmed against an API that is compatible with Alcotest. But, under the hood, they are registered with Tezt. This means that they are executed in the same way as all Tezt tests are (see Running Alcotezts below for more details).

Alcotezts are typically declared through the tezt manifest function. This will:

Create a test library that registers the tests in the modules given as argument to tezt.

Create a test executable main.exe in the test folder. The test executable links with the test library and consists of a single call to Tezt’s Test.run.

Finally, link the test library with Tezt’s main entrypoint at tezt/tests/main.exe so that it registers all Alcotezts.

Running Alcotezts¶

For a given folder $TEST_DIR, the Alcotezts contained therein can be invoked in three ways:

By executing the main.exe runner binary generated by Manifest. Through dune, this is done with dune exec $TEST_DIR/main.exe. This will execute the tests in $TEST_DIR (but not in its subdirectories).

Through Dune, by building the runtest alias in the test target folder. That is, executing dune build @$TEST_DIR/runtest. This will execute the tests in $TEST_DIR and its subdirectories.

By executing the full Tezt test suite, via the main entrypoint at tezt/tests/main.exe. Through Dune, this is done with dune exec tezt/tests/main.exe.

Execution through the test executable¶

The advantage of passing through the Manifest-generated runner is that it only depends on the tests themselves and the tested modules (and their recursive dependencies), and so is faster to compile. Furthermore, you can give arguments to Tezt to control execution, e.g. passing --list to list the tests or --verbose to see debug output.

For example, to run the tests of src/lib_clic/test:

dune exec src/lib_clic/test/main.exe

This will execute the Alcotezts in src/lib_clic/test. To pass arguments to Tezt, append -- <TEZT_ARGS> to the above command:

dune exec src/lib_clic/test/main.exe -- --list

Execution through the Dune `runtest` alias¶

The runtest alias can be used to execute all tests, including Alcotezts, in a folder and its recursive sub-folders. For example, to run all tests of protocol Alpha, run:

dune build @src/proto_alpha/runtest

On the other hand, there is no convenient way to pass arguments to the underlying tests with this method.

The Alcotezts can be dynamically disabled in the runtest alias by setting the environment variable RUNTEZTALIAS to false. For instance, to run all the unit tests in protocol Alpha exception the Alcotezts, you can run:

RUNTEZTALIAS=false dune build @src/proto_alpha/runtest

Execution through the Tezt main entrypoint¶

Finally, Alcotezts can be executed through the Tezt main entrypoint at tezt/tests/main.exe. All Alcotezts are linked with this binary, so that all system, integration and unit tests registered with either Tezt or Alcotezt can be executed by invoking:

dune exec tezt/tests/main.exe

This is used to run all tests in the CI in a unified and load-balanced manner. This is less convenient for local use though, as there is currently no way of executing the subset of tests that corresponds to a specific package, file or folder. A forthcoming release of Tezt will ameliorate this.

Notable Differences Between Alcotest and Alcotezt¶

Test naming¶

First of all, some definitions:

In Alcotest, a “test suite” is a sequence of “tests” that are each composed of a sequence of “test cases”.

In Tezt, there are only “tests”, no “test suites”.

With Alcotezt, each Alcotest “test case” becomes a Tezt “test”. The Alcotest suite, test and case name are used to form the title of the Tezt test with following format: SUITE_NAME: TEST_NAME (TEST_CASE_NAME).

Given an Alcotest consisting of a suite Suite A that contains a test Test a that itself contains the test cases Test case a1 and Test case a2. It also contains a test Test b with the test case Test case b1:

let () =
  Alcotest.run
    "Suite A"
    [
      ( "Test a",
        [
          ("Test case a1", `Quick, fun () -> ...);
          ("Test case a2", `Quick, fun () -> ...);
        ] );
      ( "Test b",
        [
          ("Test case b1", `Quick, fun () -> ...);
        ] );
    ]

Running it in with Alcotest produces:

Testing `Suite A'.
This run has ID `3F91T9S2'.

  [OK]          Test a          0   Test case a1.
  [OK]          Test a          1   Test case a2.
  [OK]          Test b          0   Test case b1.

Full test results in `/home/tezos/_build/_tests/Suite A'.
Test Successful in 0.000s. 2 tests run.

And running it with Alcotezt produces:

[17:07:42.289] [SUCCESS] (1/2) Suite A: Test a (Test case a1)
[17:07:42.289] [SUCCESS] (2/3) Suite A: Test a (Test case a2)
[17:07:42.290] [SUCCESS] (3/3) Suite A: Test b (Test case b1)

Test Output¶

Alcotezt redirects Format’s output to Tezt’s Log.debug. To see the debug output of an Alcotezt, pass the --verbose flag to Tezt. See the section Running Alcotezts above for more information on how to pass flags to Tezt when executing Alcotezts.

There is no way to redirect the output of Printf. Consequently, the output of Alcotezts that call this module directly cannot be hidden.

Integration with the `runtest` aliases¶

Alcotezts are registered as a dependency on the runtest alias. However, they are not executed through this alias in the CI. Instead, they run through the Tezt main runner to enable load balancing.