Alcotezt: An Alcotest Compatibility Wrapper for Tezt#
Alcotezt is a compatibility wrapper for tests originally written in Alcotest (now deprecated), enabling running them with Tezt.
Alcotezts (i.e. tests wrapped with Alcotezt) are programmed against an API that is compatible with Alcotest. But, under the hood, they are registered with Tezt. This means that they are executed in the same way as all Tezt tests are (see Running Alcotezts below for more details).
Alcotezts are typically declared through the tezt
manifest
function. This will:
Create a test library that registers the tests in the modules given as argument to
tezt
.Create a test executable
main.exe
in the test folder. The test executable links with the test library and consists of a single call to Tezt’s Test.run.Finally, link the test library with Tezt’s main entrypoint at
tezt/tests/main.exe
so that it registers all Alcotezts.
Running Alcotezts#
For a given folder $TEST_DIR
, the Alcotezts contained therein can be invoked in three ways:
By executing the
main.exe
runner binary generated by Manifest. Throughdune
, this is done withdune exec $TEST_DIR/main.exe
. This will execute the tests in$TEST_DIR
(but not in its subdirectories).Through Dune, by building the
runtest
alias in the test target folder. That is, executingdune build @$TEST_DIR/runtest
. This will execute the tests in$TEST_DIR
and its subdirectories.By executing the full Tezt test suite, via the main entrypoint at
tezt/tests/main.exe
. Through Dune, this is done withdune exec tezt/tests/main.exe
.
Execution through the test executable#
The advantage of passing through the Manifest-generated runner is that
it only depends on the tests themselves and the tested modules (and
their recursive dependencies), and so is faster to
compile. Furthermore, you can give arguments to Tezt to control
execution, e.g. passing --list
to list the tests or --verbose
to see debug output.
For example, to run the tests of src/lib_clic/test:
dune exec src/lib_clic/test/main.exe
This will execute the Alcotezts in src/lib_clic/test. To pass
arguments to Tezt, append -- <TEZT_ARGS>
to the above command:
dune exec src/lib_clic/test/main.exe -- --list
Execution through the Dune runtest
alias#
The runtest
alias can be used to execute all tests, including
Alcotezts, in a folder and its recursive sub-folders. For example, to
run all tests of protocol Alpha, run:
dune build @src/proto_alpha/runtest
On the other hand, there is no convenient way to pass arguments to the underlying tests with this method.
The Alcotezts can be dynamically disabled in the runtest
alias by
setting the environment variable RUNTEZTALIAS
to false
. For
instance, to run all the unit tests in protocol Alpha exception the
Alcotezts, you can run:
RUNTEZTALIAS=false dune build @src/proto_alpha/runtest
Execution through the Tezt main entrypoint#
Finally, Alcotezts can be executed through the Tezt main entrypoint
at tezt/tests/main.exe
. All Alcotezts are linked with this binary,
so that all system, integration and unit tests registered with either Tezt
or Alcotezt can be executed by invoking:
dune exec tezt/tests/main.exe
This is used to run all tests in the CI in a unified and load-balanced manner. This is less convenient for local use though, as there is currently no way of executing the subset of tests that corresponds to a specific package, file or folder. A forthcoming release of Tezt will ameliorate this.
Notable Differences Between Alcotest and Alcotezt#
Test naming#
First of all, some definitions:
In Alcotest, a “test suite” is a sequence of “tests” that are each composed of a sequence of “test cases”.
In Tezt, there are only “tests”, no “test suites”.
With Alcotezt, each Alcotest “test case” becomes a Tezt “test”. The
Alcotest suite, test and case name are used to form the title of
the Tezt test with following format: SUITE_NAME: TEST_NAME
(TEST_CASE_NAME)
.
Given an Alcotest consisting of a suite Suite A
that contains a
test Test a
that itself contains the test cases Test case a1
and Test case a2
. It also contains a test Test b
with the test
case Test case b1
:
let () =
Alcotest.run
"Suite A"
[
( "Test a",
[
("Test case a1", `Quick, fun () -> ...);
("Test case a2", `Quick, fun () -> ...);
] );
( "Test b",
[
("Test case b1", `Quick, fun () -> ...);
] );
]
Running it in with Alcotest produces:
Testing `Suite A'.
This run has ID `3F91T9S2'.
[OK] Test a 0 Test case a1.
[OK] Test a 1 Test case a2.
[OK] Test b 0 Test case b1.
Full test results in `/home/tezos/_build/_tests/Suite A'.
Test Successful in 0.000s. 2 tests run.
And running it with Alcotezt produces:
[17:07:42.289] [SUCCESS] (1/2) Suite A: Test a (Test case a1)
[17:07:42.289] [SUCCESS] (2/3) Suite A: Test a (Test case a2)
[17:07:42.290] [SUCCESS] (3/3) Suite A: Test b (Test case b1)
Test Output#
Alcotezt redirects Format
’s output to Tezt’s Log.debug.
To see the debug output of an Alcotezt, pass the --verbose
flag to
Tezt. See the section Running Alcotezts above for more
information on how to pass flags to Tezt when executing Alcotezts.
There is no way to redirect the output of Printf
. Consequently,
the output of Alcotezts that call this module directly cannot be
hidden.
Integration with the runtest
aliases#
Alcotezts are registered as a dependency on the runtest
alias. However, they are not executed through this alias in
the CI. Instead, they run through the Tezt main runner to enable load
balancing.