Alcotezt: An Alcotest Compatibility Wrapper for Tezt¶
Alcotezts (i.e. tests wrapped with Alcotezt) are programmed against an API that is compatible with Alcotest. But, under the hood, they are registered with Tezt. This means that they are executed in the same way as all Tezt tests are (see Running Alcotezts below for more details).
Alcotezts are typically declared through the
function. This will:
Create a test library that registers the tests in the modules given as argument to
Create a test executable
main.exein the test folder. The test executable links with the test library and consists of a single call to Tezt’s Test.run.
Finally, link the test library with Tezt’s main entrypoint at
tezt/tests/main.exeso that it registers all Alcotezts.
For a given folder
$TEST_DIR, the Alcotezts contained therein can be invoked in three ways:
By executing the
main.exerunner binary generated by Manifest. Through
dune, this is done with
dune exec $TEST_DIR/main.exe. This will execute the tests in
$TEST_DIR(but not in its subdirectories).
Through Dune, by building the
runtestalias in the test target folder. That is, executing
dune build @$TEST_DIR/runtest. This will execute the tests in
$TEST_DIRand its subdirectories.
By executing the full Tezt test suite, via the main entrypoint at
tezt/tests/main.exe. Through Dune, this is done with
dune exec tezt/tests/main.exe.
Execution through the test executable¶
The advantage of passing through the Manifest-generated runner is that
it only depends on the tests themselves and the tested modules (and
their recursive dependencies), and so is faster to
compile. Furthermore, you can give arguments to Tezt to control
execution, e.g. passing
--list to list the tests or
to see debug output.
For example, to run the tests of src/lib_clic/test:
dune exec src/lib_clic/test/main.exe
This will execute the Alcotezts in src/lib_clic/test. To pass
arguments to Tezt, append
-- <TEZT_ARGS> to the above command:
dune exec src/lib_clic/test/main.exe -- --list
Execution through the Dune
runtest alias can be used to execute all tests, including
Alcotezts, in a folder and its recursive sub-folders. For example, to
run all tests of protocol Alpha, run:
dune build @src/proto_alpha/runtest
On the other hand, there is no convenient way to pass arguments to the underlying tests with this method.
The Alcotezts can be dynamically disabled in the
runtest alias by
setting the environment variable
instance, to run all the unit tests in protocol Alpha exception the
Alcotezts, you can run:
RUNTEZTALIAS=false dune build @src/proto_alpha/runtest
Execution through the Tezt main entrypoint¶
Finally, Alcotezts can be executed through the Tezt main entrypoint
tezt/tests/main.exe. All Alcotezts are linked with this binary,
so that all system, integration and unit tests registered with either Tezt
or Alcotezt can be executed by invoking:
dune exec tezt/tests/main.exe
This is used to run all tests in the CI in a unified and load-balanced manner. This is less convenient for local use though, as there is currently no way of executing the subset of tests that corresponds to a specific package, file or folder. A forthcoming release of Tezt will ameliorate this.
Notable Differences Between Alcotest and Alcotezt¶
First of all, some definitions:
In Alcotest, a “test suite” is a sequence of “tests” that are each composed of a sequence of “test cases”.
In Tezt, there are only “tests”, no “test suites”.
With Alcotezt, each Alcotest “test case” becomes a Tezt “test”. The
Alcotest suite, test and case name are used to form the title of
the Tezt test with following format:
Given an Alcotest consisting of a suite
Suite A that contains a
Test a that itself contains the test cases
Test case a1
Test case a2. It also contains a test
Test b with the test
Test case b1:
let () = Alcotest.run "Suite A" [ ( "Test a", [ ("Test case a1", `Quick, fun () -> ...); ("Test case a2", `Quick, fun () -> ...); ] ); ( "Test b", [ ("Test case b1", `Quick, fun () -> ...); ] ); ]
Running it in with Alcotest produces:
Testing `Suite A'. This run has ID `3F91T9S2'. [OK] Test a 0 Test case a1. [OK] Test a 1 Test case a2. [OK] Test b 0 Test case b1. Full test results in `/home/tezos/_build/_tests/Suite A'. Test Successful in 0.000s. 2 tests run.
And running it with Alcotezt produces:
[17:07:42.289] [SUCCESS] (1/2) Suite A: Test a (Test case a1) [17:07:42.289] [SUCCESS] (2/3) Suite A: Test a (Test case a2) [17:07:42.290] [SUCCESS] (3/3) Suite A: Test b (Test case b1)
Format’s output to Tezt’s Log.debug.
To see the debug output of an Alcotezt, pass the
--verbose flag to
Tezt. See the section Running Alcotezts above for more
information on how to pass flags to Tezt when executing Alcotezts.
There is no way to redirect the output of
the output of Alcotezts that call this module directly cannot be
Integration with the
Alcotezts are registered as a dependency on the
alias. However, they are not executed through this alias in
the CI. Instead, they run through the Tezt main runner to enable load