Tezt: OCaml Tezos Test Framework¶
Tezt is a test framework for Tezos written in OCaml. It focuses on integration tests that launch external processes (in particular Tezos nodes and clients).
Tezt is pronounced /tɛzti/ (think “tezty”, as in Tez are tas*ty*).
Its main strengths are:
tests are written in the same language as Tezos itself (OCaml), which reduces context switch for developers;
tests do not actively poll the node as they passively listen to node events instead, which results in faster and more reliable tests;
in verbose mode, logs show the interleaved output of all external processes, while the tests are running;
it should be easy to use and extend.
How to Run Tests¶
If you just want to run tests and see whether they succeed, run:
If you need more control, get the list of command-line options as follows:
dune exec tezt/tests/main.exe -- --help
Command-line options give you control over verbosity and the list of tests to run.
It also allows to to keep temporary files or to keep going with other tests if a test fails.
For instance, here is how to run all tests with tag
node in verbose mode:
dune exec tezt/tests/main.exe -- --verbose node
And here is how to get the list of tests and their tags:
dune exec tezt/tests/main.exe -- --list
To speed things up, you can run tests in parallel.
parallel-tezt.Makefile and specify the number of parallel jobs with
make --makefile parallel-tezt.Makefile --jobs 8
This form of testing is used to prevent unintended changes to existing functionality by ensuring that the software behaves the same way as it did before introduced changes.
Regression tests capture commands and output of commands executed during a test. An output of regression test is stored in the repository and is expected to match exactly with the captured output on subsequent runs. An added advantage of this is that when a change in behaviour is intentional, its effect is made visible by the change in test’s output.
To run all the regression tests, use the
dune exec tezt/tests/main.exe regression
When the change in behaviour is intentional or when a new regression test is
introduced, the output of regression test must be (re-)generated. This can be
done with the
--reset-regressions option, e.g.:
dune exec tezt/tests/main.exe regression -- --reset-regressions
hook located in
executes modified tezt tests automatically. It looks for staged files
(the default) or modified files (if
--unstaged is passed) in
tezt/tests and executes them. This avoids
pushing commits that will break the CI. It is also handy to execute
the relevant subset of tests by calling
./scripts/pre_commit/pre_commit.py [--unstaged] manually.
We refer to the header of
pre_commit.py and its
for additional instructions.
Tezt is composed of some generic, non-Tezos-related modules:
Basemodule with some generally useful “pervasive” functions;
Logmodule to manage the output;
Processmodule to manage processes — the output of those processes is transparently logged;
Tempmodule to manage temporary files and directories;
JSONmodule with lightweight combinators to read JSON values;
Climodule which reads command-line options on startup, such as the list of tests to run and the verbosity level;
Testmodule with a
Test.registerfunction to register tests and a
Test.runfunction to run them and clean up after them;
Regressionmodule for regression testing by comparing command outputs with previous runs.
Progressmodule to track and display progress of a test suite.
All those modules are part of the
tezt library which can be found in directory
tezt/lib of the Tezos repository.
Tezt also contains the following Tezos-specific modules:
Constantmodule with constants such as protocol hashes or identities;
Clientmodule to run client commands;
Nodemodule to run node commands and to manage node daemons;
RPCmodule with some RPC implementations;
Clustermodule to start networks with many nodes with topologies ranging from simple cliques to complex arrangements of rings, stars, etc.
All those modules are part of the
tezt-tezos library, which depends on
and which can be found in directory
tezt/lib_tezos of the Tezos repository.
How to Write New Tests¶
The best way to get started is to have a look at existing tests in directory
tezt/tests of the Tezos repository.
Currently, all tests are part of the same binary
The source of this module is
This binary runs all tests, but you can restrict the set of tests to run
by specifying tags on the command line, or even the titles of the tests to run
All tests do not have to be implemented in
You can of course add more modules and have them be linked into
The best way to do this is to write your tests as functions and call them from
the main module.
For instance, let’s create a new basic test in a new file named
let check_node_initialization (history_mode : Node.history_mode) : Protocol.t list -> unit = Protocol.register_test ~__FILE__ ~title: (sf "node initialization (%s mode)" (Node.show_history_mode history_mode)) ~tags:["basic"; "node"; Node.show_history_mode history_mode] @@ fun protocol -> let metrics_addr = "localhost:" ^ string_of_int (Port.fresh ()) in let* node = Node.init [History_mode history_mode; Metrics_addr metrics_addr] in let* client = Client.init ~endpoint:(Node node) () in let* () = Client.activate_protocol ~protocol client in Log.info "Activated protocol." ; let* () = repeat 10 (fun () -> Client.bake_for_and_wait client) in Log.info "Baked 10 blocks." ; let* level = Node.wait_for_level node 11 in Log.info "Level is now %d." level ; let* identity = Node.wait_for_identity node in if identity = "" then Test.fail "identity is empty" ; Log.info "Identity is not empty." ; let addr = "http://" ^ metrics_addr ^ "/metrics" in let* metrics = Process.spawn ~log_output:false "curl" ["-s"; addr] |> Process.check_and_read_stdout in if metrics = "" then Test.fail "Unable to read metrics" else return () let register ~protocols = check_node_initialization Archive protocols ; check_node_initialization (Full None) protocols ; check_node_initialization (Rolling None) protocols
Then, let’s launch the test from
tezt/tests/main.ml by calling:
Basic.register ~protocols:[Alpha] ; Test.run () (* This call should already be there. *)
Finally, let’s try it with:
dune exec tezt/tests/main.exe -- basic --info
--info flag allows you to see the
Here is what you should see:
$ dune exec tezt/tests/main.exe -- basic --info [13:45:36.666] Starting test: Alpha: node initialization (archive mode) [13:45:37.525] Activated protocol. [13:45:38.215] Baked 10 blocks. [13:45:38.215] Level is now 11. [13:45:38.215] Identity is not empty. [13:45:38.231] [SUCCESS] (1/3) Alpha: node initialization (archive mode) [13:45:38.231] Starting test: Alpha: node initialization (full mode) [13:45:39.113] Activated protocol. [13:45:39.813] Baked 10 blocks. [13:45:39.813] Level is now 11. [13:45:39.813] Identity is not empty. [13:45:39.828] [SUCCESS] (2/3) Alpha: node initialization (full mode) [13:45:39.828] Starting test: Alpha: node initialization (rolling mode) [13:45:40.708] Activated protocol. [13:45:41.407] Baked 10 blocks. [13:45:41.407] Level is now 11. [13:45:41.407] Identity is not empty. [13:45:41.422] [SUCCESS] (3/3) Alpha: node initialization (rolling mode)
Detailed Walkthrough of the Basic Test¶
Let’s review what our basic test in the previous section does.
First, note that the Tezt library and its Base module are opened automatically by Dune based on its configuration file. The Base module contains useful functions such as
sf(a short-hand for
Then, we define a function
check_node_initializationwhich registers one test. It is parameterized by the history mode.
Protocol.register_testis partially applied here;
check_node_initializationis also implicitly parameterized by the list of protocols to run the test on.
Protocol.register_testregisters a test. It is a wrapper over
Test.register. This wrapper is preferred when the test is parameterized by a list of protocols. The
~__FILE__argument gives the source filename so that one can select this file with the
--fileargument, to only run tests declared in this file. Each test has a title which is used in logs and on the command-line with the
--testoption (which allows to run a particular test from its title). Each test also has a list of tags. We gave our test the tag
basicin particular. No other test has this tag, so it is easy to run all of the tests of our new
Basicmodule, and only them, by adding
basicon the command-line.
Protocol.register_testtakes a function as an argument. This function contains the implementation of the test.
First, we initialize a node with
Node.init. This creates a node and runs the node command
identity generate, then
config initand finally
run. It then waits until the node is ready, and returns the node. Note that you do not have to call
Node.init. For instance, if you want to test the behavior of the node without an identity, you can call
Node.create, followed by
Then, we initialize a client with
Client.init. We give it a node, which is the node that the client will connect to by default. Note that we can still use this client to perform operations on other nodes if we want to, it’s just convenient to specify it once and for all.
Then, we activate the protocol with the
activate protocolcommand of the client. By default, this activates the protocol given as argument, with some default parameters and using the default activator key (defined in the
Constantmodule). This activator key was added to the client by
Client.init. You can override all of this. For instance, if you don’t want the client to know the default activator key, use
Client.init(you can use
Client.import_secret_keyto import another activator key, for instance). Or, if you want to change the fitness or the parameter file, you can use the
?parameter_fileoptional arguments of
Then, we log a message using
Log.info. This message is not visible with the default verbosity, but you can see it by running
Then, we repeat
Client.bake_for10 times, to bake 10 blocks.
Then, we wait for the level of the node to be at least 11 (the activation block plus the 10 blocks that we baked) using
Node.wait_for_level. If you call this function and the level is already 11 or greater,
Node.wait_for_levelreturns immediately. (Note:
Node.wait_for_levelmakes the test fail if the node stops before reaching level 11.)
Finally, we read the identity of the node using
Node.wait_for_identitywhich returns as soon as the node reads the identity file. In fact, this was probably done much sooner, but Tezt stores the identity in case you try to query it later, just like the level. (Note:
Node.wait_for_identitymakes the test fail if the node stops before reading the identity file.)
We check that the identity is not empty, and if it is we call
Test.fail. This causes the test to terminate immediately with an error. Note that it is not the only cause of failure for this test: we already saw that
Node.wait_for_identitycan cause a test failure, and if anything goes wrong (failing to initialize the node or the client, failing to activate the protocol…)
Test.failis called automatically as well.
After the test succeeds or fails,
Test.runcleans up everything. It terminates all running processes by sending
SIGTERM. It waits for them with
waitpidto avoid zombie processes. And it removes all temporary files, in particular the data directory of the node and the base directory of the client.
We run this test three times, once per history mode:
rolling. Note that we added the history mode as a tag to
Test.register, so if we want to run only the test for history mode
full, for instance, we can simply run
dune exec tezt/tests/main.exe -- basic full. You can see our list of basic tests and their tags with
dune exec tezt/tests/main.exe -- basic --list.