Octez Metrics¶
The Octez node is able to produce metrics information and serve them in the Open Metrics format, an emerging standard for exposing metrics data, especially used in cloud-based systems.
Supported Open Metrics¶
The Octez node supports the following metrics, characterized by: the name of the metric, the type of the metric as in the open metrics specification, a user friendly description on the metric and a list of labels (that can be used to aggregate or query metrics).
For more information check the openmetrics specification: https://openmetrics.io/
Name |
Type |
Description |
Labels |
|---|---|---|---|
ocaml_gc_allocated_bytes |
Counter |
Total number of bytes allocated since the program was started. |
|
ocaml_gc_compactions |
Counter |
Number of heap compactions since the program was started. |
|
ocaml_gc_heap_words |
Gauge |
Total size of the major heap, in words. |
|
ocaml_gc_major_collections |
Counter |
Number of major collection cycles completed since the program was started. |
|
ocaml_gc_major_words |
Counter |
Number of words allocated in the major heap since the program was started. |
|
ocaml_gc_minor_collections |
Counter |
Number of minor collection cycles completed since the program was started. |
|
ocaml_gc_top_heap_words |
Counter |
Maximum size reached by the major heap, in words. |
|
octez_mempool_pending_applied |
Gauge |
Mempool pending applied operations count |
|
octez_mempool_pending_branch_delayed |
Gauge |
Mempool pending branch delayed operations count |
|
octez_mempool_pending_branch_refused |
Gauge |
Mempool pending branch refused operations count |
|
octez_mempool_pending_outdated |
Gauge |
Mempool pending outdated operations count |
|
octez_mempool_pending_prechecked |
Gauge |
Mempool pending prechecked operations count |
|
octez_mempool_pending_refused |
Gauge |
Mempool pending refused operations count |
|
octez_mempool_pending_unprocessed |
Gauge |
Mempool pending unprocessed operations count |
|
octez_node_distributed_db_requester_table_length |
Gauge |
Number of entries (to grab) from the network present |
requester_kind;entry_type |
octez_node_store_alternate_heads_count |
Gauge |
Current number of alternated heads known |
|
octez_node_store_caboose_level |
Gauge |
Current caboose level |
|
octez_node_store_checkpoint_level |
Gauge |
Current checkpoint level |
|
octez_node_store_invalid_blocks |
Gauge |
Number of blocks known to be invalid stored on disk |
|
octez_node_store_last_merge_time |
Gauge |
Time, in seconds, for the completion of the last store merge |
|
octez_node_store_last_written_block_size |
Gauge |
Size, in bytes, of the last block written in store |
|
octez_node_store_savepoint_level |
Gauge |
Current savepoint level |
|
octez_p2p_connections_active |
Gauge |
Number of active connections |
|
octez_p2p_connections_incoming |
Gauge |
Number of incoming connections |
|
octez_p2p_connections_outgoing |
Gauge |
Number of outgoing connections |
|
octez_p2p_connections_private |
Gauge |
Number of private connections |
|
octez_p2p_peers_accepted |
Gauge |
Number of accepted connections |
|
octez_p2p_peers_disconnected |
Gauge |
Number of disconnected peers |
|
octez_p2p_peers_running |
Gauge |
Number of running peers |
|
octez_p2p_points_accepted |
Gauge |
Number of accepted points |
|
octez_p2p_points_disconnected |
Gauge |
Number of disconnected points |
|
octez_p2p_points_greylisted |
Gauge |
Number of greylisted points |
|
octez_p2p_points_running |
Gauge |
Number of running points |
|
octez_p2p_points_trusted |
Gauge |
Number of trusted points |
|
octez_validator_block_already_commited_blocks_count |
Counter |
Number of requests to validate a block already handled |
|
octez_validator_block_last_finished_request_completion_timestamp |
Gauge |
Timestamp at which the latest request handled by the worker was completed |
|
octez_validator_block_last_finished_request_push_timestamp |
Gauge |
Reception timestamp of the latest request handled by the worker |
|
octez_validator_block_last_finished_request_treatment_timestamp |
Gauge |
Timestamp at which the worker started processing of the latest request it handled |
|
octez_validator_block_operations_per_pass |
Gauge |
Number of operations per pass for the last validated block |
pass_id |
octez_validator_block_outdated_blocks_count |
Counter |
Number of requests to validate a block older than the node’s checkpoint |
|
octez_validator_block_preapplication_errors_count |
Counter |
Number of refused application simulations of blocks |
|
octez_validator_block_preapplied_blocks_count |
Counter |
Number of successful application simulations of blocks |
|
octez_validator_block_precheck_failed_count |
Counter |
Number of block validation requests where the prechecking of a block failed |
|
octez_validator_block_validated_blocks_count |
Counter |
Number of requests to validate a valid block |
|
octez_validator_block_validation_errors_after_precheck_count |
Counter |
Number of requests to validate an invalid but precheckable block |
|
octez_validator_block_validation_errors_count |
Counter |
Number of requests to validate an invalid block |
|
octez_validator_chain_branch_switch_count |
Counter |
Number of times the chain_validator switched branch |
chain_id |
octez_validator_chain_consumed_gas |
Gauge |
Consumed Gas |
chain_id |
octez_validator_chain_head_cycle |
Gauge |
Current cycle |
chain_id |
octez_validator_chain_head_increment_count |
Counter |
Number of times the chain_validator incremented its head for a direct successor |
chain_id |
octez_validator_chain_head_level |
Gauge |
Current level of the node’s head |
chain_id |
octez_validator_chain_head_round |
Gauge |
Current Round |
chain_id |
octez_validator_chain_ignored_head_count |
Counter |
Number of requests where the chain validator ignored a new valid block with a lower fitness than its current head |
chain_id |
octez_validator_chain_last_finished_request_completion_timestamp |
Gauge |
Timestamp at which the latest request handled by the worker was completed |
chain_id |
octez_validator_chain_last_finished_request_push_timestamp |
Gauge |
Reception timestamp of the latest request handled by the worker |
chain_id |
octez_validator_chain_last_finished_request_treatment_timestamp |
Gauge |
Timestamp at which the worker started processing of the latest request it handled |
chain_id |
octez_version |
Gauge |
Node version |
version;chain_name;distributed_db_version;p2p_version;commit_hash;commit_date |
process_cpu_seconds_total |
Counter |
Total user and system CPU time spent in seconds. |
|
process_start_time_seconds |
Counter |
Start time of the process since unix epoch in seconds. |
|
prometheus_logs_messages_total |
Counter |
Total number of messages logged |
level;src |
Usage¶
To instruct the Octez node to produce metrics, the user needs to pass the option
--metrics-addr=<ADDR>:<PORT>. The port specified on the command line is the port
where the integrated open metrics server will be available (9932 by default).
The address defaults to localhost.
When the option is not supplied at all, no metrics are produced.
Ex.:
tezos-node run --metrics-addr=:9091
To query the open metrics server the user can simply query the node.
Ex.:
curl http://<node_addr>:9091/metrics
Collecting metrics¶
Different third-party tools can be used to query the Octez node and collect metrics from it. Let us illustrates this with the example of a Prometheus server.
Update the Prometheus configuration file (typically, prometheus.yml)
to add a “scrape job” - that is how Prometheus is made aware of a new data
source - using adequate values:
job_name: Use a unique name among other scrape jobs. All metrics collected through this job will have automatically a ‘job’ label with this value added to it
targets: The URL of Octez node.
- job_name: 'tezos-metrics'
scheme: http
static_configs:
- targets: ['localhost:9091']