V-Blaze and V-Cloud Online Help

metrics

URL:http://vblaze_name:17171/metrics

Note

The /metrics endpoint is only available on V‑Blaze deployments that have the voci-python-prometheus-client package installed.

Example Response:

# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge 
process_virtual_memory_bytes 7.576633344e+09
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge 
process_resident_memory_bytes 3.8150144e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge 
process_start_time_seconds 1.64392358343e+09
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 734.88
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 12.0
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1024.0...

Note

The response body in the sample above contains hundreds of lines and has been truncated.

Explanation:

A GET request to the /metrics endpoint returns a response similar to the sample above. The response body contains the metrics in text/plain format. Refer to Prometheus Exposition Format for more detailed information.

The # HELP lines provide a description of the associated metric. The # TYPE lines define the name and type of metric. The following list explains the different types of metrics:

gauge

A gauge includes a single value that can go up or down. For example, the following snippet shows a gauge named asr_webapi_connections where the value indicates the number of active connections:

# HELP asr_webapi_connections Number of active connections to the WebAPI
# TYPE asr_webapi_connections gauge
asr_webapi_connections 1.0
counter

A counter includes a single value that can only go up or reset to 0. For example, the following snippet shows a counter named process_cpu_seconds_total where the value is the total number of CPU time in seconds:

# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 734.88

Note

Responses returned by the /metrics endpoint include a large amount of text. For that reason, Voci does not recommend manual inspection of the metrics data. Instead, forward the metrics to other tools that can understand and analyze the Prometheus exposition format.

Table 1. Metrics Reference

Metric Name

n Time Measured

Description

asr_(n)m_acoustic_model_load

1, 5, 15

Average acoustic model load in the past n minutes.

asr_(n)m_acoustic_model_utilization_ratio

1, 5, 15

Acoustic model utilization in the past n minutes.

asr_(n)m_idle_seconds

1, 5, 15

Seconds spent idle in the past n minutes.

asr_(n)m_language_model_load

1, 5, 15

Average language model load in the past n minutes.

asr_(n)m_language_model_utilization_ratio

1, 5, 15

Language model utilization ratio in the past n minutes.

asr_(n)m_stream_audio_seconds

1, 5, 15

Seconds of stream audio processed in the past n minutes.

asr_(n)m_stream_errors

1, 5, 15

Number of stream errors in the past n minutes.

asr_(n)m_stream_latency

1, 5, 15

Average stream latency in the past n minutes.

asr_(n)m_stream_load

1, 5, 15

Average stream load in the past n minutes.

asr_(n)m_stream_max_tat_seconds

1, 5, 15

Maximum stream turnaround time in the past n minutes.

asr_(n)m_stream_max_tat_seconds

1, 5, 15

Maximum stream turnaround time in the past n minutes.

asr_(n)m_stream_rate

1, 5, 15

Average stream audio submitted per second in the past n minutes.

asr_(n)m_stream_speed

1, 5, 15

Average stream audio processed per second in the past n minutes.

asr_(n)m_stream_tat_seconds

1, 5, 15

Average stream turnaround time in the past n minutes.

asr_(n)m_stream_utilization_ratio

1, 5, 15

Stream utilization ratio in the past n minutes.

asr_(n)m_stream_warnings

1, 5, 15

Number of stream warnings in the past n minutes.

asr_(n)m_streams

1, 5, 15

Number of streams processed in the past n minutes.

asr_(n)m_utterance_audio_seconds

1, 5, 15

Utterance audio seconds processed in the past n minutes.

asr_(n)m_utterance_latency

1, 5, 15

Average utterance latency in the past n minutes.

asr_(n)m_utterance_max_tat_seconds

1, 5, 15

Maximum utterance turnaround time in the past n minutes.

asr_(n)m_utterance_rate

1, 5, 15

Average utterance audio seconds submitted per second in the past n minutes.

asr_(n)m_utterance_speed

1, 5, 15

Average utterance audio seconds processed per second in the past n minutes.

asr_(n)m_utterance_tat_seconds

1, 5, 15

Average utterance turn around time in the past n minutes.

asr_(n)m_utterances

1, 5, 15

Number of utterances processed in the past n minutes.

asr_acoustic_model_info

ASR acoustic model information.

asr_active_streams

Number of active streams.

asr_done_streams

Number of done streams.

asr_efficiency_ratio

ASR efficiency ratio.

asr_error_info

ASR stream error information.

asr_gpu_info

ASR GPU information.

asr_idle_for_seconds

Number of seconds since the ASR was last used.

asr_idle_ratio

ASR idle ratio.

asr_idle_seconds_created

Seconds spent idle.

asr_idle_seconds_total

Seconds spent idle.

asr_license_info

ASR license information.

asr_memory_free_bytes

ASR available memory.

asr_memory_huge_pages

Number of huge memory pages.

asr_memory_low_bytes

Lowest ASR memory reading since last restart.

asr_model_last_used_seconds

ASR model least recently used.

asr_queued_streams

Number of queued streams.

asr_stream_audio_seconds_created

Total stream audio processed.

asr_stream_audio_seconds_total

Total stream audio processed.

asr_stream_errors_created

Total number of stream errors.

asr_stream_errors_total

Total number of stream errors.

asr_stream_latency

Average stream latency.

asr_stream_max_tat_seconds

Maximum stream turn around time.

asr_stream_rate

Average stream audio seconds submitted per second.

asr_stream_speed

Average stream audio seconds processed per second.

asr_stream_tat_seconds

Average stream turnaround time.

asr_stream_warnings_created

Total number of stream warnings.

asr_stream_warnings_total

Total number of stream warnings.

asr_streams_created

Total number of streams processed.

asr_streams_total

Total number of streams processed.

asr_system_info

ASR system information.

asr_utterance_audio_seconds_created

Total utterance audio processed.

asr_utterance_audio_seconds_total

Total utterance audio processed.

asr_utterance_latency

Average utterance latency.

asr_utterance_max_tat_seconds

Maximum utterance turnaround time.

asr_utterance_rate

Average time in seconds of utterance audio submitted per second.

asr_utterance_speed

Average time in seconds of utterance audio processed per second.

asr_utterance_tat_seconds

Average utterance turnaround time in seconds.

asr_utterances_created

Total number of utterances processed.

asr_utterances_total

Total number of utterances processed.

asr_version_info

ASR version information.

asr_webapi_connections

Number of active connections to the WebAPI.

asr_webapi_responses_total

Number of responses for all WebAPI endpoints.

asr_webapi_streams

Number of active streams on the WebAPI.

process_cpu_seconds_total

Total user and system CPU time spent in seconds.

process_max_fds

Maximum number of open file descriptors.

process_open_fds

Number of open file descriptors.

process_resident_memory_bytes

Resident memory size in bytes.

process_start_time_seconds

Start time in seconds of the unix epoch process.

process_virtual_memory_bytes

Virtual memory size in bytes.

python_info

Python platform information.