Monitoring#
Here we describe the available monitors in Argilla:
Base Monitor: Internal mechanism to queue and log monitored predictions
ArgillaLogHTTPMiddleware: Asgi middleware to monitor API endpoints
Framework Monitors: Monitors to wrap around common NLP inference frameworks
Base Monitor#
- class argilla.monitoring.base.BaseMonitor(*args, api, dataset, sample_rate=1.0, log_interval=1.0, agent=None, tags=None, **kwargs)#
A base monitor class for easy task model monitoring
- dataset:
argilla dataset name
- sample_rate:
The portion of the data to store in argilla. Default = 0.2
- is_record_accepted()#
Return True if a record should be logged to argilla
- Return type
bool
- shutdown()#
Stop consumers
- class argilla.monitoring.base.DatasetRecordsConsumer(name, api, tags=None, metadata=None, buffer_size=10000, upload_size=256, upload_interval=1.0, retries=10, timeout=15, on_error=None)#
Consumes the records from the dataset queue.
- Parameters
name (str) –
api (argilla.client.client.Argilla) –
tags (Optional[dict]) –
metadata (Optional[dict]) –
buffer_size (int) –
- log_next_batch()#
Upload the next batch of items, return whether successful.
- pause()#
Pause the consumer.
- run()#
Runs the consumer.
- send(records)#
Send records to the consumer
- Parameters
records (Iterable[Union[argilla.client.models.TextClassificationRecord, argilla.client.models.TokenClassificationRecord, argilla.client.models.Text2TextRecord, argilla.client.models.TextGenerationRecord]]) –
- exception argilla.monitoring.base.ModelNotSupportedError#