Here we describe the Argilla listeners capabilities:

class argilla.listeners.Metrics(*args, **kwargs)#

Metrics results for a single listener execution.

The metrics object exposes the metrics configured for the listener as property values. For example, if you define a listener including the metric โ€œF1โ€, the results will be accessible as metrics.F1

class argilla.listeners.RGDatasetListener(dataset, action, metrics=None, query=None, query_params=None, condition=None, query_records=True, interval_in_seconds=30)#

The argilla dataset listener class

  • dataset (str) โ€“ The dataset over which listener is created

  • action (Union[Callable[[List[Union[TextClassificationRecord, TokenClassificationRecord, Text2TextRecord, TextGenerationRecord]], RGListenerContext], bool], Callable[[RGListenerContext], bool]]) โ€“ The action to execute when the condition is satisfied

  • metrics (Optional[List[str]]) โ€“ A list of metrics ids that will be required in condition

  • query (Optional[str]) โ€“ The query string to apply

  • query_params (Optional[Dict[str, Any]]) โ€“ Defined parameters used dynamically in the provided query

  • condition (Optional[Callable[[Search, Optional[RGListenerContext]], bool]]) โ€“ The condition to satisfy to execute the action

  • query_records (bool) โ€“ If False, the records wonโ€™t be passed as argument to the action. Default: True

  • interval_in_seconds (int) โ€“ How often the listener is executed. Default to 30 seconds

property formatted_query: Optional[str]#

Formatted query using defined query params, if any


True if listener is running

start(*action_args, **action_kwargs)#

Start listen to changes in the dataset. Additionally, args and kwargs can be passed to action by using the action_* arguments

If the listener is already started, a ValueError will be raised


Stops listener if itโ€™s still running.

If listener is already stopped, a ValueError will be raised

class argilla.listeners.RGListenerContext(listener, search=None, metrics=None, query_params=None)#

The argilla listener execution context. This class keeps the context components related to a listener

  • listener (RGDatasetListener) โ€“ The argilla listener instance

  • search (Optional[Search]) โ€“ Search results for current execution

  • metrics (Optional[Metrics]) โ€“ Metrics results for current execution

  • query_params (Optional[Dict[str, Any]]) โ€“ Dynamic parameters used in the listener query

property dataset: str#

Computed property that returns the configured listener dataset name

property query: Optional[str]#

Computed property that returns the configured listener query string

class argilla.listeners.Search(total, query_params=None)#

Search results for a single listener execution

  • total (int) โ€“ The total number of records affected by the listener query

  • query_params (Optional[Dict[str, Any]]) โ€“ The query parameters applied to the executed search

argilla.listeners.listener(dataset, query=None, metrics=None, condition=None, with_records=True, execution_interval_in_seconds=30, **query_params)#

Configures the decorated function as an argilla listener.

  • dataset (str) โ€“ The dataset name.

  • query (Optional[str]) โ€“ The query string.

  • metrics (Optional[List[str]]) โ€“ Required metrics for listener condition.

  • condition (Optional[Callable[[Search, Optional[RGListenerContext]], bool]]) โ€“ Defines condition over search and metrics that launch action when is satisfied.

  • with_records (bool) โ€“ Include records as part of action arguments. If False, only the listener context RGListenerContext will be passed. Default: True.

  • execution_interval_in_seconds (int) โ€“ Define the execution interval in seconds when listener iteration will be executed.

  • **query_params โ€“ Dynamic parameters used in the query. These parameters will be available via the listener context and can be updated for subsequent queries.