Listeners#

Here we describe the Argilla listeners capabilities

class argilla.listeners.Metrics(*args, **kwargs)#

Metrics results for a single listener execution.

The metrics object exposes the metrics configured for the listener as property values. For example, if you define a listener including the metric “F1”, the results will be accessible as metrics.F1

class argilla.listeners.RGDatasetListener(dataset: str, action: Union[Callable[[List[Union[TextClassificationRecord, TokenClassificationRecord, Text2TextRecord, TextGenerationRecord]], RGListenerContext], bool], Callable[[RGListenerContext], bool]], metrics: Optional[List[str]] = None, query: Optional[str] = None, query_params: Optional[Dict[str, Any]] = None, condition: Optional[Callable[[Search, Optional[RGListenerContext]], bool]] = None, query_records: bool = True, interval_in_seconds: int = 30)#

The argilla dataset listener class

Parameters:

  • dataset – The dataset over which listener is created

  • action – The action to execute when condition is satisfied

  • metrics – A list of metrics ids that will be required in condition

  • query – The query string to apply

  • query_params – Defined parameters used dynamically in the provided query

  • condition – The condition to satisfy to execute the action

  • query_records – If False, the records won’t be passed as argument to the action. Default: True

  • interval_in_seconds – How often the listener is executed. Default to 30 seconds

property formatted_query: Optional[str]#

Formatted query using defined query params, if any

is_running()#

True if listener is running

start(*action_args, **action_kwargs)#

Start listen to changes in the dataset. Additionally, args and kwargs can be passed to action by using the action_* arguments

If the listener is already started, a ValueError will be raised

stop()#

Stops listener if it’s still running.

If listener is already stopped, a ValueError will be raised

class argilla.listeners.RGListenerContext(listener: RGDatasetListener, search: Optional[Search] = None, metrics: Optional[Metrics] = None, query_params: Optional[Dict[str, Any]] = None)#

The argilla listener execution context. This class keeps the context components related to a listener

Parameters:

  • listener – The argilla listener instance

  • search – Search results for current execution

  • metrics – Metrics results for current execution

  • query_params – Dynamic parameters used in the listener query

property dataset: str#

Computed property that returns the configured listener dataset name

property query: Optional[str]#

Computed property that returns the configured listener query string

class argilla.listeners.Search(total: int, query_params: Optional[Dict[str, Any]] = None)#

Search results for a single listener execution

Parameters:

  • total – The total number of records affected by the listener query

  • query_params – The query parameters applied to the executed search

argilla.listeners.listener(dataset: str, query: Optional[str] = None, metrics: Optional[List[str]] = None, condition: Optional[Callable[[Search, Optional[RGListenerContext]], bool]] = None, with_records: bool = True, execution_interval_in_seconds: int = 30, **query_params)#

Configures the decorated function as a argilla listener.

Parameters:

  • dataset – The dataset name.

  • query – The query string.

  • metrics – Required metrics for listener condition.

  • condition – Defines condition over search and metrics that launch action when is satisfied.

  • with_records – Include records as part or action arguments. If False, only the listener context RGListenerContext will be passed. Default: True.

  • execution_interval_in_seconds – Define the execution interval in seconds when listener iteration will be executed.

  • **query_params – Dynamic parameters used in the query. These parameters will be available via the listener context and can be updated for subsequent queries.