rh`ddlmZddlZddlmZmZerddlZddlm Z ddl m Z GddZ y)) annotationsN) TYPE_CHECKINGAny)Tensor)SentenceTransformerceZdZdZdZ d d dZd dZ d ddZeddZ ddZ dd Z y)SentenceEvaluatora Base class for all evaluators. Notably, this class introduces the ``greater_is_better`` and ``primary_metric`` attributes. The former is a boolean indicating whether a higher evaluation score is better, which is used for choosing the best checkpoint if ``load_best_model_at_end`` is set to ``True`` in the training arguments. The latter is a string indicating the primary metric for the evaluator. This has to be defined whenever the evaluator returns a dictionary of metrics, and the primary metric is the key pointing to the primary metric, i.e. the one that is used for model selection and/or logging. Extend this class and implement __call__ for custom evaluators. c d|_d|_y)NT)greater_is_betterprimary_metricselfs /var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/sentence_transformers/evaluation/SentenceEvaluator.py__init__zSentenceEvaluator.__init__s!%"Ncy)a This is called during training to evaluate the model. It returns a score for the evaluation with a higher score indicating a better result. Args: model: the model to evaluate output_path: path where predictions and metrics are written to epoch: the epoch where the evaluation takes place. This is used for the file prefixes. If this is -1, then we assume evaluation on test data. steps: the steps in the current epoch at time of the evaluation. This is used for the file prefixes. If this is -1, then we assume evaluation at the end of the epoch. Returns: Either a score for the evaluation with a higher score indicating a better result, or a dictionary with scores. If the latter is chosen, then `evaluator.primary_metric` must be defined N)rmodel output_pathepochstepss r__call__zSentenceEvaluator.__call__s2 rc^dd}|s*|jDcic]\}}|||c}}S|jDcic]\}}|dz|z||}}}t|dr5|jj|dzs|dz|jz|_|Scc}}wcc}}w)Nc< t|S#t$r|cYSwxYwN)float ValueError)values rmaybe_to_floatz@SentenceEvaluator.prefix_name_to_metrics..maybe_to_float:s% U|#   s _r )rrreturnr)itemshasattrr startswith)rmetricsnamerkeyrs rprefix_name_to_metricsz(SentenceEvaluator.prefix_name_to_metrics9s  AHQ:3C..Q QMT]]_]zsE4#:#^E%::]] 4) *43F3F3Q3QRVY\R\3]"&*t/B/B"BD  R]s B#B)c@|jj||||yr)model_card_dataset_evaluation_metrics)rrr%rsteps r store_metrics_in_model_card_dataz2SentenceEvaluator.store_metrics_in_model_card_dataGs 44T7E4Prc|jj}|jdrd|ddz} |jd}|d|}t j dd|S#t$rY"wxYw)a Returns a human-readable description of the evaluator: BinaryClassificationEvaluator -> Binary Classification 1. Replace "CE" prefix with "CrossEncoder" 2. Remove "Evaluator" from the class name 3. Add a space before every capital letter CE CrossEncoderN Evaluatorz([a-z])([A-Z])z \g<1> \g<2>) __class____name__r$index IndexErrorresub)r class_namer5s r descriptionzSentenceEvaluator.descriptionLs|^^,,   &'*QR.8J $$[1E#FU+Jvv'DD   sA A*)A*ciS)z{ Return a dictionary with all meaningful configuration values of the evaluator to store in the model card. rr s rget_config_dictz!SentenceEvaluator.get_config_dictbs  rc (|j|fi|S)ae Call the encoder method of the model pass Args: model (SentenceTransformer): Model we are evaluating sentences (str | list[str] | np.ndarray): Text that we are embedding Returns: list[Tensor] | np.ndarray | Tensor | dict[str, Tensor] | list[dict[str, Tensor]]: The associated embedding )encode)rr sentenceskwargss r embed_inputszSentenceEvaluator.embed_inputshs u||I000r)NrB) rrrz str | NonerintrrCr!zfloat | dict[str, float])r%dict[str, float]r&strr!rD)rr) rrr%dict[str, Any]rrCr,rCr!None)r!rE)r!rF)rrr?zstr | list[str] | np.ndarrayr!zPlist[Tensor] | np.ndarray | Tensor | dict[str, Tensor] | list[dict[str, Tensor]]) r4 __module__ __qualname____doc__rrr(r-propertyr:r<rArrrr r s # ik ( 7A QT be ! 6 `aQ(Q3AQJMQY\Q Q EE* 1"101 Z 1rr ) __future__rr7typingrrnumpynptorchr)sentence_transformers.SentenceTransformerrr rrrrRs&" %Mk1k1r