rh3ddlmZddlZddlZddlZddlZddlmZddlmZddl m Z m Z m Z ddl Z ddl mZe r ddlmZmZmZej&eZd dZ d d Zdd Z d dd Zy)) annotationsN)fnmatch)Path) TYPE_CHECKINGAnyCallable)list_repo_files CrossEncoderSentenceTransformer SparseEncodercdfd }|S)a Wraps the save_pretrained method of a model to save to a subfolder. Args: _save_pretrained_fn: The original save_pretrained function subfolder: The subfolder to save to Returns: A wrapped function that saves to the specified subfolder cttjt|z dt|z fi|S)NT)exist_ok)osmakedirsr)save_directorykwargs_save_pretrained_fn subfolders v/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/sentence_transformers/backend/utils.pywrapperz)_save_pretrained_wrapper..wrapper s5 D(94tD"4#7)#CNvNN)rz str | PathreturnNone)rrrs`` r_save_pretrained_wrapperrsO Nrc &|jdd}|r||fS|j}|jd|}|jdd} | rt| |j nt|j } | rt| ||j nt||j } | r| d|nd|} |r<|j | D cgc]!} | j |j #}} nYt|j d|jdd|jd d }|Dcgc]}t|| s|}}| |v}|sn| |v}|rht|d kDr4d|vr0tjd |d |j d|d| d | rt| |j n||d<||d<|| }t|j}t|d kDr8|d|d<t|jddg|ddj |d<|rQtjd|d|j d|d|r"tjd|d|d|dd||fScc} wcc}w)a( Determines whether the model should be exported to the backend, or if it can be loaded directly. Also update the `file_name` and `subfolder` model_kwargs if necessary. These are the cases: 1. If export is set in model_kwargs, just return export 2. If `/` exists; set export to False 3. If `/` exists; set export to False and set subfolder to the backend (e.g. "onnx") 4. If `` contains a folder, add those folders to the subfolder and set the file_name to the last part We will warn if: 1. The expected file does not exist in the model directory given the optional file_name and subfolder. If there are valid files for this backend, but they're don't align with file_name, then we give a useful warning. 2. Multiple files are found in the model directory that match the target file name and the user did not specify the desired file name via `model_kwargs={"file_name": ""}` Args: load_path: The model repository or directory, as a Path instance is_local: Whether the model is local or remote, i.e. whether load_path is a local directory model_kwargs: The model_kwargs dictionary. Notable keys are "export", "file_name", and "subfolder" target_file_name: The expected file name in the model directory, e.g. "model.onnx" or "openvino_model.xml" target_file_glob: The glob pattern to match the target file name, e.g. "*.onnx" or "openvino*.xml" backend_name: The human-readable name of the backend for use in warnings, e.g. "ONNX" or "OpenVINO" Returns: Tuple[bool, dict[str, Any]]: A tuple of the export boolean and the updated model_kwargs dictionary. Notable keys in model_kwargs are "export", "file_name", and "subfolder". exportN file_namerz/**/z**/modelrevisiontoken) repo_typer"r#z Multiple z files found in z: z, defaulting to zW. Please specify the desired file name via `model_kwargs={"file_name": ""}`.zNo z found in z. Exporting the model to .z#If you intended to load one of the  zN files, please specify the desired file name via `model_kwargs={"file_name": "rz"}`.) poplowergetras_posixglob relative_tor rlenloggerwarningparts) load_pathis_local model_kwargstarget_file_nametarget_file_glob backend_namerbackendr rprimary_full_pathsecondary_full_path glob_patternpathmodel_file_names all_filesfname model_foundfile_name_partss rbackend_should_exportrD'sL  h -F |##  "G  .>?I  d3IAJY 2;;=PTU^P_PhPhPj:CY+446gW`IaIjIjIlP>P>R=UUWXhWiiy{NzQQnoPYY(@(I(I(K^eL %(1L % ~ 9o++O ?a$3B$7 [!$()9)9+r)J$b_]`^`Ma$b$k$k$m [! YMI4F4F4H3KKdeqdrrstu  NN56F5GqWZZjklZmYnnsu  < UmZs&J JJcdd|d}|r |d|dz }n |d|dz }tj|y)z Warns the user to save the model if they just exported it. Args: model_name_or_path: The model name or path is_local: Whether the model is local backend_name: The name of the backend (ONNX or OpenVINO) zSaving the exported zA model is heavily recommended to avoid having to export it again.z# Do so with `model.save_pretrained(z)`.z Do so with `model.push_to_hub(z, create_pr=True)`.N)r1r2)model_name_or_pathr5r9to_logs rbackend_warn_to_saverHsO$L>1r sF78J7MSQQ34F3II\]] NN6rc ddlm} m} m} |dk(rd|d} n |dk(rd|d} t j 5} || |dk(ret | |z } tj| d z |  z tj| d z | | z jd | j} |dk(rZt | |z }|jd d t |  z }|| z }tj|||j} |rd}|rt|jddjddjdd}t|| rd|d|d|d|d|d d }nE| t|| rd|d|d|d|d|d d }n!t|| rd|d|d|d|d|d d }tj | ||dd |d! ||"nt ||z }|jd d t |  z }|| z }tj"|||dk(rPt | | z jd }t || z jd }tj"||dddy#1swYyxYw)#Nrr onnxmodel_z.onnxopenvinoopenvino_model_z.xmlzopenvino_model.xmlzopenvino_model.binz.binT)parentsrr'(z( z, z, )z )zGHello! *This pull request has been automatically generated from the [`zT`](https://sbert.net/docs/package_reference/util.html#sentence_transformers.backend.zI) function from the Sentence Transformers library.* ## Config ```python a ``` ## Tip: Consider testing this pull request before merging by loading the model from this PR with the `revision` argument: ```python from sentence_transformers import SparseEncoder # TODO: Fill in the PR number pr_number = 2 model = SparseEncoder( "z5", revision=f"refs/pr/{pr_number}", backend="z#", model_kwargs={"file_name": "a "}, ) # Verify that everything works as expected embeddings = model.encode(["The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium."]) print(embeddings.shape) similarities = model.similarity(embeddings, embeddings) print(similarities) ``` a ``` ## Tip: Consider testing this pull request before merging by loading the model from this PR with the `revision` argument: ```python from sentence_transformers import SentenceTransformer # TODO: Fill in the PR number pr_number = 2 model = SentenceTransformer( "a ``` ## Tip: Consider testing this pull request before merging by loading the model from this PR with the `revision` argument: ```python from sentence_transformers import CrossEncoder # TODO: Fill in the PR number pr_number = 2 model = CrossEncoder( "a "}, ) # Verify that everything works as expected query = "Which planet is known as the Red Planet?" passages = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." ] scores = model.predict([(query, passage) for passage in passages]) print(scores) ``` r!z Add exported z model ) folder_path path_in_reporepo_idr$commit_messagecommit_description create_pr)sentence_transformersr r r tempfileTemporaryDirectoryrshutilmove with_suffixr-mkdirreprreplace isinstancehuggingface_hub upload_foldercopy)export_functionexport_function_nameconfigrF push_to_hubrV file_suffixr:r!r r r r save_dirdst_dirsource destinationrUopt_config_string bin_sourcebin_destinations rsave_or_push_to_hub_modelrpsxWV&[M/ J %k]$7  $ $ &Y9(! j H~/H KK#77I9M N KK#77(Y:N9[9[\b9c d((*H f 8nw.G MM$M 6(^i/F!I-K KK ,'')H !# $(L$8$8g$F$N$NtU\$]$e$efikp$q!e]3.@AU?UVjkj@@ Y""+ -+*&@]j8K&L.@AU?UVjkj@@ Y""+ -+*&@ |4.@AU?UVjkj@@ Y""+-+$*&L  ) )$$*!!.wiwymL#5# -.8G MM$M 6(^i/F!I-K KK ,*$"8ny8EEfM #'=9#<"I"I&"Q J8sY9Y9Y9s Hrys" //+VV   8 $&_ _ _ !_  _  _  _ !_ D,"GKk9k9k9 k9  k9  k9k9k9 Ek9r