rh fddlmZddlZddlmZddlmZeje Z dddZ ddZ y) ) annotationsN)loggingctjrtjr|r*tjj j |}nntj}t|Dcgc]}tj|}}tj ||tj}|||<tj|dSd}tj||Scc}w)ap Gathers a tensor from each distributed rank into a list. Always retains gradients for the local rank's tensor, and optionally retains gradients for the gathered tensors if `with_grad` is True. Args: tensor (torch.Tensor): The tensor to gather from each rank. with_grad (bool, optional): If True, the local rank's tensor retains its gradients. Defaults to False. Returns: torch.Tensor: A tensor containing the gathered tensors from all ranks, concatenated along the first dimension. If torch.distributed is not available or not initialized, returns the original tensor. r)dimzTrying to gather while torch.distributed is not available or has not been initialized, returning the original (local) tensor. This is expected if you are only using one GPU; consider not using gathering to remove this warning.)dist is_availableis_initializedtorch distributednn all_gatherget_world_sizerange zeros_likeget_rankcatlogger warning_once)tensor with_gradgathered_tensors world_size_ local_rankwarnings y/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/sentence_transformers/util/distributed.pyr r s t224 $0033>>vF ,,.JBG BSTQ 0 0 8T T OO,f 5J+1 Z (yy)q11 S   M# Us6C3ct|dS)a Gathers a tensor from each distributed rank into a list, retaining gradients for the local rank's tensor. Args: tensor (torch.Tensor): The tensor to gather from each rank. Returns: torch.Tensor: A tensor containing the gathered tensors from all ranks, concatenated along the first dimension. If torch.distributed is not available or not initialized, returns the original tensor. T)r)r )rs rall_gather_with_gradr2s f --)F)r torch.Tensorrboolreturnr )rr r"r ) __future__rr torch.distributedr rtransformers.utilsr get_logger__name__rr rrrr)s0" &   H %$N .r