rhdZddlZddlZddlmZddlmZmZmZm Z ddl Z ddl Z ddl m Z ddlmZmZmZddlmZdd lmZdd lmZdd lmZdd lmZmZmZdd lmZm Z ddl!m"Z"m#Z#m$Z$m%Z%ddl&m'Z'm(Z(m)Z)dZ* dMde jVde,de,de,de,de jVf dZ-dNdZ.dZ/dZ0ee#dGdde"Z1ee#d Gd!d"e"Z2ee#Gd#d$e"Z3Gd%d&e jhZ5Gd'd(e jhZ6 dOd)e jhd*e jVd+e jVd,e jVd-ee jVd.e,d/e,fd0Z7Gd1d2e jhZ8Gd3d4e jhZ9Gd5d6eZ:e#Gd7d8e Z;Gd9d:e jhZ<Gd;dd?e;Z>Gd@dAe jhZ?GdBdCe jhZ@e#dDGdEdFe;ZAe#GdGdHe;ZBe#dIGdJdKe;ZCgdLZDy)PzPyTorch Siglip model.N) dataclass)AnyCallableOptionalUnion)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)_calculate_fan_in_and_fan_out)ACT2FN)_prepare_4d_attention_mask)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPoolingImageClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel) ModelOutputauto_docstringcan_return_tuple torch_int) SiglipConfigSiglipTextConfigSiglipVisionConfigcd}||d|zz ks ||d|zzkDrtjdd|||z |z }|||z |z }|jd|zdz d|zdz |j|j |t j dz|j||j||y)Ncddtj|tjdz zdz S)N?@)matherfsqrt)xs }/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/siglip/modeling_siglip.pynorm_cdfz _trunc_normal_..norm_cdf(s(dhhq499S>122c99zjmean is more than 2 std from [a, b] in nn.init.trunc_normal_. The distribution of values may be incorrect.) stacklevelrr!)minmax) warningswarnuniform_erfinv_mul_r"r$add_clamp_)tensormeanstdabr'lus r&_trunc_normal_r;%s: q1s7{q1s7{ 2  ;  !d(c!"A!d(c!"A OOAEAIq1uqy) NN KKdiin$% KK MMaQMr(r4r5r6r7r8returnctj5t|dd|||j|j |dddy#1swYyxYw)anFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}( ext{mean}, ext{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq ext{mean} \leq b`. NOTE: this 'tf' variant behaves closer to Tensorflow / JAX impl where the bounds [a, b] are applied when sampling the normal distribution with mean=0, std=1.0 and the result is subsequently scaled and shifted by the mean and std args. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value rr N)torchno_gradr;r1r2)r4r5r6r7r8s r&trunc_normal_tf_r@IsI* $vq#q!, Cd#$$$s 0AAc,t|\}}|dk(r|}n|dk(r|}n |dk(r||zdz }|z }|dk(r$t|tj|dz y|dk(rCt j 5|j tj|dddy|d k(rItjd |z}t j 5|j| |dddytd |#1swYyxYw#1swYyxYw) Nfan_infan_outfan_avgr)truncated_normalg۶%?r6normaluniformr zinvalid distribution ) r r@r"r$r>r?normal_r/ ValueError) r4scalemode distributionrBrCdenomvariancebounds r&variance_scaling_rQcs3F;OFG x    '!Q&u}H))TYYx%8;N%NO  ! ]]_ 4 NNtyy2N 3 4 4  " !h,' ]]_ + OOUFE * + +0?@@ 4 4 + +s3&C>D >D Dc t|ddy)NrBrErLrMrQr4s r& lecun_normal_rV|sf8:LMr(c t|ddy)NrBrGrSrTrUs r&default_flax_embed_initrXsf8(Cr(z} Base class for vision model's outputs that also contains image embeddings of the pooling of the last hidden states. ) custom_introceZdZUdZdZeejed<dZ eejed<dZ ee ejdfed<dZ ee ejdfed<y)SiglipVisionModelOutputz image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)` *optional* returned when model is initialized with `with_projection=True`): The image embeddings obtained by applying the projection layer to the pooler_output. N image_embedslast_hidden_state. hidden_states attentions) __name__ __module__ __qualname____doc__r\rr> FloatTensor__annotations__r]r^tupler_r(r&r[r[sr 15L(5,,-459x 1 129=AM8E%"3"3S"89:A:>Ju00#567>r(r[ze Base class for text model's outputs that also contains a pooling of the last hidden states. ceZdZUdZdZeejed<dZ eejed<dZ ee ejdfed<dZ ee ejdfed<y)SiglipTextModelOutputz text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)` *optional* returned when model is initialized with `with_projection=True`): The text embeddings obtained by applying the projection layer to the pooler_output. N text_embedsr].r^r_) r`rarbrcrjrr>rdrer]r^rfr_rgr(r&ririsr 04K%++,359x 1 129=AM8E%"3"3S"89:A:>Ju00#567>r(riceZdZUdZdZeejed<dZ eejed<dZ eejed<dZ eejed<dZ eejed<dZ eed<dZeed <d eefd Zy) SiglipOutputa loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `return_loss` is `True`): Contrastive loss for image-text similarity. logits_per_image (`torch.FloatTensor` of shape `(image_batch_size, text_batch_size)`): The scaled dot product scores between `image_embeds` and `text_embeds`. This represents the image-text similarity scores. logits_per_text (`torch.FloatTensor` of shape `(text_batch_size, image_batch_size)`): The scaled dot product scores between `text_embeds` and `image_embeds`. This represents the text-image similarity scores. text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`SiglipTextModel`]. image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`SiglipVisionModel`]. text_model_output (`BaseModelOutputWithPooling`): The output of the [`SiglipTextModel`]. vision_model_output (`BaseModelOutputWithPooling`): The output of the [`SiglipVisionModel`]. Nlosslogits_per_imagelogits_per_textrjr\text_model_outputvision_model_outputr<cHtfdjDS)Nc3dK|]'}|dvr|nt|j)yw))rprqN)getattrto_tuple).0kselfs r& z(SiglipOutput.to_tuple..s=  LLDGRYZ^`aRbRkRkRm m s-0)rfkeysrxs`r&ruzSiglipOutput.to_tuples# YY[   r()r`rarbrcrmrr>rdrernrorjr\rprrqrfrrurgr(r&rlrls&)-D(5$$ %,48hu001837OXe//07/3K%++,304L(5,,-448186:3: %* r(rlceZdZdeffd Zdej dededej fdZd dejdej fd Z xZ S) SiglipVisionEmbeddingsconfigcft|||_|j|_|j |_|j |_tj|j|j|j |j d|_ |j |j zdz|_ |j|_ tj|j|j|_|jdt!j"|jj%ddy)Nvalid) in_channels out_channels kernel_sizestridepaddingr) position_idsrF persistent)super__init__r~ hidden_size embed_dim image_size patch_sizerConv2d num_channelspatch_embedding num_patches num_positions Embeddingposition_embeddingregister_bufferr>arangeexpandrxr~ __class__s r&rzSiglipVisionEmbeddings.__init__s  ++ ++ ++!yy++??  !OOt>1D!--"$,,t/A/A4>>"R ^U\\$:L:L-M-T-TU\-]jopr( embeddingsheightwidthr<c|jd}|jjjd}tjj s%||k(r ||k(r|j|j S|jjjd}|jd}||jz}||jz} t|dz} |jd| | |}|jdddd}tjj||| fdd }|jddddjdd|}|S) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing and no class embeddings. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rrrg?r r)bicubicF)sizerL align_corners)shaperweightr>jit is_tracingr unsqueezerrreshapepermuter functional interpolateview) rxrrrrrpatch_pos_embeddim new_height new_widthsqrt_num_positionss r&interpolate_pos_encodingz/SiglipVisionEmbeddings.interpolate_pos_encodingsE!&&q) //66<O>O&PPJr(F) r`rarbrrr>Tensorintrrdr __classcell__rs@r&r}r}s`q1q($5<<$$UX$]b]i]i$L E$5$5 Z_ZfZf r(r}c eZdZdeffd Z ddeejdeejdeejdejfdZ xZ S) SiglipTextEmbeddingsr~cNt||j}tj|j ||_tj|j||_|jdtj|jjddy)NrrFr) rrrrr vocab_sizetoken_embeddingmax_position_embeddingsrrr>rrrxr~rrs r&rzSiglipTextEmbeddings.__init__s && !||F,=,=yI"$,,v/M/My"Y  ELL)G)GHOOPWXej  r( input_idsr inputs_embedsr<c8||jdn|jd}|jjjd}||kDrtd|d|||jddd|f}||j |}|j|}||z}|S)NrrzRSequence length must be less than max_position_embeddings (got `sequence length`: z and max_position_embeddings: )rrrrJrr)rxrrr seq_lengthmax_position_embeddingposition_embeddingsrs r&rzSiglipTextEmbeddings.forward&s -6,AY__R(}GZGZ[]G^ !%!8!8!?!?!E!Ea!H . .d,<=S LongTensorrdrrrrs@r&rrsk  /  153759 E,,-u//0  1 12   r(rmodulequerykeyvalueattention_maskscalingdropoutc tj||jdd|z}|||z}tjj |dtj j|j}tjj|||j}tj||} | jddj} | |fS)Nrr)rr)ptrainingrr)) r>matmulrrrsoftmaxfloat32rrrr contiguous) rrrrrrrkwargs attn_weights attn_outputs r&eager_attention_forwardrAs<<s}}R'<=GL!#n4 ==((2U]](SVVW\WbWbcL==((6??([L,,|U3K''1-88:K  $$r(c eZdZdZfdZ ddej deej deej eej ffdZ xZ S)SiglipAttentionz=Multi-headed attention from 'Attention Is All You Need' paperct|||_|j|_|j |_|j|j z|_|j|j z|jk7r&td|jd|j d|jdz|_ |j|_ d|_ tj|j|j|_tj|j|j|_tj|j|j|_tj|j|j|_y)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).F)rrr~rrnum_attention_heads num_headshead_dimrJrKattention_dropoutr is_causalrLineark_projv_projq_projout_projrs r&rzSiglipAttention.__init__[s  ++33$..8 ==4>> )T^^ ;MdnnM]^NN#2' ]]D( // ii? ii? ii?  $..$..A r(r^rr<c :|j\}}}|j|}|j|}|j|} |j |||j |j jdd}|j |||j |j jdd}| j |||j |j jdd} t} |jjdk7rt|jj} | |||| ||j|j|jsdn |j\} } | j!|||j#} |j%| } | | fS)z#Input shape: Batch x Time x Channelrr)eager)rrr)rrrrrrrrrr~_attn_implementationrrrKrrrrr) rxr^rr batch_sizerrqueriesrzvaluesattention_interfacerrs r&rzSiglipAttention.forwardosa-:,?,?) J ++m,{{=)]+,,z:t~~t}}U__`acdeyyZOYYZ[]^_ZT^^T]]S]]^_abc(? ;; + +w 6"9$++:Z:Z"[ $7     nnJJ#}}C$,, % ! \"))*j)LWWY mmK0 L((r(N) r`rarbrcrr>rrrfrrrs@r&rrXsVGB.26$)||$)!.$) u||Xell33 4 $)r(rcVeZdZfdZdej dej fdZxZS) SiglipMLPct|||_t|j|_t j|j|j|_ t j|j|j|_ yr) rrr~r hidden_act activation_fnrrrintermediate_sizefc1fc2rs r&rzSiglipMLP.__init__sd  #F$5$5699V//1I1IJ99V55v7I7IJr(r^r<cl|j|}|j|}|j|}|Sr)rrr)rxr^s r&rzSiglipMLP.forwards4/ **=9 / r()r`rarbrr>rrrrs@r&rrs$KU\\ellr(rc eZdZdeeefffd Z ddejdejde e de ejfdZ xZS) SiglipEncoderLayerr~cDt||j|_t j |j|j |_t||_ t j |j|j |_ t||_ y)Neps) rrrrr LayerNormlayer_norm_eps layer_norm1r self_attn layer_norm2rmlprs r&rzSiglipEncoderLayer.__init__sm ++<<Frrboolrfrdrrrs@r&rrsb%u%79I%IJ%-2 $||$ $$D> $ u  ! $r(rc>eZdZUeed<dZdZgdZdZdZ dZ dZ dZ y)SiglipPreTrainedModelr~siglipT)rr}r#SiglipMultiheadAttentionPoolingHeadc t|trt|jtr |jjj n|jj }t jj|jjdtj|z yt|t jrt|jyt|trt jj!|j"jt jj!|j$jt jj!|j&jt jj!|j(jt jj+|j"j,t jj+|j$j,t jj+|j&j,t jj+|j(j,yt|t.rt jj!|j0jt jj!|j2jt jj|j0j,dt jj|j2j,dyt|t4rt jj!|j6j8t jj!|j:j<j8t jj+|j:j>j8yt|t@rrtCjDtCjFd}|jHj8jK||jLj8jOyt|tPrnt jj|jRj|jjj dz|jjTzyt|t jVt jXfrLt[|j|j,*t jj+|j,yyt|t j\rJ|j,j8jO|jj8jKdyy)zInitialize the weightsrrFgư>r rN)/ isinstancer}r~r vision_configrrinitrIrrnpr$rrXrxavier_uniform_rrrrzeros_biasrrrrprobedata attentionin_proj_weight in_proj_bias SiglipModelr>logr4 logit_scalefill_ logit_biaszero_SiglipForImageClassification classifierinitializer_factorrrrVr )rxrrlogit_scale_inits r& _init_weightsz#SiglipPreTrainedModel._init_weightss f4 5dkk<8 ))55[[,,  GGOOF55<  ,$yyc):;     # # ) )*: ;    " " ( ( *  < = GGOO!!((KK--994?$++B`B``  BII 6 7 &-- ({{&v{{+'  - KK   " " $ MM   $ $S ).r(N) r`rarbrrebase_model_prefixsupports_gradient_checkpointing_no_split_modules_supports_flash_attn_supports_sdpa_supports_flex_attn_supports_attention_backendr0rgr(r&rrs:  &*#  N"&,*r(rc xeZdZdZdeffd Ze d deejdee dee de fdZ xZ S) SiglipEncoderz Transformer encoder consisting of `config.num_hidden_layers` self attention layers. Each layer is a [`SiglipEncoderLayer`]. Args: config: SiglipConfig r~ct|||_tjt |j Dcgc] }t|c}|_d|_ ycc}w)NF) rrr~r ModuleListrangenum_hidden_layersrlayersgradient_checkpointing)rxr~rrs r&rzSiglipEncoder.__init__ sO  mmvOgOgIh$iA%7%?$ij &+#%jsA#rroutput_hidden_statesr<c||n|jj}||n|jj}|rdnd}|rdnd}|}|jD]&}|r||fz}||||} | d}|s|| dfz}(|r||fz}t |||S)ad Args: inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. Nrg)rrr)r]r^r_)r~rr@r>r) rxrrrr@encoder_statesall_attentionsr^ encoder_layer layer_outputss r&rzSiglipEncoder.forward's<2C1N-TXT_T_TqTq$8$D $++JjJj  40d% ![[ FM#!/=2B!B)"3M *!,M !/=3C2E!E F +}.>>N+(%  r(r)r`rarbrcrrrrr>rrrrrrs@r&r9r9sl,|,26,0/3 < !.< $D> < 'tn <  < < r(r9ceZdZdeffd Zee d deejdeejdeejdee dee de f d Z xZ S) SiglipTextTransformerr~cFt|||_|j}t ||_t ||_tj||j|_ tj||j|_|jdk(|_y)Nrflash_attention_2)rrr~rrrr9encoderrr r final_layer_normrprojection_sizeheadr_use_flash_attention_2rs r&rzSiglipTextTransformer.__init__hs}  && .v6$V, " YFrrrrrrs@r&rGrGgs Y/ Y-115/3,0/3 . ELL). !.. u||, . $D> . 'tn .  $. . r(rGzK The text model from SigLIP without any head or projection on top. ceZdZUeed<deffd ZdejfdZdZ e e d de e jde e jde e jd e ed e edef d ZxZS) SiglipTextModelr~cdt||t||_|j yr)rrrG text_model post_initrs r&rzSiglipTextModel.__init__s&  /7 r(r<cB|jjjSrrYrrr{s r&get_input_embeddingsz$SiglipTextModel.get_input_embeddingss))999r(c:||jj_yrr\)rxrs r&set_input_embeddingsz$SiglipTextModel.set_input_embeddingss5:""2r(rrrrr@c.|j|||||S)a Examples: ```python >>> from transformers import AutoTokenizer, SiglipTextModel >>> model = SiglipTextModel.from_pretrained("google/siglip-base-patch16-224") >>> tokenizer = AutoTokenizer.from_pretrained("google/siglip-base-patch16-224") >>> # important: make sure to set padding="max_length" as that's how the model was trained >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding="max_length", return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled (EOS token) states ```rrrrr@)rY)rxrrrrr@s r&rzSiglipTextModel.forwards)6)%/!5   r(rU)r`rarbrrerrModuler]r_rrrr>rrrrrrs@r&rWrWs / :bii:;-115/3,0/3  ELL) !. u||,  $D>  'tn   $  r(rWc jeZdZdeffd Zee ddeedeedeede fdZ xZ S) SiglipVisionTransformerr~cLt|||_|j}t ||_t ||_tj||j|_ t|dsdn |j|_|jrt||_yy)Nrvision_use_headT)rrr~rr}rr9rJrr r post_layernormhasattrrfuse_headrrMrs r&rz SiglipVisionTransformer.__init__s  && 08$V,  ll9&:O:OP$+F4E$FFLbLb ==;FCDI r(rr@rr<cv||n|jj}||n|jj}|j||}|j |||}|j }|j |}|jr|j|nd}t|||j|jS)N)r)rrr@rP) r~rr@rrJr]rgrirMrr^r_) rxrrr@rr^rSr]rQs r&rzSiglipVisionTransformer.forwards2C1N-TXT_T_TqTq$8$D $++JjJj  Ogh +/<<'/!5,8,  ,== //0AB8<  "344 )/')77&11   r(NNF) r`rarbrrrrrrrrrrs@r&rdrdsm D1 D-1/338  $D> 'tn  #+4.  $   r(rdc.eZdZdZdeffd ZdZxZS)rzMultihead Attention Pooling.r~ct|tjt j dd|j |_tjj|j |jd|_ tj|j |j|_ t||_y)NrT) batch_firstr)rrr Parameterr>randnrr!MultiheadAttentionrr#r r  layernormrrrs r&rz,SiglipMultiheadAttentionPoolingHead.__init__s \\%++aF4F4F"GH 44V5G5GIcIcqu4vf&8&8f>S>STV$r(c|jd}|jj|dd}|j|||d}|}|j |}||j |z}|dddfS)Nrr)rr!repeatr#rrr)rx hidden_staterr!rs r&rz+SiglipMultiheadAttentionPoolingHead.forwardsv!''*  !!*a3~~e\<HK ~~l3 $((<"88 AqD!!r()r`rarbrcrrrrrs@r&rrs&%1% "r(rzM The vision model from SigLIP without any head or projection on top. c eZdZUeed<dZdeffd ZdejfdZ e e d de e de e de defd ZxZS) SiglipVisionModelr~rcdt||t||_|j yr)rrrd vision_modelrZrs r&rzSiglipVisionModel.__init__/s)  3F; r(r<cB|jjjSr)ryrrr{s r&r]z&SiglipVisionModel.get_input_embeddings7s  ++;;;r(rr@rc,|j||||S)a Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, SiglipVisionModel >>> model = SiglipVisionModel.from_pretrained("google/siglip-base-patch16-224") >>> processor = AutoProcessor.from_pretrained("google/siglip-base-patch16-224") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled features ```rrr@r)ry)rxrrr@rs r&rzSiglipVisionModel.forward:s(<  %/!5%= !  r(rk)r`rarbrremain_input_namerrrbr]rrrrrrrrs@r&rwrw&s $O1! 'tn ! #' ! $ ! ! r(rwc eZdZUeed<deffd Ze ddeejdeejdeejdee dee dejf d Z e dd eejdee dee d e dejf d Z ee ddeejd eejdeejdeejd ee dee dee d e defdZxZS)r&r~ct||t|jts"t dt |jdt|jts"t dt |jd|j}|j}tj|}tj|}|j|_ |j|_ tjt!j"d|_tjt!j"d|_|j)y)NzMconfig.text_config is expected to be of type SiglipTextConfig but is of type .zQconfig.vision_config is expected to be of type SiglipVisionConfig but is of type r)rrr text_configr TypeErrortyperrrW _from_configrwrYryrror>rpr(r*rZ)rxr~rrrYryrs r&rzSiglipModel.__init__ds  &,,.>?++,-Q0  &..0BC--./q2  (( ,, %11+> (55mD %//(55<< A7,,u{{1~6 r(rrrrr@r<c||n|jj}||n|jj}|j|||||}|j}|S)aJ Returns: text_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`SiglipTextModel`]. Examples: ```python >>> from transformers import AutoTokenizer, AutoModel >>> import torch >>> model = AutoModel.from_pretrained("google/siglip-base-patch16-224") >>> tokenizer = AutoTokenizer.from_pretrained("google/siglip-base-patch16-224") >>> # important: make sure to set padding="max_length" as that's how the model was trained >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding="max_length", return_tensors="pt") >>> with torch.no_grad(): ... text_features = model.get_text_features(**inputs) ```ra)r~rr@rYrQ)rxrrrrr@ text_outputsrTs r&get_text_featureszSiglipModel.get_text_featuressr:2C1N-TXT_T_TqTq$8$D $++JjJj 48??)%/!5 4C4 %22 r(rrc||n|jj}||n|jj}|j||||}|j}|S)a Returns: image_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`SiglipVisionModel`]. Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, AutoModel >>> import torch >>> model = AutoModel.from_pretrained("google/siglip-base-patch16-224") >>> processor = AutoProcessor.from_pretrained("google/siglip-base-patch16-224") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor(images=image, return_tensors="pt") >>> with torch.no_grad(): ... image_features = model.get_image_features(**inputs) ```r|)r~rr@ryrQ)rxrrr@rvision_outputsrTs r&get_image_featureszSiglipModel.get_image_featuressrB2C1N-TXT_T_TqTq$8$D $++JjJj 6:5F5F%/!5%= 6G6 '44 r( return_lossc ||n|jj}||n|jj}|j||||} |j |||||} | j } | j } | | j dddz } | | j dddz } tj| | jj| j} |jj| j|jj| j}}| |jz|z} | j}d}|rtj| j!d| j }tj"|  d|zz}tj$j&j)|| z}tj*|d  }|j-}t/||| | | | | S) a return_loss (`bool`, *optional*): Whether or not to return the contrastive loss. Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, AutoModel >>> import torch >>> model = AutoModel.from_pretrained("google/siglip-base-patch16-224") >>> processor = AutoProcessor.from_pretrained("google/siglip-base-patch16-224") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> texts = ["a photo of 2 cats", "a photo of 2 dogs"] >>> # important: we pass `padding=max_length` since the model was trained with this >>> inputs = processor(text=texts, images=image, padding="max_length", return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> logits_per_image = outputs.logits_per_image >>> probs = torch.sigmoid(logits_per_image) # these are the probabilities >>> print(f"{probs[0][0]:.1%} that image 0 is '{texts[0]}'") 31.9% that image 0 is 'a photo of 2 cats' ```Nr|rar)rT)rrkeepdimr)devicer)rmrnrorjr\rprq)r~rr@ryrYrQnormr>rtrrr(r*expeyer ones_likerr logsigmoidsumr5rl)rxrrrrrrr@rrrr\rjror(r*rnrmrm1_diag1logliknlls r&rzSiglipModel.forwardsX2C1N-TXT_T_TqTq$8$D $++JjJj 6:5F5F%/!5%= 6G6 48??)%/!5 4C4 &33 "00 $l&7&7!T&7&RR !K$4$4qb$$4$OO  ,,{LNN4D4G4G HZHZ4[\"&"2"2"5"5k6H6H"I4??K]K]^i^p^pKqZ )KOO,== J*,,. ))O003Orrrdrrrrrlrrrs@r&r&r&`s |@-115/3,0/3 +ELL)+!.+u||, + $D> + 'tn +   ++Z59,0/3). .u001.$D>.'tn . #' .    ..`15481537&*,0/3).^ E,,-^ u001^ !. ^ u//0 ^ d^ ^ $D>^ 'tn^ #'^  ^ ^ r(r&z SigLIP vision encoder with an image classification head on top (a linear layer on top of the pooled final hidden states of the patch tokens) e.g. for ImageNet. ceZdZdZdeddffd Zee d dee jdee jdee dee d e de f d Z xZS) r,rr~r<Nc~t|||j|_tj |j }|j |_|jdkDr4tj|j j|jntj|_ |jy)Nr) rr num_labelsrwrrryrrrIdentityr-rZ)rxr~ryrs r&rz%SiglipForImageClassification.__init__Os   ++)55f6J6JK (55OUN_N_bcNcBIIf**668I8I Jikititiv  r(labelsrr@rc||n|jj}||n|jj}|j||||}|j}t j |d}|j|}d} ||j|j}|jj|jdk(rd|j_ nl|jdkDrL|jt jk(s|jt jk(rd|j_ nd|j_ |jjdk(rIt} |jdk(r&| |j!|j!} n| ||} n|jjdk(r=t#} | |j%d|j|j%d} n,|jjdk(rt'} | ||} t)| ||j*|j, S) a$ labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoImageProcessor, SiglipForImageClassification >>> import torch >>> from PIL import Image >>> import requests >>> torch.manual_seed(3) # doctest: +IGNORE_RESULT >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> # note: we are loading a `SiglipModel` from the hub here, >>> # so the head will be randomly initialized, hence the predictions will be random if seed is not set above. >>> image_processor = AutoImageProcessor.from_pretrained("google/siglip-base-patch16-224") >>> model = SiglipForImageClassification.from_pretrained("google/siglip-base-patch16-224") >>> inputs = image_processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits >>> # model predicts one of the two classes >>> predicted_class_idx = logits.argmax(-1).item() >>> print("Predicted class:", model.config.id2label[predicted_class_idx]) Predicted class: LABEL_1 ```N)rr@rrr regressionsingle_label_classificationmulti_label_classificationr)rmlogitsr^r_)r~rr@ryr]r>r5r-rr problem_typerrlongrr squeezer rr rr^r_) rxrrrr@rrsequence_outputrrmloss_fcts r&rz$SiglipForImageClassification.forwardasR2C1N-TXT_T_TqTq$8$D $++JjJj /3.?.? /!5%= /@/ "33 **_!<1  YYv}}-F{{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#FNN$4fnn6FGD#FF3D))-JJ+- B @&++b/R))-II,./$!//))   r()NNNNF)r`rarbr}rrrrrr>rrrrrrs@r&r,r,Fs%O|$04)-,0/3). X u||,X &X $D> X 'tn X #' X  X X r(r,)r&rrWrwr,)rr gr!)r rBrG)r)Ercr"r- dataclassesrtypingrrrrnumpyrr>rtorch.nnr r r torch.nn.initr activationsrmodeling_attn_mask_utilsrmodeling_layersrmodeling_outputsrrrmodeling_utilsrrutilsrrrrconfiguration_sigliprrrr;rfloatr@rQrVrXr[rirlrbr}rrrrrrr9rGrWrdrrwr&r,__all__rgr(r&rs !11 AA7!B9bbFMMTT! J\_$ LL$ %$27$BG$SX$ \\$4A2ND  ?k ?   ?  ?K ?   ?   ;     FERYYER%299%^% II% <<% % << % U\\* %  %%.;)bii;)~  -3-`<*O<*<*@M BIIM `< BII< ~ 0 +0  0 f- bii- `""))"0 2 -2  2 jb 'b b J o #8o  o d r(