rh~ (dZddlZddlZddlmZmZddlZddlZddlm Z ddl m Z ddl m Z ddlmZmZdd lmZdd lmZmZdd lmZd d lmZej4eZGdde j:Zej>j@dZ!dZ"Gdde j:Z#d2dejHde%de&dejHfdZ'Gdde j:Z(Gdde j:Z)Gdde j:Z*Gdd e j:Z+d!Z,d"Z-Gd#d$e Z.Gd%d&e j:Z/d'e j:ddfd(Z0eGd)d*eZ1eGd+d,e1Z2ed-.Gd/d0e1eZ3gd1Z4y)3zPyTorch ViTDet backbone.N)OptionalUnion)nn)ACT2FN)GradientCheckpointingLayer)BackboneOutputBaseModelOutput)PreTrainedModel)auto_docstringlogging) BackboneMixin) VitDetConfigc`eZdZdZfdZdZdejdejfdZxZ S)VitDetEmbeddingsz This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) to be consumed by a Transformer. cpt||j|j}}|j|j }}t |tjjr|n||f}t |tjjr|n||f}|d|dz|d|dzz}||_ ||_||_||_ |jr?|dz}tjtj d||j |_nd|_tj$|||||_y)Nrr) kernel_sizestride)super__init__pretrain_image_size patch_size num_channels hidden_size isinstance collectionsabcIterable image_size num_patches use_absolute_position_embeddingsr Parametertorchzerosposition_embeddingsConv2d projection) selfconfigr rrrr! num_positions __class__s }/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/vitdet/modeling_vitdet.pyrzVitDetEmbeddings.__init__+s !'!;!;V=N=NJ $*$7$79K9Kk #-j+//:R:R#SZZdfpYq #-j+//:R:R#SZZdfpYq !!} 1 5*Q-:VW=:XY $$(&  2 2'!OM')||EKK=RXRdRd4e'fD $'+D $))L+:^hic|r |ddddf}|jd}ttj|}||z|k7r t dt j js ||k7s||k7r]tjj|jd||djdddd||fdd }|jddddS|jd||dS) a Calculate absolute positional embeddings. If needed, resize embeddings and remove cls_token dimension for the original embeddings. Args: abs_pos_embeddings (`torch.Tensor`): Absolute positional embeddings with (1, num_position, num_channels). has_cls_token (`bool`): If true, has 1 embedding in abs_pos_embeddings for cls token. height (`int`): Height of input image tokens. width (`int`): Width of input image tokens. Returns: Absolute positional embeddings after processing with shape (1, height, width, num_channels) Nrz5Absolute position embeddings must be a square number.rrbicubicF)sizemode align_corners) shapeintmathsqrt ValueErrorr$jit is_tracingr functional interpolatereshapepermute)r)abs_pos_embeddings has_cls_tokenheightwidth num_positionr3new_abs_pos_embeddingss r-get_absolute_positionsz'VitDetEmbeddings.get_absolute_positionsAs$ !3AqrE!: )//2 499\*+ $;, &TU U 99   !dfn %']]%>%>"**1dD"=EEaAqQe_# &?& "*11!Q1= =%--aC Cr. pixel_valuesreturncz|jd}||jk7rtd|jd|d|j|}|jc|j dddd}||j |jd|jd|jdz}|j dddd}|S) NrzoMake sure that the channel dimension of the pixel values match with the one set in the configuration. Expected z but got .rr1rT)r6rr:r(r&r@rG)r)rHr embeddingss r-forwardzVitDetEmbeddings.forwardgs#))!, 4,, ,!../yaI __\2  # # /#++Aq!Q7J#d&A&A(($ 0@0@0CZEUEUVWEX'J$++Aq!Q7Jr.) __name__ __module__ __qualname____doc__rrGr$TensorrM __classcell__r,s@r-rr%s0 j,$DLELLU\\r.rcTtdt||zdz }|jd|k7rttjj |j d|jddjddd|d}|j d|jdd}n|}tj|dddft||z dz}tj|dddft||z dz}||z |dz t||z dzz}||jS) a Get relative positional embeddings according to the relative positions of query and key sizes. Args: q_size (`int`): Size of query q. k_size (`int`): Size of key k. rel_pos (`torch.Tensor`): Relative position embeddings (num_embeddings, num_channels). Returns: Extracted positional embeddings according to relative positions. r1rrr0linear)r3r4N?) r7maxr6rr=r>r?r@r$arangelong)q_sizek_sizerel_pos max_rel_distrel_pos_resizedq_coordsk_coordsrelative_coordss r- get_rel_posrc}s% q3vv..23L}}Q<'--33 OOAw}}Q/ 4 < bhwkzbhwc,wkc->bhwkN)rcr6r?r$einsumview)attnqueries rel_pos_h rel_pos_wr[r\queries_height queries_width keys_height keys_widthrelative_heightrelative_width batch_size_dimr_qrelative_weights r-!add_decomposed_relative_positionsrvs,%+!NM$K!.+yIO  IFN J3 //*nmS ICll#3S/JOll#3S.IO *nm[*U !Q1d* + , !Q4* + , d:~ 5{Z7OP  Kr.c,eZdZdZdfd ZddZxZS)VitDetAttentionz=Multi-head Attention block with relative position embeddings.c t||j}|j}||_||z}|dz|_t j||dz|j|_ t j|||_ |j|_ |jrot jtjd|dzdz ||_t jtjd|dzdz ||_yy)z Args: config (`VitDetConfig`): Model configuration. input_size (`tuple[int]`, *optional*): Input resolution, only required in case relative position embeddings are added. grbiasr1rrN)rrrnum_attention_heads num_headsscalerLinearqkv_biasqkvproj use_relative_position_embeddingsr#r$r%rirj)r)r* input_sizersr}head_dimr,s r-rzVitDetAttention.__init__s   .. ")#t^ 99S#'@IIc3' 060W0W-  0 0\\%++a*Q-6G!6KX*VWDN\\%++a*Q-6G!6KX*VWDN 1r.c 0|j\}}}}|j|j|||zd|jdj ddddd}|jd||jz||zdj d\}} } ||j z| jddz} |jr(t| ||j|j||f||f} | jd} | | z}|j||j||d}|j ddddd}|j|||d}|j|}|r>| j||j| jd| jd} || f} | S|f} | S) Nrr0r1rr)rs)r6rr?r}r@unbindr~ transposerrvrirjsoftmaxrfr)r) hidden_stateoutput_attentionsrqrCrDrrrrhkeysvaluesattention_scoresattention_probsoutputss r-rMzVitDetAttention.forwards'3'9'9$ FE1hh|$,,Z%DNN\^_gghiklnoqrtuv # AzDNN/JFUZN\^ _ f fgh iv#djj0DNN2r4JJ  0 0@ '4>>4>>FTY?]cej\k  +22r2:&/ #((T^^VUTVW #++Aq!Q: #++JrJ yy. -55DNNO,A,A",EG\G\]_G`O$_5G$oGr.N)FrNrOrPrQrrMrSrTs@r-rxrxsGX4r.rxinput drop_probtrainingrIc|dk(s|s|Sd|z }|jdfd|jdz zz}|tj||j|j z}|j |j||z}|S)aF Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). Comment by Ross Wightman: This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. rr)r)dtypedevice)r6ndimr$randrrfloor_div)rrr keep_probr6 random_tensoroutputs r- drop_pathr sCx I I [[^  Q 7 7E 5 ELL YYM YYy !M 1F Mr.cxeZdZdZd deeddffd ZdejdejfdZ de fdZ xZ S) VitDetDropPathzXDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).NrrIc0t|||_yr)rrr)r)rr,s r-rzVitDetDropPath.__init__"s "r. hidden_statescDt||j|jSr)rrr)r)rs r-rMzVitDetDropPath.forward&s FFr.c d|jS)Nzp=)rr)s r- extra_reprzVitDetDropPath.extra_repr)sDNN#$$r.r) rNrOrPrQrfloatrr$rRrMstrrrSrTs@r-rrsGb#(5/#T#GU\\GellG%C%r.rc*eZdZdZdfd ZdZxZS)VitDetLayerNormaL A LayerNorm variant, popularized by Transformers, that performs point-wise mean and variance normalization over the channel dimension for inputs that have shape (batch_size, channels, height, width). https://github.com/facebookresearch/ConvNeXt/blob/d1fa8f6fef0a165b27399986cc2bdacc92777e40/models/convnext.py#L119 ct|tjt j ||_tjt j||_||_ |f|_ yr) rrrr#r$onesweightr%r{epsnormalized_shape)r)rrr,s r-rzVitDetLayerNorm.__init__4sT ll5::.>#?@ LL-=!>? !1 3r.c|jdd}||z jdjdd}||z tj||jzz }|j ddddf|z|j ddddfz}|S)NrT)keepdimr1)meanpowr$r9rrr{)r)xuss r-rMzVitDetLayerNorm.forward;s FF1dF # UKKN  4  0 UejjTXX. . KK4 & *TYYq$}-E Er.)gư>rrTs@r-rr-s 4r.rc(eZdZdZfdZdZxZS)VitDetResBottleneckBlockz The standard bottleneck residual block without the last activation layer. It contains 3 conv layers with kernels 1x1, 3x3, 1x1. ct|tj||dd|_t ||_t|j|_ tj||ddd|_ t ||_ t|j|_ tj||dd|_ t ||_y)ar Args: config (`VitDetConfig`): Model configuration. in_channels (`int`): Number of input channels. out_channels (`int`): Number of output channels. bottleneck_channels (`int`): Number of output channels for the 3x3 "bottleneck" conv layers. rFrzr)paddingr{N)rrrr'conv1rnorm1r hidden_actact1conv2norm2act2conv3norm3)r)r* in_channels out_channelsbottleneck_channelsr,s r-rz!VitDetResBottleneckBlock.__init__Is YY{,?O $%89 6,,- YY24GTU\ab $%89 6,,- YY2L!%P $\2 r.cN|}|jD] }||} ||z}|Sr)children)r)routlayers r-rMz VitDetResBottleneckBlock.forwardas5]]_ E*C #g r.rrTs@r-rrCs 30r.rcdeZdZdededdffd Zdej dej fdZxZS) VitDetMlp in_featureshidden_featuresrINct|tj|||_t |j |_tj|||_tj|j|_ yr) rrrrfc1rractfc2Dropout dropout_probdrop)r)r*rrr,s r-rzVitDetMlp.__init__ksZ 99[/:&++,99_k:JJv223 r.rc|j|}|j|}|j|}|j|}|j|}|Sr)rrrr)r)rs r-rMzVitDetMlp.forwardrsH HHQK HHQK IIaL HHQK IIaLr.) rNrOrPr7rr$rRrMrSrTs@r-rrjs84C4#4$4%,,r.rc `|j\}}}}|||zz |z}|||zz |z}tjj|ddd|d|f}||z||z} }|j |||z|| |z||}|j ddddddj j d|||} | || ffS)a  Partition into non-overlapping windows with padding if needed. Args: hidden_state (`torch.Tensor`): Input tokens with [batch_size, height, width, num_channels]. window_size (`int`): Window size. Returns: `tuple(torch.FloatTensor)` comprising various elements: - windows: windows after partition with [batch_size * num_windows, window_size, window_size, num_channels]. - (padded_height, padded_width): padded height and width before partition rrrr1rr0)r6rr=padrfr@ contiguous) r window_sizerqrCrDr pad_height pad_width padded_height padded_widthwindowss r-window_partitionr|s/;.@.@+J| 44 CJu{22kAI==$$\Aq!Y:3VWL"(:"5uy7H}dN^N^._ +M+!% /"0" /q1 (,   a .}d>N>NP`cikpbqrM!4>>-#@@ %txx =@Y7Z([[ %--aAq9  " " MM-8M "W,r.)rrF)NF)rNrOrPrQrrr7boolrr$rRrrrrMrSrTs@r-rrsMqv!"!49!LO!im! !L-1"' (||(ELL)( ( uU\\5<</0% 2EE F (r.rceZdZdeddffd Z d dej deej deded ede e e ff d Z xZ S) VitDetEncoderr*rINc t|||_|j}t j d|j |dDcgc]}|j}}g}t|D]I}|jt|||||jvr |jnd||jvKtj||_d|_ycc}w)Nrcpu)r)rrrF)rrr*num_hidden_layersr$linspaceritemrangeappendrwindow_block_indicesrresidual_block_indicesr ModuleListrgradient_checkpointing)r)r*depthrrlayersir,s r-rzVitDetEncoder.__init__ s  ((-2NN1f>S>SUZch,ijq!&&(jju A MM#1!#4676;V;V6V 2 2\]'(F,I,I'I   ]]6* &+#ksCrrroutput_hidden_states return_dictc|rdnd}|rdnd}t|jD]1\}} |r||fz}|||nd} | || |} | d}|s)|| dfz}3|r||fz}|std|||fDSt|||S)Nrrc3&K|] }|| ywrr).0vs r- z(VitDetEncoder.forward..?smq_`_lmslast_hidden_stater attentions) enumeraterrr ) r)rrrrrall_hidden_statesall_self_attentionsr  layer_modulelayer_head_mask layer_outputss r-rMzVitDetEncoder.forward#s#7BD$5b4(4 POA|#$58H$H!.7.CilO(IZ[M)!,M &9]1=Mt|tjtjfrtjj |j jjtjd|jjj|j j|j _|j%|jjjyyt|tj rJ|jjj|j jj#dyt|t$rtjj |j&jjtjd|jjj|j&j|j&_yt|t(r|jj*rtjj |j,jjtjd|jj|j,_tjj |j.jjtjd|jj|j._yt|t0r|j2|j4|j6fD] }t9||j:|j<fD]K}|j jj#d|jjjM|j>j jj|j>jjjyy)zInitialize the weightsr)rstdNrW) rrrr'r$ trunc_normal_rdatator$float32r*initializer_rangerr{zero_rfill_rr&rxrrirjrrrrr'rrr)r)rrs r- _init_weightsz#VitDetPreTrainedModel._init_weights]s fryy"))4 5"$!6!6 ""%%emm43DKKDaDa"7"b$$% MM {{&   &&('  - KK   " " $ MM   $ $S )  0 1.0gg.C.C**//225==AKK11/D/b++112  & & +  0T[[5a5a$&GG$9$9  %%((7KK11%:%F   ! %'GG$9$9  %%((7KK11%:%F   !  8 9 ,, fllC ( ' ( ,, 5 ( !!'', %%' ( LL   $ $ * * , LL   " " ( ( *:r.)rNrOrPr__annotations__base_model_prefixmain_input_namesupports_gradient_checkpointing_no_split_modulesrrrr'rr4rr.r-r)r)UsK  $O&*#)+E"))RYY *L$M)+RV)+r.r)ceZdZdeffd ZdefdZdeee efddfdZ e dde e jd e e jd e ed e ed e edeeeff d ZxZS) VitDetModelr*ct||||_t||_t ||_|jyr)rrr*rrLrencoder post_init)r)r*r,s r-rzVitDetModel.__init__s;   *62$V,  r.rIc.|jjSrrLr(rs r-get_input_embeddingsz VitDetModel.get_input_embeddings)))r.heads_to_pruneNc|jD]7\}}|jj|jj |9y)z Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)itemsr=rr prune_heads)r)rCrheadss r- _prune_headszVitDetModel._prune_headssE +002 CLE5 LL  u % / / ; ;E B Cr.rHrrrrc||n|jj}||n|jj}||n|jj}| t d|j ||jj }|j|}|j|||||}|d}|s |f|ddzSt||j|jS)a Examples: ```python >>> from transformers import VitDetConfig, VitDetModel >>> import torch >>> config = VitDetConfig() >>> model = VitDetModel(config) >>> pixel_values = torch.randn(1, 3, 224, 224) >>> with torch.no_grad(): ... outputs = model(pixel_values) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 768, 14, 14] ```Nz You have to specify pixel_values)rrrrrrr) r*rruse_return_dictr: get_head_maskrrLr=r rr) r)rHrrrrembedding_outputencoder_outputssequence_outputs r-rMzVitDetModel.forwards82C1N-TXT_T_TqTq$8$D $++JjJj &1% = 'tn = d^ =  uo% &= = r.r;zF ViTDet backbone, to be used with frameworks like Mask R-CNN. ) custom_introc ~eZdZfdZdefdZe d dejde e de e de e de f dZ xZ S) VitDetBackbonect||t| |t||_t ||_t|jdzDcgc]}|jc}|_ |jycc}w)Nr) rr_init_backbonerrLrr=rrr num_featuresr>)r)r*rrr,s r-rzVitDetBackbone.__init__sq   v&*62$V, 9>v?W?WZ[?[9\]AV//] ^sBrIc.|jjSrr@rs r-rAz#VitDetBackbone.get_input_embeddingsrBr.rHrrrc||n|jj}||n|jj}||n|jj}|j |}|j |d||}|r |j n|d}d}t|j|D]\} } | |jvs|| fz }|s|r |f|ddz} | S|f|ddz} | St||r |j nd|jS)a Examples: ```python >>> from transformers import VitDetConfig, VitDetBackbone >>> import torch >>> config = VitDetConfig() >>> model = VitDetBackbone(config) >>> pixel_values = torch.randn(1, 3, 224, 224) >>> with torch.no_grad(): ... outputs = model(pixel_values) >>> feature_maps = outputs.feature_maps >>> list(feature_maps[-1].shape) [1, 768, 14, 14] ```NT)rrrrrr1) feature_mapsrr) r*rJrrrLr=rzip stage_names out_featuresr r) r)rHrrrrLrrrXstagerrs r-rMzVitDetBackbone.forwards56&1% ; d^ ;  ; ; r.rR)r;r)rR)rF)5rQcollections.abcrr8typingrrr$torch.utils.checkpointr activationsrmodeling_layersrmodeling_outputsr r modeling_utilsr utilsr r utils.backbone_utilsrconfiguration_vitdetr get_loggerrNloggerModulerr;script_if_tracingrcrvrxrRrrrrrrrrrrrr'r)r;rR__all__rr.r-rls " !9?-,1.   H %UryyUp!3!3H&R;bii;~U\\eTV[VbVb* %RYY %bii,$ryy$N $2@>N,Nb8 BII8 v *RYY *4 *0+O0+0+fT 'T T n K *MK  K \ Er.