rhZdZddlmZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z mZddlZdd lmZdd lmZmZmZmZmZmZdd lmZdd lmZmZmZm Z m!Z!m"Z"d dl#m$Z$e!jJe&Z'dZ(dZ)gdZ*dZ+dZ,e GddeZ-e GddeZ.e GddeZ/e GddeZ0dMdZ1dNdZ2 dO dPdZ3GddejhjjZ6Gd d!ejhjjZ7Gd"d#ejhjjZ8Gd$d%ejhjjZ9Gd&d'ejhjjZ:Gd(d)ejhjjZ;Gd*d+ejhjjZ<Gd,d-ejhjjZ=Gd.d/ejhjjZ>Gd0d1ejhjjZ?Gd2d3ejhjjZ@Gd4d5ejhjjZAGd6d7eZBd8ZCd9ZDdQd:ZEGd;dejhjjZGed?eCGd@dAeBZHGdBdCejhjjZIGdDdEejhjjZJedFeCGdGdHeBZKedIeCGdJdKeBeZLgdLZMy)RzTF 2.0 Swin Transformer model.) annotationsN)Iterable) dataclass)partial)AnyCallable)ACT2FN)TFPreTrainedModelTFSequenceClassificationLossget_initializerkeraskeras_serializable unpack_inputs) shape_list) ModelOutputadd_code_sample_docstringsadd_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings) SwinConfigrzµsoft/swin-tiny-patch4-window7-224)r1iztabby, tabby catcJeZdZUdZdZded<dZded<dZded<dZded<y) TFSwinEncoderOutputaH Swin encoder's outputs, with potential hidden states and attentions. Args: last_hidden_state (`tf.Tensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. hidden_states (`tuple(tf.Tensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `tf.Tensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(tf.Tensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `tf.Tensor` (one for each stage) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. reshaped_hidden_states (`tuple(tf.Tensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `tf.Tensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Ntf.Tensor | Nonelast_hidden_statetuple[tf.Tensor, ...] | None hidden_states attentionsreshaped_hidden_states) __name__ __module__ __qualname____doc__r__annotations__r r!r"|/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/swin/modeling_tf_swin.pyrrDs72+/'.26M/6/3J,3;?8?r)rcXeZdZUdZdZded<dZded<dZded<dZded<dZ ded <y) TFSwinModelOutputa Swin model's outputs that also contains a pooling of the last hidden states. Args: last_hidden_state (`tf.Tensor` of shape `(batch_size, sequence_length, hidden_size)`): Sequence of hidden-states at the output of the last layer of the model. pooler_output (`tf.Tensor` of shape `(batch_size, hidden_size)`, *optional*, returned when `add_pooling_layer=True` is passed): Average pooling of the last layer hidden-state. hidden_states (`tuple(tf.Tensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `tf.Tensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(tf.Tensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `tf.Tensor` (one for each stage) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. reshaped_hidden_states (`tuple(tf.Tensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `tf.Tensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nrr pooler_outputrr r!r") r#r$r%r&rr'r-r r!r"r(r)r*r,r,esB6+/'.&*M#*26M/6/3J,3;?8?r)r,cheZdZUdZdZded<dZded<dZded<dZded<dZ ded <e d Z y) TFSwinMaskedImageModelingOutputa Swin masked image model outputs. Args: loss (`tf.Tensor` of shape `(1,)`, *optional*, returned when `bool_masked_pos` is provided): Masked image modeling (MLM) loss. reconstruction (`tf.Tensor` of shape `(batch_size, num_channels, height, width)`): Reconstructed pixel values. hidden_states (`tuple(tf.Tensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `tf.Tensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(tf.Tensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `tf.Tensor` (one for each stage) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. reshaped_hidden_states (`tuple(tf.Tensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `tf.Tensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nrlossreconstructionrr r!r"cNtjdt|jS)Nzlogits attribute is deprecated and will be removed in version 5 of Transformers. Please use the reconstruction attribute to retrieve the final output instead.)warningswarn FutureWarningr1selfs r*logitsz&TFSwinMaskedImageModelingOutput.logitss%  ]  """r)) r#r$r%r&r0r'r1r r!r"propertyr8r(r)r*r/r/sS6"D !'+N$+26M/6/3J,3;?8? ##r)r/cXeZdZUdZdZded<dZded<dZded<dZded<dZ ded <y) TFSwinImageClassifierOutputa Swin outputs for image classification. Args: loss (`tf.Tensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Classification (or regression if config.num_labels==1) loss. logits (`tf.Tensor` of shape `(batch_size, config.num_labels)`): Classification (or regression if config.num_labels==1) scores (before SoftMax). hidden_states (`tuple(tf.Tensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `tf.Tensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(tf.Tensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `tf.Tensor` (one for each stage) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. reshaped_hidden_states (`tuple(tf.Tensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `tf.Tensor` (one for the output of the embeddings + one for the output of each stage) of shape `(batch_size, hidden_size, height, width)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs reshaped to include the spatial dimensions. Nrr0r8rr r!r") r#r$r%r&r0r'r8r r!r"r(r)r*r;r;sA6"D !#F #26M/6/3J,3;?8?r)r;c t|\}}}}tj||||z|||z||f}tj|d}tj|d|||f}|S)z2 Partitions the given input into windows. rrr )rtfreshape transpose) input_feature window_size batch_sizeheightwidth num_channelswindowss r*window_partitionrLst/9.G+J|JJ V{*K+9M{\hiMll=*<=Gjj2{K"NOG Nr)c ttj|d}tj||z||zz tj}tjj ||}tj ||||z||z||df}tj|d}tj ||||df}|S)z? Merges windows to produce higher resolution features. rrAr=)rBshapecastint32mathfloordivrCrD)rKrFrHrIxyrGs r*window_reverserUs !A + ";)rkepsilondropoutr()super__init__TFSwinPatchEmbeddingsri num_patches grid_size patch_grid embed_dimuse_mask_tokenuse_absolute_embeddingsrlayersLayerNormalizationrlDropouthidden_dropout_probroconfig)r7r}rwkwargs __class__s r*rqzTFSwinEmbeddings.__init__s "6" 5fCU V00<<//99)),'-'E'E$LL333N ||++F,F,FY+W  r)c|jr'|jdd|jfdd|_nd|_|jr4|jd|j dz|jfdd|_nd|_|jryd|_t|ddMtj|jj5|jjddddt|d ddtj|jj5|jjdd|jjgdddt|d dNtj|j j5|j jddddyy#1swYxYw#1swYqxYw#1swYyxYw) Nrzeros mask_tokenrN initializerrkpositional_embeddings)rrkTrirlro)rw add_weightrvrrxrsposition_embeddingsbuiltgetattrrB name_scoperirkbuildrlr}ror7rbs r*rzTFSwinEmbeddings.builds   "ooQ4>>4JX_frosDO"DO  ' ''+D$$q($..9wUl(7(D $(,D $ ::  4+T 2 >t4499: 2%%++D1 2 4 & 2tyy~~. E tT[[-B-B CD E 4D ) 5t||001 ) ""4( ) ) 6  2 2 E E ) )s$ F=&3G G=G GGc|j||\}}|j||}t|\}}}||tj|j |d} tj| |d} tj |d} tj| | j} |d| z z| | zz}|j||jz}|j||}||fS)Nr_rrrArX) rirlrrBrepeatr expand_dimsrOdtyperro) r7 pixel_valuesbool_masked_posr_ embeddingsoutput_dimensionsrGseq_len_ mask_tokensmasks r*callzTFSwinEmbeddings.call6s)-(=(=lU](=(^% %YYzHY= !+J!7 GQ  &))DOOZCK))K!>/26D774!2!23D#sTz2[45GGJ  # # /#d&>&>>J\\*x\@ ,,,r)F)r}rrwboolreturnNonerbtf.TensorShaperr)NF)r tf.Tensorr bool | Noner_rr!tuple[tf.Tensor, tuple[int, int]])r#r$r%r&rqrr __classcell__rs@r*rgrg s> )6^c-%-8C-VZ- *-r)rgc<eZdZdZfdZddZdddZd dZxZS) rrz# Image to Patch Embedding. c Bt|di||j|j}}|j|j }}t |tjjr|n||f}t |tjjr|n||f}|d|dz|d|dzz}||_||_||_||_ |d|dz|d|dzf|_ tjj||j|jdd|_y)Nrrvalid projection)filters kernel_sizestridespaddingrkr()rprq image_size patch_sizerJrv isinstance collectionsabcrrsrtrryConv2Dr) r7r}r~rrrJ hidden_sizersrs r*rqzTFSwinPatchEmbeddings.__init__Ss "6"!'!2!2F4E4EJ $*$7$79I9Ik #-j+//:R:R#SZZdfpYq #-j+//:R:R#SZZdfpYq !!} 1 5*Q-:VW=:XY $$(&$Q-:a=8*Q-:VW=:XY,,--OO . r)cR||jdzdk7r>dddd|jd||jdzz ff}tj||}||jdzdk7r>ddd|jd||jdzz fdf}tj||}|S)Nrrrr)rrBpad)r7rrHrI pad_valuess r* maybe_padzTFSwinPatchEmbeddings.maybe_padhs 4??1% % * &&1dooa6H5SWSbSbcdSeKe6e2fgJ66, ;L DOOA& &! + &1dooa.@6DOO\]L^C^.^*_aghJ66, ;Lr)ct|\}}}}tjr||jk7r t d|j |||}tj |d}|j||}tj |d}t|\}} }}||f} tj||| df}tj |d}|| fS)NzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.rr>r rrrr rr>rArr>r) rrBexecuting_eagerlyrJ ValueErrorrrDrrC) r7rr_rrJrHrIrrGchannelsrs r*rzTFSwinPatchEmbeddings.callqs)3L)A&<   !ld6G6G&Gw ~~lFEB ||L,? __\H_E \\*l; .8.D+ Hfe#UOZZ Z2,FG \\*i8 ,,,r)c|jryd|_t|dd\tj|jj 5|jj ddd|jgdddyy#1swYyxYw)NTr)rrrBrrrkrrJrs r*rzTFSwinPatchEmbeddings.builds} ::  4t , 8t334 M%%tT49J9J&KL M M 9 M Ms *A??B)rrrHintrIrrrr)rrr_rrrN r#r$r%r&rqrrrrrs@r*rrrrNs *-0Mr)rrcPeZdZdZ d dfd ZddZd d dZddZxZS) TFSwinPatchMergingaB Patch Merging Layer. Args: input_resolution (`tuple[int]`): Resolution of input feature. dim (`int`): Number of input channels. norm_layer (`keras.layer.Layer`, *optional*, defaults to `keras.layers.LayerNormalization`): Normalization layer class. c t|d i|||_||_tj j d|zdd|_|'tj jdd|_ y|d|_ y) Nr>F reduction)use_biasrkrmrlrnrkrjr() rprqinput_resolutiondimrryDenserrzrl)r7rr norm_layerr~rs r*rqzTFSwinPatchMerging.__init__sr "6" 0++AGe++V   7767RDI"/DIr)cz|dzdk(xs|dzdk(}|r&dd|dzfd|dzfdf}tj||}|S)Nr>rrr)rBr)r7rErHrI should_padrs r*rzTFSwinPatchMerging.maybe_padsSqjAo:519>  1fqj/Auqy>6JJFF=*=Mr)c|\}}t|\}}}tj|||||f}|j|||}|ddddddddddf} |ddddddddddf} |ddddddddddf} |ddddddddddf} tj| | | | gd}tj||dd|zf}|j ||}|j ||}|S)Nrr>rrAr?r)rrBrCrconcatrlr) r7rEinput_dimensionsr_rHrIrGrrJinput_feature_0input_feature_1input_feature_2input_feature_3s r*rzTFSwinPatchMerging.calls)( &0&?# A| =:vul2[\ }feD '14a4Aq(89'14a4Aq(89'14a4Aq(89'14a4Aq(89 ?O_Ve"fhjk  JA ,<=  -( C }xH r)c|jryd|_t|dd]tj|jj 5|jj ddd|jzgdddt|dd^tj|jj 5|jj ddd|jzgdddyy#1swYuxYw#1swYyxYw)NTrr?rl) rrrBrrrkrrrlrs r*rzTFSwinPatchMerging.builds ::  4d + 7t~~223 A$$dD!dhh,%?@ A 4 & 2tyy~~. < tQ\ :; < < 3 A A <