rhbdZddlZddlmZddlmZmZddlZddlZddlm Z ddl m Z ddl m Z dd lmZdd lmZmZmZdd lmZdd lmZmZmZdd lmZmZmZmZddlm Z ejBe"Z#eedGddeZ$Gdde jJZ&Gdde jJZ'Gdde jJZ(Gdde jJZ)Gdde jJZ*Gdde jJZ+Gd d!e jJZ,Gd"d#e jJZ-Gd$d%e jJZ.Gd&d'eZ/Gd(d)e jJZ0Gd*d+e jJZ1Gd,d-e jJZ2eGd.d/eZ3eGd0d1e3Z4eGd2d3e3Z5ed4Gd5d6e3Z6ed7Gd8d9e3Z7gd:Z8y);zPyTorch Bros model.N) dataclass)OptionalUnion)nn)CrossEntropyLoss)ACT2FN)GradientCheckpointingLayer)"BaseModelOutputWithCrossAttentions,BaseModelOutputWithPoolingAndCrossAttentionsTokenClassifierOutput)PreTrainedModel)apply_chunking_to_forward find_pruneable_heads_and_indicesprune_linear_layer) ModelOutputauto_docstringcan_return_tuplelogging) BrosConfigz@ Base class for outputs of token classification models. ) custom_introceZdZUdZdZeejed<dZ eejed<dZ eejed<dZ ee ejed<dZ ee ejed<y)BrosSpadeOutputa loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Classification loss. initial_token_logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.num_labels)`): Classification scores for entity initial tokens (before SoftMax). subsequent_token_logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, sequence_length+1)`): Classification scores for entity sequence tokens (before SoftMax). Nlossinitial_token_logitssubsequent_token_logits hidden_states attentions)__name__ __module__ __qualname____doc__rrtorch FloatTensor__annotations__rrrtuplery/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/bros/modeling_bros.pyrr*s~)-D(5$$ %,8<(5#4#45<;?Xe&7&78?8 B||BB2T]]5G5G1aQUQnQnrsQs5tt ))\--/1A1A1CD"Mr)r r!r"r1r$TensorrJ __classcell__r7s@r*r,r,As#3u|| r)r,cVeZdZfdZdej dej fdZxZS)BrosPositionalEmbedding2Dct||j|_t||_t||_yN)r0r1dim_bboxr, x_pos_emb y_pos_embr5r6r7s r*r1z"BrosPositionalEmbedding2D.__init__Ws1  26:26:r)bboxr9cg}t|jD]U}|dzdk(r&|j|j|d|f1|j|j |d|fWt j |d}|S)Nr;r.r<r=)rangerSappendrTrUr$rA)r5rWstacki bbox_pos_embs r*rJz!BrosPositionalEmbedding2D.forward^s|t}}% ;A1uz T^^DaL9: T^^DaL9:  ; yyB/ r)rKrNs@r*rPrPVs#;ELLU\\r)rPc>eZdZfdZdej fdZxZS)BrosBboxEmbeddingsct|t||_t j |j |jd|_y)NF)bias) r0r1rPbbox_sinusoid_embrLineardim_bbox_sinusoid_emb_2ddim_bbox_projectionbbox_projectionrVs r*r1zBrosBboxEmbeddings.__init__js= !:6!B!yy)H)H&JdJdkpqr)rWc|jdd}|dddddddf|dddddddfz }|j|}|j|}|S)Nrr) transposerbrf)r5rWbbox_tbbox_posr]s r*rJzBrosBboxEmbeddings.forwardos\1%$1a-(6!T1a-+@@--h7 ++L9 r)rKrNs@r*r_r_isr ELLr)r_c eZdZdZfdZ d deejdeejdeejdeejdejf dZxZ S) BrosTextEmbeddingszGConstruct the embeddings from word, position and token_type embeddings.cdt|tj|j|j |j |_tj|j|j |_ tj|j|j |_ tj|j |j|_ tj|j|_t#|dd|_|j'dt)j*|jj-d|j'dt)j.|j0j3t(j4|j0j6d y) N) padding_idxepsposition_embedding_typeabsolute position_ids)rr<token_type_idsdtypedeviceF) persistent)r0r1r Embedding vocab_size hidden_size pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingstype_vocab_sizetoken_type_embeddings LayerNormlayer_norm_epsDropouthidden_dropout_probdropoutgetattrrqr4r$r3expandzerosrsr?longrwrVs r*r1zBrosTextEmbeddings.__init__{s8 !||F,=,=v?Q?Q_e_r_rs#%<<0N0NPVPbPb#c %'\\&2H2H&J\J\%]"f&8&8f>S>STzz&"<"<= '.v7PR\']$ ^U\\&:X:X-Y-`-`ah-ij   KK!!&&(jj((//    r) input_idsrtrs inputs_embedsr9cT||j}n|jdd}|d}||jddd|f}|st|dr-|jddd|f}|j |d|}|}n:t j |t j|jj}||j|}|j|} || z} |jdk(r|j|} | | z } |j| } |j| } | S)Nr<rrtrrurr)r?rshasattrrtrr$rrrwr}rrqrrr) r5rrtrsr input_shape seq_lengthbuffered_token_type_ids buffered_token_type_ids_expandedr embeddingsrs r*rJzBrosTextEmbeddings.forwards=  #..*K',,.s3K ^  ,,Q  ^ J"%::  ' ': 5"&":":<"H  - -J^^J/ \\*- r))NNNN) r r!r"r#r1rr$rLrJrMrNs@r*rlrlxsvQ 4-115/304 #ELL)#!.#u||, #   - #  #r)rlceZdZfdZ d dej dej deej deej deej deej deej d eej fd ZxZ S) BrosSelfAttentionct||j|jzdk7r2t |ds&t d|jd|jd|j|_t |j|jz |_|j|jz|_tj|j|j|_ tj|j|j|_ tj|j|j|_ tj|j|_t#|dd|_|j$dk(s|j$d k(rF|j&|_tj(d |j&zd z |j|_|j,|_y) Nrembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()rqrr relative_keyrelative_key_queryr;r)r0r1r{num_attention_headsr ValueErrorintattention_head_size all_head_sizerrcquerykeyvaluerattention_probs_dropout_probrrrqr~rydistance_embedding is_decoderrVs r*r1zBrosSelfAttention.__init__s     : : :a ?PVXhHi#F$6$6#78 445Q8  $*#=#= #&v'9'9F 9T=Y=Y]q=q+1+I+ID (&(ll1v7U7U3UXY3Y[_[s[s&tD # ++r)rr]attention_mask head_maskencoder_hidden_statesencoder_attention_maskoutput_attentionsr9c|jdd|j|jf}|j|j |j dd} |du} | rc|j |j |j dd} |j|j |j dd} |}n`|j |j |j dd} |j|j |j dd} tj| | j dd} |jdk(s|jdk(rF|jd}tj|tj|jj dd}tj|tj|jj dd}||z }|j||j zdz }|j#| j$ }|jdk(rtj&d | |}| |z} nE|jdk(r6tj&d | |}tj&d | |}| |z|z} | j\}}}}|j ||||}|j)gd }tj&d | |f}| |z} | t+j,|jz } || |z} t/j0d| }|j3|}|||z}tj|| }|j)ddddj5}|jdd|j6fz}|j|}|r||fn|f}|j8r|dz}|S)Nrr<rr;rrru)rvzbhld,lrd->bhlrzbhrd,lrd->bhlr)r;rrrzbnid,bijd->bnijr=rrR)shaperrrr@rhrrr$matmulrqr?r3rrwrr~torveinsumpermutemathsqrtrSoftmaxr contiguousrr)r5rr]rrrrr hidden_shape query_layeris_cross_attention key_layer value_layerattention_scoresrposition_ids_lposition_ids_rdistancepositional_embeddingrelative_position_scoresrelative_position_scores_queryrelative_position_scores_key batch_sizen_headd_headbbox_pos_scoresattention_probs context_layernew_context_layer_shapeoutputss r*rJzBrosSelfAttention.forwards&++A.D4L4LdNfNfg jj/44\BLLQPQR 3$> !67<<\JTTUVXYZI**%:;@@NXXYZ\]^K3N/44\BLLQPQRI**]388FPPQRTUVK!<< Y5H5HR5PQ  ' '> 9T=Y=Y]q=q&++-a0J"\\*EJJ}OcOcdiijlnopN"\\*EJJ}OcOcdiijkmopN%6H#'#:#:8dFbFb;bef;f#g #7#:#:ARAR#:#S ++~=+0<<8H+Wk+l(#36N#N --1EE16>NP[]q1r./4||,,7  - 9O _kB %--aAq9DDF "/"4"4"6s";t?Q?Q>S"S* **,CD 6G=/2mM] ??'Gr)NNNNF) r r!r"r1r$rLrr'rJrMrNs@r*rrs,826,08<9=49P||PllP!. P ELL) P ( 5 P!) 6P$ELL1P u|| Pr)rcneZdZfdZdej dej dej fdZxZS)BrosSelfOutputc(t|tj|j|j|_tj |j|j|_tj|j|_ yNro) r0r1rrcr{denserrrrrrVs r*r1zBrosSelfOutput.__init__'s` YYv1163E3EF f&8&8f>S>STzz&"<"<= r)r input_tensorr9cr|j|}|j|}|j||z}|SrRrrrr5rrs r*rJzBrosSelfOutput.forward-7 =1  ]3 }|'CD r)rKrNs@r*rr&1> U\\RWR^R^r)rceZdZfdZdZ d dej dej deej deej deej deej d eed e ej fd Z xZ S) BrosAttentionct|t||_t ||_t |_yrR)r0r1rr5routputset pruned_headsrVs r*r1zBrosAttention.__init__5s0 %f- $V, Er)c>t|dk(ryt||jj|jj|j \}}t |jj||j_t |jj||j_t |jj||j_ t |jj|d|j_ |jjt|z |j_|jj|jjz|j_ |j j||_y)Nrrr=)lenrr5rrrrrrrrrrunion)r5headsindexs r* prune_headszBrosAttention.prune_heads;s u:? 7  II ) ) II ) )     u-TYY__eD *499==%@ ,TYY__eD .t{{/@/@%QO )- (E(EE (R %"&))"?"?$))B_B_"_  --33E:r)rr]rrrrrr9c r|j|||||||}|j|d|} | f|ddz} | S)Nrr]rrrrrrr)r5r) r5rr]rrrrr self_outputsattention_outputrs r*rJzBrosAttention.forwardPs\yy'%)"7#9/!  ;;|A F#% QR(88r)r) r r!r"r1rr$rLrboolr'rJrMrNs@r*rr4s" ;226,08<9=,1||ll!.  ELL)  ( 5 !) 6$D> u|| r)rcVeZdZfdZdej dej fdZxZS)BrosIntermediatect|tj|j|j |_t|jtrt|j|_ y|j|_ yrR) r0r1rrcr{intermediate_sizer isinstance hidden_actstrr intermediate_act_fnrVs r*r1zBrosIntermediate.__init__js] YYv1163K3KL f'' -'-f.?.?'@D $'-'8'8D $r)rr9cJ|j|}|j|}|SrR)rr)r5rs r*rJzBrosIntermediate.forwardrs& =1 00? r)rKrNs@r*rris#9U\\ellr)rcneZdZfdZdej dej dej fdZxZS) BrosOutputc(t|tj|j|j |_tj|j |j|_tj|j|_ yr) r0r1rrcrr{rrrrrrrVs r*r1zBrosOutput.__init__ys` YYv779K9KL f&8&8f>S>STzz&"<"<= r)rrr9cr|j|}|j|}|j||z}|SrRrrs r*rJzBrosOutput.forwardrr)rKrNs@r*rrxrr)rceZdZfdZ d dej dej deejdeejdeejdeejdeed e ej fd Z d Z xZ S) BrosLayercbt||j|_d|_t ||_|j |_|j|_|jr*|j st|dt ||_ t||_ t||_ y)Nrz> should be used as a decoder model if cross attention is added)r0r1chunk_size_feed_forward seq_len_dimr attentionradd_cross_attention Exceptioncrossattentionr intermediaterrrVs r*r1zBrosLayer.__init__s '-'E'E$&v. ++#)#=#=  # #??4&(f ghh"/"7D ,V4 ( r)rr]rrrrrr9c|j|||||}|d} |jr|dd} n|dd} |jrA|?t|drtd|d|j | |||||} | d} | | ddz} t |j |j|j| } | f| z} |jr| d z} | S) N)r]rrrrrr<rz'If `encoder_hidden_states` are passed, z` has to be instantiated with cross-attention layers by setting `config.add_cross_attention=True`)rrrrrrR) rrrrrrfeed_forward_chunkrr) r5rr]rrrrrself_attention_outputsrrcross_attention_outputs layer_outputs r*rJzBrosLayer.forwards%"& %)/ "0" 2!4 ??,Qr2G,QR0G ??4@t-.=dVDde'+&9&9 -#&;'="3 ':' # 7q9  7" ==G0  # #  ( (       /G+ ??'Gr)cL|j|}|j||}|SrR)rr)r5rintermediate_outputrs r*rzBrosLayer.feed_forward_chunks,"//0@A{{#68HI r)r) r r!r"r1r$rLrr%rr'rJrrMrNs@r*rrs )$7;15=A>B,16||6ll6!!2!23 6 E--. 6 ((9(9: 6!)):): ;6$D>6 u|| 6pr)rc.eZdZfdZe d dej dej deejdeejdeejdeejdee d ee d ee d e e ej e ffd Z xZS) BrosEncoderct|||_tjt |j Dcgc] }t|c}|_ycc}wrR) r0r1r6r ModuleListrYnum_hidden_layersrlayer)r5r6_r7s r*r1zBrosEncoder.__init__sC  ]]uVE]E]?^#_!If$5#_` #_sArr]rrrrroutput_hidden_states return_dictr9c \|rdnd} |rdnd} |r|jjrdnd} t|jD]V\} }|r| |fz} ||| nd}||||||||}|d}|s.| |dfz} |jjsN| |dfz} X|r| |fz} t || | | S)Nr(rrrr;)last_hidden_staterrcross_attentions)r6r enumerater r )r5rr]rrrrrr rall_hidden_statesall_self_attentionsall_cross_attentionsr\ layer_modulelayer_head_mask layer_outputss r*rJzBrosEncoder.forwards#7BD$5b4%64;;;Z;Zr`d(4 VOA|#$58H$H!.7.CilO(+)-)&;'="3M*!,M &9]1=MB,1/4&*. ||. ll. !!2!23 . E--. . ((9(9: . !)):): ;. $D>. 'tn. d^.  uU\\"$FF G. . r)rcVeZdZfdZdej dej fdZxZS) BrosPoolerct|tj|j|j|_tj |_yrR)r0r1rrcr{rTanh activationrVs r*r1zBrosPooler.__init__ s9 YYv1163E3EF '')r)rr9c\|dddf}|j|}|j|}|S)Nr)rr)r5rfirst_token_tensor pooled_outputs r*rJzBrosPooler.forwards6+1a40 #56  6 r)rKrNs@r*rr s#$ U\\ellr)rcVeZdZfdZdej dej fdZxZS)BrosRelationExtractorcRt||j|_|j|_|j|_|j |_tj|j |_ tj|j|j|j z|_ tj|j|j|j z|_ tjtjd|j|_y)Nr)r0r1 n_relationsr{backbone_hidden_sizehead_hidden_sizeclassifier_dropout_probrrdroprcrr Parameterr$r dummy_noderVs r*r1zBrosRelationExtractor.__init__s !--$*$6$6! & 2 2'-'E'E$JJt;;< YYt88$:J:JTMbMb:bc 99T668H8H4K`K`8`a,,u{{1d6O6O'PQr)rrc |j|j|}|jjdj d|j dd}t j||gd}|j|j|}|j|j d|j d|j|j}|j|j d|j d|j|j}t j|jdddd|jdddd}|S)Nrraxisr;r)rr(r* unsqueezerepeatr?r$rArr@r$r&rr)r5rr dummy_vecrelation_scores r*rJzBrosRelationExtractor.forward*sjj;!78 OO--a0779>>!;LaP IIy)41= HHTYYy12 !&&   Q !1!1!!4d6F6FH]H] NN9>>!#4innQ6GIYIY[_[p[pq    1a +Y->->q!Q-J r)rKrNs@r*r"r"s$ R5<<ELLr)r"c<eZdZUeed<dZdejfdZy)BrosPreTrainedModelr6brosmodulecV|jj}t|tjrY|j j jd||j%|jj jyyt|tjrf|j j jd||j2|j j |jjyyt|tjrJ|jj j|j j jdyt|tr,tjj|j |yy)zInitialize the weightsr.)meanstdNg?)r8)r6initializer_rangerrrcweightdatanormal_razero_ryrnrfill_r"initr*)r5r5r8s r* _init_weightsz!BrosPreTrainedModel._init_weightsBs&kk++ fbii ( MM   & &CS & 9{{&   &&('  - MM   & &CS & 9!!- ""6#5#56<<>.  - KK   " " $ MM   $ $S )  5 6 GGOOF--3O 77r)N) r r!r"rr&base_model_prefixrModuler@r(r)r*r3r3=s 8BII8r)r3ceZdZdfd ZdZdZdZee dde e jde e jde e jde e jd e e jd e e jd e e jd e e jd e e jde e de e de e de ee jeffdZxZS) BrosModelct||||_t||_t ||_t||_|r t|nd|_ |jy)zv add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer N) r0r1r6rlrr_bbox_embeddingsrencoderrpooler init_weights)r5r6add_pooling_layerr7s r*r1zBrosModel.__init__XsZ   ,V41&9"6* ,=j(4  r)c.|jjSrRrr})r5s r*get_input_embeddingszBrosModel.get_input_embeddingshs...r)c&||j_yrRrL)r5rs r*set_input_embeddingszBrosModel.set_input_embeddingsks*/'r)c|jD]7\}}|jj|jj |9y)z Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)itemsrGr rr)r5heads_to_pruner rs r* _prune_headszBrosModel._prune_headsnsE +002 CLE5 LL  u % / / ; ;E B Cr)rrWrrtrsrrrrrr rr9c T| | n|jj} | | n|jj} | | n|jj} | | t d||j } n!||j dd} n t d| t d| \}}| |j n |j }|tj| |}|pt|jdr4|jjddd|f}|j||}|}n&tj| tj|}|j|| |}|jj rE|C|j \}}}||f}| tj||} |j#| }nd}|j%||jj&}|j|||| }|j(dd k(r|ddddgd f}||jj*z}|j-|}|j/||||||| | d }|d}|j0|j1|nd}t3|||j4|j6|j8S)a bbox ('torch.FloatTensor' of shape '(batch_size, num_boxes, 4)'): Bounding box coordinates for each token in the input sequence. Each bounding box is a list of four values (x1, y1, x2, y2), where (x1, y1) is the top left corner, and (x2, y2) is the bottom right corner of the bounding box. Examples: ```python >>> import torch >>> from transformers import BrosProcessor, BrosModel >>> processor = BrosProcessor.from_pretrained("jinho8345/bros-base-uncased") >>> model = BrosModel.from_pretrained("jinho8345/bros-base-uncased") >>> encoding = processor("Hello, my dog is cute", add_special_tokens=False, return_tensors="pt") >>> bbox = torch.tensor([[[0, 0, 1, 1]]]).repeat(1, encoding["input_ids"].shape[-1], 1) >>> encoding["bbox"] = bbox >>> outputs = model(**encoding) >>> last_hidden_states = outputs.last_hidden_state ```NzDYou cannot specify both input_ids and inputs_embeds at the same timer<z5You have to specify either input_ids or inputs_embedszYou have to specify bbox)rwrtru)rrsrtr)rrr;rr;rrrT)r]rrrrrr rr)r pooler_outputrrr)r6rr use_return_dictrr?rwr$onesrrrtrrrget_extended_attention_maskrinvert_attention_mask get_head_maskr r bbox_scalerFrGrHr rrr)r5rrWrrtrsrrrrrr rrrrrwrrextended_attention_maskencoder_batch_sizeencoder_sequence_lengthr encoder_hidden_shapeencoder_extended_attention_maskembedding_output scaled_bboxbbox_position_embeddingsencoder_outputssequence_outputr s r*rJzBrosModel.forwardvsP2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "#..*K  &',,.s3KTU U <78 8!, J%.%:!! @T@T  !"ZZ FCN  !t(89*.//*H*HKZK*X'3J3Q3QR\^h3i0!A!&[ SY!Z150P0PQ_alnt0u ;; ! !&;&G=R=W=W=Y :  7$68O#P %-).4HQW)X&.2.H.HI_.` +.2 +&&y$++2O2OP ??%)' +  ::b>Q 1667DT[[333 #'#7#7 #D ,, 12"7#B/!5'  *!,8< 8O O4UY ;-')77&11,==   r))T NNNNNNNNNNNN)r r!r"r1rMrOrSrrrr$rLrrr'r rJrMrNs@r*rDrDVsZ /0C-1'+1515/3,0048<9=,0/3&*} ELL)} u||$} !. } !. } u||, } ELL)}   -}  ( 5} !) 6} $D>} 'tn} d^}  uU\\"$PP Q} } r)rDceZdZdgZfdZee ddeejdeejdeejdeejdeejdeejd eejd eejd eejd ee d ee dee de e eje ffdZxZS)BrosForTokenClassificationrHc`t|||j|_t||_t |dr |j n |j}tj||_ tj|j|j|_ |jyNclassifier_dropout)r0r1 num_labelsrDr4rrlrrrrrcr{ classifierrIr5r6rlr7s r*r1z#BrosForTokenClassification.__init__s   ++f% )09M)NF % %TZTnTn zz"45 ))F$6$68I8IJ r)rrWrbbox_first_token_maskrtrsrrlabelsrr rr9c | | n|jj} |j|||||||| | d } | d}|j|}|j |}d}| t }|J|j d}||j d|j|| j d|}n2||j d|j| j d}t||| j| jS)a bbox ('torch.FloatTensor' of shape '(batch_size, num_boxes, 4)'): Bounding box coordinates for each token in the input sequence. Each bounding box is a list of four values (x1, y1, x2, y2), where (x1, y1) is the top left corner, and (x2, y2) is the bottom right corner of the bounding box. bbox_first_token_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to indicate the first token of each bounding box. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. Examples: ```python >>> import torch >>> from transformers import BrosProcessor, BrosForTokenClassification >>> processor = BrosProcessor.from_pretrained("jinho8345/bros-base-uncased") >>> model = BrosForTokenClassification.from_pretrained("jinho8345/bros-base-uncased") >>> encoding = processor("Hello, my dog is cute", add_special_tokens=False, return_tensors="pt") >>> bbox = torch.tensor([[[0, 0, 1, 1]]]).repeat(1, encoding["input_ids"].shape[-1], 1) >>> encoding["bbox"] = bbox >>> outputs = model(**encoding) ```NT) rWrrtrsrrrr rrr<rlogitsrr) r6rWr4rrnrr@rmr rr)r5rrWrrprtrsrrrqrr rrrfrtrloss_fcts r*rJz"BrosForTokenClassification.forward sZ&1%O 'tnO d^O  uU\\"$99 :O O r)ria Bros Model with a token classification head on top (initial_token_layers and subsequent_token_layer on top of the hidden-states output) e.g. for Named-Entity-Recognition (NER) tasks. The initial_token_classifier is used to predict the first token of each entity, and the subsequent_token_classifier is used to predict the subsequent tokens within an entity. Compared to BrosForTokenClassification, this model is more robust to serialization errors since it predicts next token from one token. c!eZdZdgZfdZee ddeejdeejdeejdeejdeejdeejd eejd eejd eejd eejd ee dee dee de e eje ffdZxZS)!BrosSpadeEEForTokenClassificationrHc ft||||_|j|_|j|_|j |_t||_t|dr |jn |j}tjtj|tj|j |j tj|tj|j |j|_t#||_|j'yrk)r0r1r6rmr$r{r%rDr4rrlrr Sequentialrrcinitial_token_classifierr"subsequent_token_classifierrIros r*r1z*BrosSpadeEEForTokenClassification.__init__is    ++!--$*$6$6!f% )09M)NF % %TZTnTn  )+ JJ) * IIf((&*<*< = JJ) * IIf((&*;*; < ) %,A+H( r)rrWrrprtrsrrinitial_token_labelssubsequent_token_labelsrr rr9c | | n|jj} |j|||||||| | d }|d}|jddj }|j |jddj }|j ||jd}d|z }|j\}}|j}tj|tj|dgj|gdj}|j|dddddftj |j"j$}tj&||dzj|tj}|j|dddddftj |j"j$}|j)dj}d}| | t+}| j)d} |;|j)d}||j)d|j,|| |}n#||j)d|j,| }| j)d} ||j)d|dz|| |}||z}t/||||j0|j2 S) a> bbox ('torch.FloatTensor' of shape '(batch_size, num_boxes, 4)'): Bounding box coordinates for each token in the input sequence. Each bounding box is a list of four values (x1, y1, x2, y2), where (x1, y1) is the top left corner, and (x2, y2) is the bottom right corner of the bounding box. bbox_first_token_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to indicate the first token of each bounding box. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. initial_token_labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for the initial token classification. subsequent_token_labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for the subsequent token classification. Examples: ```python >>> import torch >>> from transformers import BrosProcessor, BrosSpadeEEForTokenClassification >>> processor = BrosProcessor.from_pretrained("jinho8345/bros-base-uncased") >>> model = BrosSpadeEEForTokenClassification.from_pretrained("jinho8345/bros-base-uncased") >>> encoding = processor("Hello, my dog is cute", add_special_tokens=False, return_tensors="pt") >>> bbox = torch.tensor([[[0, 0, 1, 1]]]).repeat(1, encoding["input_ids"].shape[-1], 1) >>> encoding["bbox"] = bbox >>> outputs = model(**encoding) ```NT rrWrrtrsrrrr rrrr,rwrvr<)rrrrr)r6rWr4rhrr|r}squeezerrwr$rArrr masked_fillfinforvmineyer@rrmrrr)r5rrWrrprtrsrrr~rrr rrlast_hidden_statesrrinv_attention_maskrmax_seq_lengthrwinvalid_token_maskself_token_masksubsequent_token_maskrruinitial_token_losssubsequent_token_losss r*rJz)BrosSpadeEEForTokenClassification.forwardsd&1%  +0G0S')H$8#<#?&" &..B.G.GDOO.\^r%s"&=&B&B2&F #$,',,R!1CDEZ['(=>% ! &(==D!5$;!//))   r)) NNNNNNNNNNNNN)r r!r"rwr1rrrr$rLrrr'rrJrMrNs@r*ryry]sj+4&2-1'+158<15/3,0047;:>,0/3&*o ELL)o u||$o !. o ( 5 o !. o u||,o ELL)o   -o 'u||4o "*%,,!7o $D>o 'tno d^o  uU\\"O3 4o o r)ryz Bros Model with a token classification head on top (a entity_linker layer on top of the hidden-states output) e.g. for Entity-Linking. The entity_linker is used to predict intra-entity links (one entity to another entity). ceZdZdgZfdZee ddeejdeejdeejdeejdeejdeejd eejd eejd eejd ee d ee dee de e eje ffdZxZS)!BrosSpadeELForTokenClassificationrHc@t||||_|j|_|j|_|j |_t||_t|dr |jn |jt||_ |jyrk)r0r1r6rmr$r{r%rDr4rrlrr" entity_linkerrIrVs r*r1z*BrosSpadeELForTokenClassification.__init__s    ++!--$*$6$6!f% &-f6J&K " "QWQkQk26: r)rrWrrprtrsrrrqrr rr9c | | n|jj} |j|||||||| | d } | d}|jddj }|j ||j d}d}| et}|j\}}|j}tj||dzj|tj}|jd}tj|tj |dgtj|gd }|j#|dddddftj$|j&j(}|j#|dddddftj$|j&j(}||jd|dz|| jd|}t+||| j,| j. S) a bbox ('torch.FloatTensor' of shape '(batch_size, num_boxes, 4)'): Bounding box coordinates for each token in the input sequence. Each bounding box is a list of four values (x1, y1, x2, y2), where (x1, y1) is the top left corner, and (x2, y2) is the bottom right corner of the bounding box. bbox_first_token_mask (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to indicate the first token of each bounding box. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. Examples: ```python >>> import torch >>> from transformers import BrosProcessor, BrosSpadeELForTokenClassification >>> processor = BrosProcessor.from_pretrained("jinho8345/bros-base-uncased") >>> model = BrosSpadeELForTokenClassification.from_pretrained("jinho8345/bros-base-uncased") >>> encoding = processor("Hello, my dog is cute", add_special_tokens=False, return_tensors="pt") >>> bbox = torch.tensor([[[0, 0, 1, 1]]]).repeat(1, encoding["input_ids"].shape[-1], 1) >>> encoding["bbox"] = bbox >>> outputs = model(**encoding) ```NTrrrrr<rur,rs)r6rWr4rhrrrrrrwr$rrrr@rArrrrvrr rr)r5rrWrrprtrsrrrqrr rrrrtrrurrrwrmasks r*rJz)BrosSpadeELForTokenClassification.forward sX&1%?Ev{{SUW[G\]D$!//))   r)rgrvrNs@r*rrsT+4& -1'+158<15/3,004)-,0/3&*Y ELL)Y u||$Y !. Y ( 5 Y !. Y u||,Y ELL)Y   -Y &Y $D>Y 'tnY d^Y  uU\\"$99 :Y Y r)r)r3rDriryr)9r#r dataclassesrtypingrrr$torch.utils.checkpointrtorch.nnr activationsr modeling_layersr modeling_outputsr r r modeling_utilsr pytorch_utilsrrrutilsrrrrconfiguration_brosr get_loggerr loggerrrBr,rPr_rlrrrrrrrrr"r3rDriryr__all__r(r)r*rs !" %!9 .llKK*   H %  :k:  :" * &  >>Bi iZ RYY 1BII1j ryy   J*JZ5 "))5 r  BIID8/880^ #^ ^ Ba !4a a HM (;M M ` l (;l  l ^ r)