rhxdZddlZddlmZmZmZddlZddlZddlmZddl m Z m Z m Z ddl mZddlmZdd lmZmZmZmZmZmZdd lmZmZmZmZmZdd lmZm Z m!Z!d d l"m#Z#e!jHe%Z&GddejNZ(d@dZ)GddejNZ*GddejNZ+GddejNZ,GddejNZ-GddejNZ.GddejNZ/GddejNZ0Gdd ejNZ1 dAd!ejNd"ejdd#ejdd$ejdd%eejdd&e3d'e3d(eejdfd)Z4Gd*d+ejNZ5Gd,d-ejNZ6Gd.d/eZ7Gd0d1ejNZ8eGd2d3eZ9eGd4d5e9Z:eGd6d7e9Z;ed89Gd:d;e9Z<ed<9Gd=d>e9Z=gd?Z>y)BzPyTorch MarkupLM model.N)CallableOptionalUnion)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPoolingMaskedLMOutputQuestionAnsweringModelOutputSequenceClassifierOutputTokenClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModelapply_chunking_to_forward find_pruneable_heads_and_indicesprune_linear_layer)auto_docstringcan_return_tuplelogging)MarkupLMConfigc*eZdZdZfdZddZxZS)XPathEmbeddingszConstruct the embeddings from xpath tags and subscripts. We drop tree-id in this version, as its info can be covered by xpath. c t||j|_tj|j |jz|j |_tj|j|_ tj|_ tj|j |jzd|j z|_ tjd|j z|j |_tjt!|jDcgc],}tj"|j$|j .c}|_tjt!|jDcgc],}tj"|j(|j .c}|_ycc}wcc}w)N)super__init__ max_depthrLinearxpath_unit_hidden_size hidden_sizexpath_unitseq2_embeddingsDropouthidden_dropout_probdropoutReLU activationxpath_unitseq2_inner inner2emb ModuleListrange Embeddingmax_xpath_tag_unit_embeddingsxpath_tag_sub_embeddingsmax_xpath_subs_unit_embeddingsxpath_subs_sub_embeddings)selfconfig_ __class__s /var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/markuplm/modeling_markuplm.pyr"zXPathEmbeddings.__init__7sZ )))+63P3PSWSaSa3acicucu)v&zz&"<"<= '')$&IIf.K.Kdnn.\^_bhbtbt^t$u!1v'9'9#96;M;MN(* t~~.  VAA6C`C`a ) %*,t~~.  VBBFDaDab * &   s 11G1Gc g}g}t|jD]^}|j|j||dddd|f|j|j||dddd|f`t j |d}t j |d}||z}|j|j|j|j|}|S)Ndim) r0r#appendr3r5torchcatr.r*r,r-)r6xpath_tags_seqxpath_subs_seqxpath_tags_embeddingsxpath_subs_embeddingsixpath_embeddingss r:forwardzXPathEmbeddings.forwardQs " "t~~& eA ! ( ()I)F)Fq)I.YZ\]_`Y`Ja)b c ! ( ()J)G)G)J>Z[]^`aZaKb)c d e!& *?R H % *?R H03HH>>$,,ttG`G`aqGr7s*tu)NN)__name__ __module__ __qualname____doc__r"rH __classcell__r9s@r:rr1s  4 rIrc|j|j}tj|dj ||z|z}|j |zS)a Replace non-padding symbols with their position numbers. Position numbers begin at padding_idx+1. Padding symbols are ignored. This is modified from fairseq's `utils.make_positions`. Args: x: torch.Tensor x: Returns: torch.Tensor rr=)neintr@cumsumtype_aslong) input_ids padding_idxpast_key_values_lengthmaskincremental_indicess r:"create_position_ids_from_input_idsr[dsW << $ ( ( *D <<!4<eZdZdZfdZdZ ddZxZS)MarkupLMEmbeddingszGConstruct the embeddings from word, position and token_type embeddings.cdt|||_tj|j |j |j|_tj|j|j |_ |j|_ t||_ tj|j|j |_tj |j |j"|_tj$|j&|_|j+dt-j.|jj1dd|j|_tj|j|j |j2|_ y)N)rWeps position_ids)rr<F) persistent)r!r"r7rr1 vocab_sizer& pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingsr#rrGtype_vocab_sizetoken_type_embeddings LayerNormlayer_norm_epsr(r)r*register_bufferr@arangeexpandrWr6r7r9s r:r"zMarkupLMEmbeddings.__init__ws:  !||F,=,=v?Q?Q_e_r_rs#%<<0N0NPVPbPb#c )) / 7%'\\&2H2H&J\J\%]"f&8&8f>S>STzz&"<"<=   ELL)G)GHOOPWXej  "..#%<<  * *F,>,>DL\L\$  rIc|jdd}|d}tj|jdz||jzdztj|j }|j dj|S)z We are provided embeddings directly. We cannot infer which are padded so just generate sequential position ids. Args: inputs_embeds: torch.Tensor Returns: torch.Tensor Nr<rdtypedevicer)sizer@rmrWrUrs unsqueezern)r6 inputs_embeds input_shapesequence_lengthras r:&create_position_ids_from_inputs_embedsz9MarkupLMEmbeddings.create_position_ids_from_inputs_embedss$((*3B/ %a.||   q /D4D4D"Dq"HPUPZPZcpcwcw %%a(// <>*::;5::V\7N  !![[44uzzd;'4>>*::;5::V\8N)"66|D $ : :> J00P%(;;>SSVff ^^J/ \\*- rI)NNNNNNr)rJrKrLrMr"ryrHrNrOs@r:r]r]ts,Q 2=& 2rIr]cneZdZfdZdej dej dej fdZxZS)MarkupLMSelfOutputc(t|tj|j|j|_tj |j|j|_tj|j|_ yNr_) r!r"rr$r&denserjrkr(r)r*ros r:r"zMarkupLMSelfOutput.__init__s` YYv1163E3EF f&8&8f>S>STzz&"<"<= rI hidden_states input_tensorreturncr|j|}|j|}|j||z}|SNrr*rjr6rrs r:rHzMarkupLMSelfOutput.forward7 =1  ]3 }|'CD rIrJrKrLr"r@TensorrHrNrOs@r:rr1> U\\RWR^R^rIrcVeZdZfdZdej dej fdZxZS)MarkupLMIntermediatect|tj|j|j |_t|jtrt|j|_ y|j|_ yr) r!r"rr$r&intermediate_sizer isinstance hidden_actstrr intermediate_act_fnros r:r"zMarkupLMIntermediate.__init__s] YYv1163K3KL f'' -'-f.?.?'@D $'-'8'8D $rIrrcJ|j|}|j|}|Sr)rrr6rs r:rHzMarkupLMIntermediate.forwards& =1 00? rIrrOs@r:rrs#9U\\ellrIrcneZdZfdZdej dej dej fdZxZS)MarkupLMOutputc(t|tj|j|j |_tj|j |j|_tj|j|_ yr) r!r"rr$rr&rrjrkr(r)r*ros r:r"zMarkupLMOutput.__init__s` YYv779K9KL f&8&8f>S>STzz&"<"<= rIrrrcr|j|}|j|}|j||z}|Srrrs r:rHzMarkupLMOutput.forwardrrIrrOs@r:rrrrIrcVeZdZfdZdej dej fdZxZS)MarkupLMPoolerct|tj|j|j|_tj |_yr)r!r"rr$r&rTanhr,ros r:r"zMarkupLMPooler.__init__s9 YYv1163E3EF '')rIrrc\|dddf}|j|}|j|}|S)Nr)rr,)r6rfirst_token_tensor pooled_outputs r:rHzMarkupLMPooler.forward s6+1a40 #56  6 rIrrOs@r:rrs#$ U\\ellrIrcVeZdZfdZdej dej fdZxZS)MarkupLMPredictionHeadTransformcht|tj|j|j|_t |jtrt|j|_ n|j|_ tj|j|j|_ yr) r!r"rr$r&rrrrr transform_act_fnrjrkros r:r"z(MarkupLMPredictionHeadTransform.__init__s{ YYv1163E3EF f'' -$*6+<+<$=D !$*$5$5D !f&8&8f>S>STrIrrcl|j|}|j|}|j|}|Sr)rrrjrs r:rHz'MarkupLMPredictionHeadTransform.forwards4 =1 --m< }5 rIrrOs@r:rrs$UU\\ellrIrc*eZdZfdZdZdZxZS)MarkupLMLMPredictionHeadcHt|t||_t j |j |jd|_t jtj|j|_ |j|j_ y)NF)bias) r!r"r transformrr$r&rcdecoder Parameterr@r{rros r:r"z!MarkupLMLMPredictionHead.__init__(sm 8@yy!3!3V5F5FUS LLV->->!?@ !II rIc:|j|j_yr)rrr6s r: _tie_weightsz%MarkupLMLMPredictionHead._tie_weights5s II rIcJ|j|}|j|}|Sr)rrrs r:rHz MarkupLMLMPredictionHead.forward8s$}5  ]3 rI)rJrKrLr"rrHrNrOs@r:rr's &&rIrcVeZdZfdZdej dej fdZxZS)MarkupLMOnlyMLMHeadcBt|t||_yr)r!r"r predictionsros r:r"zMarkupLMOnlyMLMHead.__init__@s 3F;rIsequence_outputrc(|j|}|Sr)r)r6rprediction_scoress r:rHzMarkupLMOnlyMLMHead.forwardDs ,,_=  rIrrOs@r:rr?s#<!u||! !rIrmodulequerykeyvalueattention_maskscalingr* head_maskc .tj||jdd|z} |#|ddddddd|jdf} | | z} tj j | dtjj|j} tj j| ||j} || |jddddz} tj| |} | jddj} | | fS)Nr r<)r>rr)ptrainingr)r@matmul transposeshaper functionalsoftmaxfloat32torrr*rview contiguous) rrrrrrr*rkwargs attn_weights causal_mask attn_outputs r:eager_attention_forwardrJs<<s}}Q':;gEL!$Q1o " o%=> #k1 ==((2U]](SVVW\WbWbcL==((6??([L#innQAq&AA ,,|U3K''1-88:K  $$rIc eZdZfdZ ddej deejdeejdeede ej f dZ xZ S) MarkupLMSelfAttentionc$t||j|jzdk7r2t |ds&t d|jd|jd||_|j|_t|j|jz |_|j|jz|_ tj|j|j|_ tj|j|j|_ tj|j|j|_tj|j |_|j |_|jdz|_y)Nrembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()g)r!r"r&num_attention_headshasattr ValueErrorr7rRattention_head_size all_head_sizerr$rrrr(attention_probs_dropout_probr*attention_dropoutrros r:r"zMarkupLMSelfAttention.__init__gsC     : : :a ?PVXhHi#F$6$6#78 445Q8   #)#=#= #&v'9'9F ! u|| !rIrc eZdZfdZdZ d dej deejdeejdee de ej f dZ xZ S) MarkupLMAttentionct|t||_t ||_t |_yr)r!r"rr6routputset pruned_headsros r:r"zMarkupLMAttention.__init__s0 )&1 (0 ErIc>t|dk(ryt||jj|jj|j \}}t |jj||j_t |jj||j_t |jj||j_ t |jj|d|j_ |jjt|z |j_|jj|jjz|j_ |j j||_y)Nrrr=)lenrr6rrrrrrrrrrunion)r6headsindexs r: prune_headszMarkupLMAttention.prune_headss u:? 7 49900$))2O2OQUQbQb u -TYY__eD *499==%@ ,TYY__eD .t{{/@/@%QO )- (E(EE (R %"&))"?"?$))B_B_"_  --33E:rIrrrrrc p|j|f|||d|}|j|d|}|f|ddz}|SN)rrrrr)r6r) r6rrrrr self_outputsattention_outputrs r:rHzMarkupLMAttention.forwards_!tyy  )/     ;;|A F#% QR(88rIr) rJrKrLr"rr@rrrrr~rHrNrOs@r:rrst" ;*7;15,1 ||!!2!23E--.  $D>  u|| rIrc eZdZfdZ d dej deejdeejdeede ej f dZ dZ xZ S) MarkupLMLayerct||j|_d|_t ||_t ||_t||_ y)Nr) r!r"chunk_size_feed_forward seq_len_dimr attentionr intermediaterrros r:r"zMarkupLMLayer.__init__sI '-'E'E$*6208$V, rIrrrrrc |j|f|||d|}|d}|dd}t|j|j|j|} | f|z}|Sr)r rfeed_forward_chunkrr ) r6rrrrrself_attention_outputsrr layer_outputs r:rHzMarkupLMLayer.forwards"0 " )/ "  " 2!4(,0  # #T%A%A4CSCSUe  /G+rIcL|j|}|j||}|Sr)r r)r6rintermediate_outputrs r:r z MarkupLMLayer.feed_forward_chunks,"//0@A{{#68HI rIr) rJrKrLr"r@rrrrr~rHr rNrOs@r:rrst-7;15,1 ||!!2!23E--.  $D>  u|| 2rIrceZdZfdZe d dej deejdeejdee dee dee de e ej e ffd Z xZS) MarkupLMEncoderct|||_tjt |j Dcgc] }t|c}|_d|_ ycc}w)NF) r!r"r7rr/r0num_hidden_layersrlayergradient_checkpointing)r6r7rFr9s r:r"zMarkupLMEncoder.__init__sN  ]]5IaIaCb#caM&$9#cd &+#$dsA#rrrroutput_hidden_states return_dictrc |rdnd}|rdnd} t|jD]4\} } |r||fz}||| nd} | d||| |d|} | d}|s,| | dfz} 6|r||fz}t||| S)N)rrrrrr)last_hidden_stater attentions) enumeraterr )r6rrrrrrrall_hidden_statesall_self_attentionsrF layer_modulelayer_head_mask layer_outputss r:rHzMarkupLMEncoder.forwards#7BD$5b4(4 POA|#$58H$H!.7.CilO(+-)"3   M*!,M &9]1=M & 'tn & d^&  uU\\"O3 4& & rIrcheZdZUeed<dZdZedee e e jfffd Z xZS)MarkupLMPreTrainedModelr7markuplmclt|tjrm|jjj d|j j|j%|jjjyyt|tjrz|jjj d|j j|j2|jj|jjyyt|tjrJ|jjj|jjjdyt|tr%|jjjyy)zInitialize the weightsr)meanstdN?)rrr$weightdatanormal_r7initializer_rangerzero_r1rWrjfill_r)r6rs r: _init_weightsz%MarkupLMPreTrainedModel._init_weights/s' fbii ( MM   & &CT[[5R5R & S{{&   &&('  - MM   & &CT[[5R5R & S!!- ""6#5#56<<>.  - KK   " " $ MM   $ $S )  8 9 KK   " " $:rIpretrained_model_name_or_pathc*t||g|i|Sr)r!from_pretrained)clsr2 model_argsrr9s r:r4z'MarkupLMPreTrainedModel.from_pretrainedAsw&'D\z\U[\\rI)rJrKrLr__annotations__base_model_prefixr1 classmethodrrrosPathLiker4rNrOs@r:r%r%)sH "%$]HU3PRP[P[K[E\<]]]rIr%ceZdZdfd ZdZdZdZee dde e jde e jde e jde e jd e e jd e e jd e e jd e e jd e e de e de e deeeffdZxZS) MarkupLMModelct||||_t||_t ||_|r t|nd|_|jy)zv add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer N) r!r"r7r]rrencoderrpooler post_init)r6r7add_pooling_layerr9s r:r"zMarkupLMModel.__init__IsM   ,V4&v. 0AnV,t  rIc.|jjSrrrers r:get_input_embeddingsz"MarkupLMModel.get_input_embeddingsYs...rIc&||j_yrrD)r6rs r:set_input_embeddingsz"MarkupLMModel.set_input_embeddings\s*/'rIc|jD]7\}}|jj|jj |9y)z Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)itemsr?rr r)r6heads_to_prunerrs r: _prune_headszMarkupLMModel._prune_heads_sE +002 CLE5 LL  u % / / ; ;E B CrIrVrBrCrrrarrvrrrrc | | n|jj} | | n|jj} | | n|jj} | | t d|#|j |||j } n!||j dd} n t d| |jn |j} |tj| | }|&tj| tj| }|jdjd}|j|j }d |z d z}||jdk(rh|jd jd jdjd}|j!|jj"dddd}nB|jdk(r/|jdjdjd}|jt%|j'j }ndg|jj"z}|j)|||||| }|j+|||| | d}|d }|j,|j-|nd}t/|||j0|j2S)a xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. Examples: ```python >>> from transformers import AutoProcessor, MarkupLMModel >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base") >>> model = MarkupLMModel.from_pretrained("microsoft/markuplm-base") >>> html_string = " Page Title " >>> encoding = processor(html_string, return_tensors="pt") >>> outputs = model(**encoding) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 4, 768] ```NzDYou cannot specify both input_ids and inputs_embeds at the same timer<z5You have to specify either input_ids or inputs_embeds)rsrqrr)rrr*gr)rVrBrCrarrvT)rrrr)r pooler_outputrr)r7rruse_return_dictr%warn_if_padding_and_no_attention_maskrtrsr@r}r{rUrurrrr>rnrnext parametersrr?r@rrr)r6rVrBrCrrrarrvrrrrwrsextended_attention_maskembedding_outputencoder_outputsrrs r:rHzMarkupLMModel.forwardgsN2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y. Q#..*K  &',,.s3KTU U%.%:!! @T@T  !"ZZ FCN  !"[[EJJvVN"0":":1"="G"G"J"9"<"<4::"<"N#&)@#@H"L  }}!#%//2<!>>I??))%)' + ,,  #/!5 ' *!,8< 8O O4UY )-')77&11   rI)T) NNNNNNNNNNN)rJrKrLr"rErGrKrrrr@ LongTensorrrrr~rrHrNrOs@r:r=r=FsJ /0C1559596:59371559,0/3&*c E,,-c !!1!12c !!1!12 c !!2!23 c !!1!12 c u//0c E--.c   1 12c $D>c 'tnc d^c  u00 1c c rIr=c!eZdZfdZee ddeejdeejdeejdeejdeejdeejdeejd eejd eejd eejd ee d ee dee de e eje ffdZ xZS)MarkupLMForQuestionAnsweringct|||j|_t|d|_t j |j|j|_|jyNF)rB) r!r" num_labelsr=r&rr$r& qa_outputsrAros r:r"z%MarkupLMForQuestionAnswering.__init__sU   ++%fF ))F$6$68I8IJ rIrVrBrCrrrarrvstart_positions end_positionsrrrrc | | n|jj} |j||||||||| | d }|d}|j|}|j dd\}}|j dj }|j dj }d}| | t| jdkDr| j d} t| jdkDr| j d} |jd}| jd|| jd|t|}||| }||| }||zd z }t||||j|j S) ae xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. Examples: ```python >>> from transformers import AutoProcessor, MarkupLMForQuestionAnswering >>> import torch >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base-finetuned-websrc") >>> model = MarkupLMForQuestionAnswering.from_pretrained("microsoft/markuplm-base-finetuned-websrc") >>> html_string = " My name is Niels " >>> question = "What's his name?" >>> encoding = processor(html_string, questions=question, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**encoding) >>> answer_start_index = outputs.start_logits.argmax() >>> answer_end_index = outputs.end_logits.argmax() >>> predict_answer_tokens = encoding.input_ids[0, answer_start_index : answer_end_index + 1] >>> processor.decode(predict_answer_tokens).strip() 'Niels' ```NT rBrCrrrarrvrrrrrr<r=) ignore_indexr)loss start_logits end_logitsrr)r7rNr&r[splitsqueezerrrtclamp_rrrr)r6rVrBrCrrrarrvr\r]rrrrrlogitsrbrc total_loss ignored_indexloss_fct start_lossend_losss r:rHz$MarkupLMForQuestionAnswering.forwards`&1%\ 'tn\ d^\  uU\\"$@@ A\ \ rIrWzC MarkupLM Model with a `token_classification` head on top. ) custom_introceZdZfdZee ddeejdeejdeejdeejdeejdeejdeejd eejd eejd ee d ee d ee de e eje ffdZ xZS)MarkupLMForTokenClassificationcdt|||j|_t|d|_|j |j n |j }tj||_ tj|j|j|_ |jyrY)r!r"rZr=r&classifier_dropoutr)rr(r*r$r& classifierrAr6r7rqr9s r:r"z'MarkupLMForTokenClassification.__init__Ds   ++%fF )/)B)B)NF % %TZTnTn zz"45 ))F$6$68I8IJ rIrVrBrCrrrarrvlabelsrrrrc t| | n|jj} |j||||||||| | d } | d}|j|}d}| Ft }||j d|jj | j d}t||| j| jS)a xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`. Examples: ```python >>> from transformers import AutoProcessor, AutoModelForTokenClassification >>> import torch >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base") >>> processor.parse_html = False >>> model = AutoModelForTokenClassification.from_pretrained("microsoft/markuplm-base", num_labels=7) >>> nodes = ["hello", "world"] >>> xpaths = ["/html/body/div/li[1]/div/span", "/html/body/div/li[1]/div/span"] >>> node_labels = [1, 2] >>> encoding = processor(nodes=nodes, xpaths=xpaths, node_labels=node_labels, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**encoding) >>> loss = outputs.loss >>> logits = outputs.logits ```NTr_rr<rargrr) r7rNr&rrrrrZrrr)r6rVrBrCrrrarrvrtrrrrrrrarjs r:rHz&MarkupLMForTokenClassification.forwardRs\&1%L 'tnL d^L  uU\\"N2 3L L rIroz MarkupLM Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks. ceZdZfdZee ddeejdeejdeejdeejdeejdeejdeejd eejd eejd ee d ee d ee de e eje ffdZ xZS)!MarkupLMForSequenceClassificationcnt|||j|_||_t ||_|j |j n |j}tj||_ tj|j|j|_ |jyr)r!r"rZr7r=r&rqr)rr(r*r$r&rrrArss r:r"z*MarkupLMForSequenceClassification.__init__s   ++ %f- )/)B)B)NF % %TZTnTn zz"45 ))F$6$68I8IJ rIrVrBrCrrrarrvrtrrrrc | | n|jj} |j||||||||| | d } | d}|j|}|j |}d}| |jj |j dk(rd|j_nl|j dkDrL| jtjk(s| jtjk(rd|j_nd|j_|jj dk(rIt}|j dk(r&||j| j}n||| }n|jj dk(r=t}||jd|j | jd}n,|jj dk(rt}||| }t!||| j"| j$ S) a xpath_tags_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Tag IDs for each token in the input sequence, padded up to config.max_depth. xpath_subs_seq (`torch.LongTensor` of shape `(batch_size, sequence_length, config.max_depth)`, *optional*): Subscript IDs for each token in the input sequence, padded up to config.max_depth. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoProcessor, AutoModelForSequenceClassification >>> import torch >>> processor = AutoProcessor.from_pretrained("microsoft/markuplm-base") >>> model = AutoModelForSequenceClassification.from_pretrained("microsoft/markuplm-base", num_labels=7) >>> html_string = " Page Title " >>> encoding = processor(html_string, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**encoding) >>> loss = outputs.loss >>> logits = outputs.logits ```NTr_r regressionsingle_label_classificationmulti_label_classificationr<rv)r7rNr&r*rr problem_typerZrrr@rUrRr rerrrrrr)r6rVrBrCrrrarrvrtrrrrrrgrarjs r:rHz)MarkupLMForSequenceClassification.forwardsZ&1%\ 'tn\ d^\  uU\\"$<< =\ \ rIry)rWryror=r%)r)rN)?rMr:typingrrrr@torch.utils.checkpointrtorch.nnrrr activationsr modeling_layersr modeling_outputsr rrrrrmodeling_utilsrrrrrutilsrrrconfiguration_markuplmr get_loggerrJloggerModulerr[r]rrrrrrrrfloatrrrrrr%r=rWrory__all__rrIr:rsi ,, AA!9?>2   H %/ bii/ f 4 __F   299  RYY  RYY  bii$ryy0!"))!$(,% II% <<% % << % U\\* %  %% %%87BII7v* *\%.%R. bii. b]o]]8E +E E Pj #:j j Z ^ %<^  ^ B o (?o  o d rI