rh~dZddlmZmZmZddlZddlZddlmZddlm Z m Z m Z ddl m Z ddlmZdd lmZmZmZmZmZmZdd lmZmZdd lmZmZmZdd lmZm Z m!Z!d dl"m#Z#e!jHe%Z&ejNZ(GddejRZ* d@dejRdejVdejVdejVdeejVde,de,deejVfdZ-GddejRZ.GddejRZ/GddejRZ0Gd d!ejRZ1Gd"d#ejRZ2Gd$d%eZ3Gd&d'ejRZ4Gd(d)ejRZ5Gd*d+ejRZ6Gd,d-ejRZ7Gd.d/ejRZ8eGd0d1eZ9eGd2d3e9Z:eGd4d5e9Z;ed67Gd8d9e9Z<ed:7Gd;de9Z>gd?Z?y)AzPyTorch LayoutLM model.)CallableOptionalUnionN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPoolingMaskedLMOutputQuestionAnsweringModelOutputSequenceClassifierOutputTokenClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel)apply_chunking_to_forward find_pruneable_heads_and_indicesprune_linear_layer)auto_docstringcan_return_tuplelogging)LayoutLMConfigc4eZdZdZfdZ ddZxZS)LayoutLMEmbeddingszGConstruct the embeddings from word, position and token_type embeddings.ct|tj|j|j |j |_tj|j|j |_ tj|j|j |_ tj|j|j |_ tj|j|j |_ tj|j|j |_tj|j|j |_t#|j |j$|_tj(|j*|_|j/dt1j2|jj5ddy)N) padding_idxeps position_ids)rF) persistent)super__init__r Embedding vocab_size hidden_size pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingsmax_2d_position_embeddingsx_position_embeddingsy_position_embeddingsh_position_embeddingsw_position_embeddingstype_vocab_sizetoken_type_embeddingsLayoutLMLayerNormlayer_norm_eps LayerNormDropouthidden_dropout_probdropoutregister_buffertorcharangeexpandselfconfig __class__s /var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/layoutlm/modeling_layoutlm.pyr'zLayoutLMEmbeddings.__init__1s[ !||F,=,=v?Q?Q_e_r_rs#%<<0N0NPVPbPb#c %'\\&2S2SU[UgUg%h"%'\\&2S2SU[UgUg%h"%'\\&2S2SU[UgUg%h"%'\\&2S2SU[UgUg%h"%'\\&2H2H&J\J\%]"*6+=+=6CXCXYzz&"<"<=   ELL)G)GHOOPWXej  c||j}n|jdd}|d}| |jn |j}||jddd|f}|&tj|tj |}||j |}|} |j|} |j|dddddf} |j|dddddf} |j|dddddf} |j|dddddf}|j|dddddf|dddddfz }|j|dddddf|dddddfz }|j|}| | z| z| z| z|z|z|z|z}|j|}|j|}|S#t$r}td|d}~wwxYw)Nr$rdtypedevicerr z:The `bbox`coordinate values should be within 0-1000 range.)sizerIr#r=zeroslongr,r.r0r1 IndexErrorr2r3r5r8r;)rA input_idsbboxtoken_type_idsr# inputs_embeds input_shape seq_lengthrIwords_embeddingsr.left_position_embeddingsupper_position_embeddingsright_position_embeddingslower_position_embeddingser2r3r5 embeddingss rDforwardzLayoutLMEmbeddings.forwardBs1  #..*K',,.s3K ^ %.%:!! @T@T  ,,Q  ^ J ! "& '( ((  ( (  ( $  $$ $$ $ ^^J/ \\*- ) bYZ`a a bs,A,F77 G G  G)NNNNN)__name__ __module__ __qualname____doc__r'r\ __classcell__rCs@rDrr.s!Q &  5rErmodulequerykeyvalueattention_maskscalingr; head_maskc .tj||jdd|z} |#|ddddddd|jdf} | | z} tj j | dtjj|j} tj j| ||j} || |jddddz} tj| |} | jddj} | | fS)NrJr r$)dimrH)ptrainingr)r=matmul transposeshaper functionalsoftmaxfloat32torHr;rnview contiguous) rcrdrerfrgrhr;rikwargs attn_weights causal_mask attn_outputs rDeager_attention_forwardr|{s<<s}}Q':;gEL!$Q1o " o%=> #k1 ==((2U]](SVVW\WbWbcL==((6??([L#innQAq&AA ,,|U3K''1-88:K  $$rEc eZdZfdZ ddej deejdeejdeede ej f dZ xZ S) LayoutLMSelfAttentionc$t||j|jzdk7r2t |ds&t d|jd|jd||_|j|_t|j|jz |_|j|jz|_ tj|j|j|_ tj|j|j|_ tj|j|j|_tj|j |_|j |_|jdz|_y)Nrembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()g)r&r'r*num_attention_headshasattr ValueErrorrBintattention_head_size all_head_sizerLinearrdrerfr9attention_probs_dropout_probr;attention_dropoutrhr@s rDr'zLayoutLMSelfAttention.__init__sC     : : :a ?PVXhHi#F$6$6#78 445Q8   #)#=#= #&v'9'9F ! u|| !rEr~cneZdZfdZdej dej dej fdZxZS)LayoutLMSelfOutputc(t|tj|j|j|_tj |j|j|_tj|j|_ yNr!) r&r'rrr*denser8r7r9r:r;r@s rDr'zLayoutLMSelfOutput.__init__s` YYv1163E3EF f&8&8f>S>STzz&"<"<= rEr input_tensorrcr|j|}|j|}|j||z}|SNrr;r8rArrs rDr\zLayoutLMSelfOutput.forward7 =1  ]3 }|'CD rEr]r^r_r'r=rr\rarbs@rDrr1> U\\RWR^R^rErc eZdZfdZdZ d dej deejdeejdee de ej f dZ xZ S) LayoutLMAttentionct|t||_t ||_t |_yr)r&r'r~rAroutputset pruned_headsr@s rDr'zLayoutLMAttention.__init__s0 )&1 (0 ErEc>t|dk(ryt||jj|jj|j \}}t |jj||j_t |jj||j_t |jj||j_ t |jj|d|j_ |jjt|z |j_|jj|jjz|j_ |j j||_y)Nrrrl)lenrrArrrrrdrerfrrrunion)rAheadsindexs rD prune_headszLayoutLMAttention.prune_headss u:? 7 49900$))2O2OQUQbQb u -TYY__eD *499==%@ ,TYY__eD .t{{/@/@%QO )- (E(EE (R %"&))"?"?$))B_B_"_  --33E:rErrgrirrc p|j|f|||d|}|j|d|}|f|ddz}|SN)rgrirrr)rAr) rArrgrirrx self_outputsattention_outputrs rDr\zLayoutLMAttention.forwards_!tyy  )/     ;;|A F#% QR(88rEr) r]r^r_r'rr=rrrrrr\rarbs@rDrrst" ;*7;15,1 ||!!2!23E--.  $D>  u|| rErcVeZdZfdZdej dej fdZxZS)LayoutLMIntermediatect|tj|j|j |_t|jtrt|j|_ y|j|_ yr) r&r'rrr*intermediate_sizer isinstance hidden_actstrr intermediate_act_fnr@s rDr'zLayoutLMIntermediate.__init__s] YYv1163K3KL f'' -'-f.?.?'@D $'-'8'8D $rErrcJ|j|}|j|}|Sr)rrrArs rDr\zLayoutLMIntermediate.forwards& =1 00? rErrbs@rDrrs#9U\\ellrErcneZdZfdZdej dej dej fdZxZS)LayoutLMOutputc(t|tj|j|j |_tj|j |j|_tj|j|_ yr) r&r'rrrr*rr8r7r9r:r;r@s rDr'zLayoutLMOutput.__init__ s` YYv779K9KL f&8&8f>S>STzz&"<"<= rErrrcr|j|}|j|}|j||z}|Srrrs rDr\zLayoutLMOutput.forward&rrErrbs@rDrrrrErc eZdZfdZ d dej deejdeejdeede ej f dZ dZ xZ S) LayoutLMLayerct||j|_d|_t ||_t ||_t||_ y)Nr) r&r'chunk_size_feed_forward seq_len_dimr attentionr intermediaterrr@s rDr'zLayoutLMLayer.__init__/sI '-'E'E$*6208$V, rErrgrirrc |j|f|||d|}|d}|dd}t|j|j|j|} | f|z}|Sr)rrfeed_forward_chunkrr) rArrgrirrxself_attention_outputsrr layer_outputs rDr\zLayoutLMLayer.forward7s"0 " )/ "  " 2!4(,0  # #T%A%A4CSCSUe  /G+rEcL|j|}|j||}|Sr)rr)rArintermediate_outputrs rDrz LayoutLMLayer.feed_forward_chunkPs,"//0@A{{#68HI rEr) r]r^r_r'r=rrrrrr\rrarbs@rDrr.st-7;15,1 ||!!2!23E--.  $D>  u|| 2rErceZdZfdZe d dej deejdeejdee dee dee de e ej e ffd Z xZS) LayoutLMEncoderct|||_tjt |j Dcgc] }t|c}|_d|_ ycc}w)NF) r&r'rBr ModuleListrangenum_hidden_layersrlayergradient_checkpointing)rArBirCs rDr'zLayoutLMEncoder.__init__XsN  ]]5IaIaCb#caM&$9#cd &+#$dsA#rrgriroutput_hidden_states return_dictrc |rdnd}|rdnd} t|jD]4\} } |r||fz}||| nd} | d||| |d|} | d}|s,| | dfz} 6|r||fz}t||| S)N)rrgrirrr)last_hidden_stater attentions) enumeraterr )rArrgrirrrrxall_hidden_statesall_self_attentionsr layer_modulelayer_head_mask layer_outputss rDr\zLayoutLMEncoder.forward^s#7BD$5b4(4 POA|#$58H$H!.7.CilO(+-)"3   M*!,M &9]1=M & 'tn & d^&  uU\\"O3 4& & rErcVeZdZfdZdej dej fdZxZS)LayoutLMPoolerct|tj|j|j|_tj |_yr)r&r'rrr*rTanh activationr@s rDr'zLayoutLMPooler.__init__s9 YYv1163E3EF '')rErrc\|dddf}|j|}|j|}|S)Nr)rr)rArfirst_token_tensor pooled_outputs rDr\zLayoutLMPooler.forwards6+1a40 #56  6 rErrbs@rDrrs#$ U\\ellrErcVeZdZfdZdej dej fdZxZS)LayoutLMPredictionHeadTransformcht|tj|j|j|_t |jtrt|j|_ n|j|_ tj|j|j|_ yr) r&r'rrr*rrrrr transform_act_fnr8r7r@s rDr'z(LayoutLMPredictionHeadTransform.__init__s{ YYv1163E3EF f'' -$*6+<+<$=D !$*$5$5D !f&8&8f>S>STrErrcl|j|}|j|}|j|}|Sr)rrr8rs rDr\z'LayoutLMPredictionHeadTransform.forwards4 =1 --m< }5 rErrbs@rDrrs$UU\\ellrErc*eZdZfdZdZdZxZS)LayoutLMLMPredictionHeadcHt|t||_t j |j |jd|_t jtj|j|_ |j|j_ y)NF)bias) r&r'r transformrrr*r)decoder Parameterr=rLrr@s rDr'z!LayoutLMLMPredictionHead.__init__sm 8@yy!3!3V5F5FUS LLV->->!?@ !II rEc:|j|j_yr)rrrAs rD _tie_weightsz%LayoutLMLMPredictionHead._tie_weightss II rEcJ|j|}|j|}|Sr)rrrs rDr\z LayoutLMLMPredictionHead.forwards$}5  ]3 rE)r]r^r_r'rr\rarbs@rDrrs &&rErcVeZdZfdZdej dej fdZxZS)LayoutLMOnlyMLMHeadcBt|t||_yr)r&r'r predictionsr@s rDr'zLayoutLMOnlyMLMHead.__init__s 3F;rEsequence_outputrc(|j|}|Sr)r)rArprediction_scoress rDr\zLayoutLMOnlyMLMHead.forwards ,,_=  rErrbs@rDrrs#<!u||! !rErc&eZdZUeed<dZdZdZy)LayoutLMPreTrainedModelrBlayoutlmTcXt|tjrm|jjj d|j j|j%|jjjyyt|tjrz|jjj d|j j|j2|jj|jjyyt|trJ|jjj|jjjdyt|tr%|jjjyy)zInitialize the weightsr)meanstdN?)rrrweightdatanormal_rBinitializer_rangerzero_r(r r6fill_r)rArcs rD _init_weightsz%LayoutLMPreTrainedModel._init_weightss$ fbii ( MM   & &CT[[5R5R & S{{&   &&('  - MM   & &CT[[5R5R & S!!- ""6#5#56<<>.  1 2 KK   " " $ MM   $ $S )  8 9 KK   " " $:rEN)r]r^r_r__annotations__base_model_prefixsupports_gradient_checkpointingrrrErDr r s "&*#%rEr c`eZdZfdZdZdZdZee dde e jde e jde e jde e jd e e jd e e jd e e jd e e d e e de e deeeffdZxZS) LayoutLMModelct||||_t||_t ||_t||_|jyr) r&r'rBrr[rencoderrpooler post_initr@s rDr'zLayoutLMModel.__init__sG   ,V4&v. $V,  rEc.|jjSrr[r,rs rDget_input_embeddingsz"LayoutLMModel.get_input_embeddingss...rEc&||j_yrr#)rArfs rDset_input_embeddingsz"LayoutLMModel.set_input_embeddingss*/'rEc|jD]7\}}|jj|jj |9y)z Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)itemsrrrr)rAheads_to_prunerrs rD _prune_headszLayoutLMModel._prune_headssE +002 CLE5 LL  u % / / ; ;E B CrErOrPrgrQr#rirRrrrrc ||n|jj}| | n|jj} | | n|jj} | | t d|#|j |||j } n!||j dd} n t d| |jn |j} |tj| | }|&tj| tj| }|)tj| dztj| }|jdjd } | j|j } d | z tj|jj z} ||j#dk(rh|jd jd jdjd}|j%|jj&dddd}nB|j#d k(r/|jdjdjd}|jt)|j+j }ndg|jj&z}|j-||||| }|j/|| ||| d}|d }|j1|}t3|||j4|j6S)a bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. Examples: ```python >>> from transformers import AutoTokenizer, LayoutLMModel >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMModel.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "world"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> outputs = model( ... input_ids=input_ids, bbox=bbox, attention_mask=attention_mask, token_type_ids=token_type_ids ... ) >>> last_hidden_states = outputs.last_hidden_state ```NzDYou cannot specify both input_ids and inputs_embeds at the same timer$z5You have to specify either input_ids or inputs_embeds)rIrG)rrJ)rHrr)rOrPr#rQrRT)rirrr)r pooler_outputrr)rBrruse_return_dictr%warn_if_padding_and_no_attention_maskrKrIr=onesrLrM unsqueezerurHfinfominrlr?rnext parametersr[rr rrr)rArOrPrgrQr#rirRrrrrSrIextended_attention_maskembedding_outputencoder_outputsrrs rDr\zLayoutLMModel.forwardsj2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y. Q#..*K  &',,.s3KTU U%.%:!! @T@T  !"ZZ FCN  !"[[EJJvVN <;;{T1FSD"0":":1"="G"G"J"9"<"<4::"<"N#&)@#@EKKPTPZPZD[D_D_"_  }}!#%//2<!>>I??%)' + ,,  #/!5 ' *!, O4 )-')77&11   rE) NNNNNNNNNN)r]r^r_r'r$r&r*rrrr= LongTensorrrrrrr\rarbs@rDrrs1 /0C15+/6:59371559,0/3&*s E,,-s u''(s !!2!23 s !!1!12 s u//0 s E--.s   1 12s $D>s 'tns d^s  u00 1s s rErceZdZddgZfdZdZdZdZee dde e jde e jd e e jd e e jd e e jd e e jd e e jde e jde ede ede edeeeffdZxZS)LayoutLMForMaskedLMzcls.predictions.decoder.biaszcls.predictions.decoder.weightct||t||_t ||_|j yr)r&r'rr rclsr!r@s rDr'zLayoutLMForMaskedLM.__init__}s4  %f- &v. rEcB|jjjSrr r[r,rs rDr$z(LayoutLMForMaskedLM.get_input_embeddings}}''777rEcB|jjjSr)r=rrrs rDget_output_embeddingsz)LayoutLMForMaskedLM.get_output_embeddingssxx##+++rEc||jj_|j|jj_yr)r=rrr)rAnew_embeddingss rDset_output_embeddingsz)LayoutLMForMaskedLM.set_output_embeddingss,'5$$2$7$7!rErOrPrgrQr#rirRlabelsrrrrc r| | n|jj} |j|||||||| | d } | d} |j| }d}|Ft }||j d|jj |j d}t||| j| jS)a2 bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ..., config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]` Examples: ```python >>> from transformers import AutoTokenizer, LayoutLMForMaskedLM >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMForMaskedLM.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "[MASK]"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> labels = tokenizer("Hello world", return_tensors="pt")["input_ids"] >>> outputs = model( ... input_ids=input_ids, ... bbox=bbox, ... attention_mask=attention_mask, ... token_type_ids=token_type_ids, ... labels=labels, ... ) >>> loss = outputs.loss ```NT)rgrQr#rirRrrrrr$losslogitsrr) rBr.r r=rrvr)rrr)rArOrPrgrQr#rirRrFrrrrrr masked_lm_lossloss_fcts rDr\zLayoutLMForMaskedLM.forwards@&1%] 'tn] d^]  un$ %] ] rEr;z LayoutLM Model with a sequence classification head on top (a linear layer on top of the pooled output) e.g. for document image classification tasks such as the [RVL-CDIP](https://www.cs.cmu.edu/~aharley/rvl-cdip/) dataset. ) custom_introcteZdZfdZdZee ddeejdeejdeejdeejdeejdeejd eejd eejd ee d ee d ee de e effdZxZS)!LayoutLMForSequenceClassificationc,t|||j|_t||_t j |j|_t j|j|j|_ |jyr r&r' num_labelsrr rr9r:r;rr* classifierr!r@s rDr'z*LayoutLMForSequenceClassification.__init__i   ++%f- zz&"<"<= ))F$6$68I8IJ rEcB|jjjSrr?rs rDr$z6LayoutLMForSequenceClassification.get_input_embeddingsr@rErOrPrgrQr#rirRrFrrrrc | | n|jj} |j|||||||| | d } | d} |j| } |j | }d}||jj |j dk(rd|j_nl|j dkDrL|jtjk(s|jtjk(rd|j_nd|j_|jj dk(rIt}|j dk(r&||j|j}n|||}n|jj dk(r=t}||jd|j |jd}n,|jj dk(rt}|||}t!||| j"| j$ S) aB bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Examples: ```python >>> from transformers import AutoTokenizer, LayoutLMForSequenceClassification >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMForSequenceClassification.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "world"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> sequence_label = torch.tensor([1]) >>> outputs = model( ... input_ids=input_ids, ... bbox=bbox, ... attention_mask=attention_mask, ... token_type_ids=token_type_ids, ... labels=sequence_label, ... ) >>> loss = outputs.loss >>> logits = outputs.logits ```NT rOrPrgrQr#rirRrrrr regressionsingle_label_classificationmulti_label_classificationr$rH)rBr.r r;rU problem_typerTrHr=rMrr squeezerrvrrrr)rArOrPrgrQr#rirRrFrrrrrrJrIrLs rDr\z)LayoutLMForSequenceClassification.forwards@&1%n 'tnn d^n  u.. /n n rErQa3 LayoutLM Model with a token classification head on top (a linear layer on top of the hidden-states output) e.g. for sequence labeling (information extraction) tasks such as the [FUNSD](https://guillaumejaume.github.io/FUNSD/) dataset and the [SROIE](https://rrc.cvc.uab.es/?ch=13) dataset. cteZdZfdZdZee ddeejdeejdeejdeejdeejdeejd eejd eejd ee d ee d ee de e effdZxZS)LayoutLMForTokenClassificationc,t|||j|_t||_t j |j|_t j|j|j|_ |jyrrSr@s rDr'z'LayoutLMForTokenClassification.__init__rVrEcB|jjjSrr?rs rDr$z3LayoutLMForTokenClassification.get_input_embeddingsr@rErOrPrgrQr#rirRrFrrrrc | | n|jj} |j|||||||| | d } | d} |j| } |j | }d}|>> from transformers import AutoTokenizer, LayoutLMForTokenClassification >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("microsoft/layoutlm-base-uncased") >>> model = LayoutLMForTokenClassification.from_pretrained("microsoft/layoutlm-base-uncased") >>> words = ["Hello", "world"] >>> normalized_word_boxes = [637, 773, 693, 782], [698, 773, 733, 782] >>> token_boxes = [] >>> for word, box in zip(words, normalized_word_boxes): ... word_tokens = tokenizer.tokenize(word) ... token_boxes.extend([box] * len(word_tokens)) >>> # add bounding boxes of cls + sep tokens >>> token_boxes = [[0, 0, 0, 0]] + token_boxes + [[1000, 1000, 1000, 1000]] >>> encoding = tokenizer(" ".join(words), return_tensors="pt") >>> input_ids = encoding["input_ids"] >>> attention_mask = encoding["attention_mask"] >>> token_type_ids = encoding["token_type_ids"] >>> bbox = torch.tensor([token_boxes]) >>> token_labels = torch.tensor([1, 1, 0, 0]).unsqueeze(0) # batch size of 1 >>> outputs = model( ... input_ids=input_ids, ... bbox=bbox, ... attention_mask=attention_mask, ... token_type_ids=token_type_ids, ... labels=token_labels, ... ) >>> loss = outputs.loss >>> logits = outputs.logits ```NTrYrr$rH) rBr.r r;rUrrvrTrrr)rArOrPrgrQr#rirRrFrrrrrrJrIrLs rDr\z&LayoutLMForTokenClassification.forwards|&1%Z 'tnZ d^Z  u++ ,Z Z rEr`ceZdZdfd ZdZee ddeejdeejdeejdeejdeejdeejd eejd eejd eejd ee d ee dee de e effdZxZS)LayoutLMForQuestionAnsweringct|||j|_t||_t j |j|j|_|jy)z has_visual_segment_embedding (`bool`, *optional*, defaults to `True`): Whether or not to add visual segment embeddings. N) r&r'rTrr rrr* qa_outputsr!)rArBhas_visual_segment_embeddingrCs rDr'z%LayoutLMForQuestionAnswering.__init__sU   ++%f- ))F$6$68I8IJ rEcB|jjjSrr?rs rDr$z1LayoutLMForQuestionAnswering.get_input_embeddingsr@rErOrPrgrQr#rirRstart_positions end_positionsrrrrc | | n|jj} |j|||||||| | d } | d}|j|}|j dd\}}|j dj }|j dj }d}|| t|jdkDr|j d}t| jdkDr| j d} |jd}|jd|}| jd|} t|}|||}||| }||zd z }t|||| j| j S) a4 bbox (`torch.LongTensor` of shape `(batch_size, sequence_length, 4)`, *optional*): Bounding boxes of each input sequence tokens. Selected in the range `[0, config.max_2d_position_embeddings-1]`. Each bounding box should be a normalized version in (x0, y0, x1, y1) format, where (x0, y0) corresponds to the position of the upper left corner in the bounding box, and (x1, y1) represents the position of the lower right corner. See [Overview](#Overview) for normalization. Example: In the example below, we prepare a question + context pair for the LayoutLM model. It will give us a prediction of what it thinks the answer is (the span of the answer within the texts parsed from the image). ```python >>> from transformers import AutoTokenizer, LayoutLMForQuestionAnswering >>> from datasets import load_dataset >>> import torch >>> tokenizer = AutoTokenizer.from_pretrained("impira/layoutlm-document-qa", add_prefix_space=True) >>> model = LayoutLMForQuestionAnswering.from_pretrained("impira/layoutlm-document-qa", revision="1e3ebac") >>> dataset = load_dataset("nielsr/funsd", split="train") >>> example = dataset[0] >>> question = "what's his name?" >>> words = example["words"] >>> boxes = example["bboxes"] >>> encoding = tokenizer( ... question.split(), words, is_split_into_words=True, return_token_type_ids=True, return_tensors="pt" ... ) >>> bbox = [] >>> for i, s, w in zip(encoding.input_ids[0], encoding.sequence_ids(0), encoding.word_ids(0)): ... if s == 1: ... bbox.append(boxes[w]) ... elif i == tokenizer.sep_token_id: ... bbox.append([1000] * 4) ... else: ... bbox.append([0] * 4) >>> encoding["bbox"] = torch.tensor([bbox]) >>> word_ids = encoding.word_ids(0) >>> outputs = model(**encoding) >>> loss = outputs.loss >>> start_scores = outputs.start_logits >>> end_scores = outputs.end_logits >>> start, end = word_ids[start_scores.argmax(-1)], word_ids[end_scores.argmax(-1)] >>> print(" ".join(words[start : end + 1])) M. Hamann P. Harper, P. Martinez ```NTrYrrr$r) ignore_indexrJ)rI start_logits end_logitsrr)rBr.r rgsplitr^rwrrKclamprrrr)rArOrPrgrQr#rirRrjrkrrrrrrJrnro total_loss ignored_indexrL start_lossend_losss rDr\z$LayoutLMForQuestionAnswering.forwardsD&1%m 'tnm d^m  u22 3m m rEre)r;rQr`rerr )rN)@r`typingrrrr=torch.utils.checkpointrtorch.nnrrr activationsr modeling_layersr modeling_outputsr rrrrrmodeling_utilsrr pytorch_utilsrrrutilsrrrconfiguration_layoutlmr get_loggerr]loggerr8r6Modulerrfloatr|r~rrrrrrrrrrr rr;rQr`re__all__rrErDrsy,, AA!9Gll>>2   H %LLIIh(,% II% <<% % << % U\\* %  %% %%87BII7v  * *\ 299  RYY %.%R. bii. d RYY  bii$ryy0!"))!%o%%0O +O O du 1u u p ~ (?~  ~ Bj %<j j ZA #:A A H rE