rh־dZddlmZmZmZddlZddlZddlmZddlm Z m Z m Z ddl m Z ddlmZmZdd lmZdd lmZdd lmZdd lmZdd lmZmZmZmZddlmZm Z ddl!m"Z"ddl#m$Z$m%Z%m&Z&m'Z'm(Z(ddl)m*Z*e'r ddl+m,Z,ddl-m.Z.e(j^e0Z1GddejdZ3 d2dejhdejjdejjdejjdeejjde6de6fdZ7Gdd ejhZ8Gd!d"eZ9e%Gd#d$e Z:Gd%d&e:Z;e%Gd'd(e:Z<Gd)d*e:eZ=e%d+,Gd-d.e:Z>e%Gd/d0e:Z?gd1Z@y)3zPyTorch OPT model.)CallableOptionalUnionN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)Cache DynamicCache)GenerationMixin)AttentionMaskConverter)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputWithPastCausalLMOutputWithPastQuestionAnsweringModelOutput SequenceClassifierOutputWithPast)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)TransformersKwargsauto_docstringcan_return_tupleis_torch_flex_attn_availablelogging) OPTConfig) BlockMask)make_flex_block_causal_maskcxeZdZdZdedeffd Z d dejdedeejffd Z xZ S) OPTLearnedPositionalEmbeddingzN This module learns positional embeddings up to a fixed maximum size. num_embeddings embedding_dimcNd|_t| ||jz|yN)offsetsuper__init__)selfr$r% __class__s w/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/opt/modeling_opt.pyr+z&OPTLearnedPositionalEmbedding.__init__8s$  $++5}Eattention_maskpast_key_values_length position_idsc|8tj|d}||zdz j}|dd|df}t|||j zS)z3`input_ids_shape` is expected to be [bsz x seqlen].Nrdim)torchcumsumlongr*forwardr))r,r0r1r2r-s r.r9z%OPTLearnedPositionalEmbedding.forward>s^   <<A>L(>9A=CCEL'+A+B(BCLw|dkk9::r/)rN) __name__ __module__ __qualname____doc__intr+r6 LongTensorrr9 __classcell__r-s@r.r#r#3s]FsF3F'(37 ;((;!$;u//0 ;;r/r#modulequerykeyvaluer0scalingdropoutc tj||jdd|z}|||z}tjj |dtj j|j}tjj|||j}tj||} | jddj} | |fS)N)r5dtypeptrainingrr() r6matmul transposer functionalsoftmaxfloat32torKrGrN contiguous) rBrCrDrEr0rFrGkwargs attn_weights attn_outputs r.eager_attention_forwardrYPs<<s}}R'<=GL!#n4 ==((2U]](SVVW\WbWbcL==((6??([L,,|U3K''1-88:K  $$r/c(eZdZdZ d dedeeffd Z ddejdee ejdeejdeejd e d eejd e ejeejee ffd Z xZS) OPTAttentionz=Multi-headed attention from 'Attention Is All You Need' paperconfig layer_idxc t|||_|j|_|j |_|j|_|j|_ ||_ |-tjd|jjd|j|j z|_d|_|j|j z|jk7r&t#d|jd|j d|jdz|_t'j(|j|j|j|_t'j(|j|j|j|_t'j(|j|j|j|_t'j(|j|j|j|_y) NzInstantiating z without passing a `layer_idx` is not recommended and will lead to errors during the forward call if caching is used. Please make sure to provide a `layer_idx` when creating this class.Tz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).g࿩bias)r*r+r\ hidden_size embed_dimnum_attention_heads num_headsattention_dropoutrG enable_biasr]logger warning_oncer-r:head_dim is_causal ValueErrorrFrLineark_projv_projq_projout_proj)r,r\r]rVr-s r.r+zOPTAttention.__init__js  ++33// !--"      !8!8 9:,,  $..8  MMDNN *t~~ =MdnnM]$T^^$4B8 }}d* iiTEUEUV iiTEUEUV iiTEUEUV  $..$..tGWGWX r/ hidden_statespast_key_valuer0layer_head_maskoutput_attentionscache_positionreturnc |j\}} } |j||jz} | j|d|j|j j dd} |j|} |j|} | j|d|j|j j dd} | j|d|j|j j dd} |#|j| | |jd|i\} } t}|jjdk7rN|jjdk(r|rtjdnt |jj}||| | | |f|j"sd n |j$d d |\}}|j'|| dj)}|j+|}|sd}||fS) z#Input shape: Batch x Time x ChannelrIrr(Nrueagersdpaz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.?)rGrF)sizerorFviewrdrirPrmrnupdater]rYr\_attn_implementationrgrhrrNrGreshaperUrp)r,rqrrr0rsrtrurVbsztgt_len_ query_states key_states value_statesattention_interfacerXrWs r.r9zOPTAttention.forwards(,,.Wa{{=1DLL@ #((b$..$--PZZ[\^_` [[/ {{=1 __S"dnndmmLVVWXZ[\ #((b$..$--PZZ[\^_`  %'5'<'<L$..;K^:\( $J )@ ;; + +w 6{{//69>O##L '>dkk>^>^&_#$7      % $}}C$,, %  % ! \"))#w;FFH mmK0  LL((r/N)NNNFN)r:r;r<r=rrr>r+r6Tensortupleboolr r9r@rAs@r.r[r[gsG $(!Y!YC=!YL9=1526"'15<)||<)!u||!45<)!. <) "%,,/ <)  <)!.<) u||Xell3Xe_D E<)r/r[cteZdZddedeeffd Z ddejdeejdeejdee ejdee d ee d eejd eejd e e d e ejee ejejfffdZxZS)OPTDecoderLayerr\r]ct||j|_t |||_|j |_|j|_t|j|_ tj|j|j|_tj|j|j |j"|_tj|j |j|j"|_tj|j|j|_y)N)r\r]elementwise_affiner_)r*r+rarbr[ self_attndo_layer_norm_beforerGr activation_function activation_fnr LayerNormlayer_norm_elementwise_affineself_attn_layer_normrlffn_dimrffc1fc2final_layer_norm)r,r\r]r-s r.r+zOPTDecoderLayer.__init__s ++%VyI$*$?$?!~~ #F$>$>?$&LL NNv/S/S% !99T^^V^^&BTBTU99V^^T^^&BTBTU " T^^PVPtPt ur/rqr0rsrrrt use_cacher2rurVrvc |} |jr|j|}|jd|||||||d| \}} tjj ||j |j }| |z}|js|j|}|j} |jd|jd}|} |jr|j|}|j|}|j|}|j|}tjj ||j |j }| |zj| }|js|j|}|f} |r| | fz } | S)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`, *optional*): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). past_key_value (`Tuple(torch.FloatTensor)`, *optional*): cached past key and value projection states cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence.. )rqrrr2r0rsrtrurLrI)rrrrrQrGrNshaperr|rrrrr})r,rqr0rsrrrtrr2rurVresidualself_attn_weightshidden_states_shapeoutputss r.r9zOPTDecoderLayer.forwards<!  $ $ 55mDM,:4>> , ')%)+/) ,  , ( ( --mt||VZVcVc-d  =0 (( 55mDM,11%--b-2D2DR2HI    $ $ 11-@M/ **=9 /  --mt||VZVcVc-d !M1778KL (( 11-@M "  )+ +Gr/r)NNNFFNN)r:r;r<rrr>r+r6rrrr?rr FloatTensorr9r@rAs@r.rrsvyvXc]v(26268<,1$)3715P||P!.P"%,,/ P !u||!45 P $D> PD>Pu//0P!.P-.P u  (51B1BEDUDU1U+V"WW XPr/rc@eZdZUeed<dZdZdgZdZdZ dZ dZ dZ dZ y)OPTPreTrainedModelr\modelTrc|jj}t|tjrY|j j jd||j%|jj jyyt|tjrf|j j jd||j2|j j |jjyyt|tjrJ|j j jd|jj jyy)Nrz)meanstdr{)r\init_std isinstancerrlweightdatanormal_r`zero_ Embedding padding_idxrfill_)r,rBrs r. _init_weightsz OPTPreTrainedModel._init_weights>skk"" fbii ( MM   & &CS & 9{{&   &&('  - MM   & &CS & 9!!- ""6#5#56<<>.  - MM   $ $S ) KK   " " $.r/N)r:r;r<r__annotations__base_model_prefixsupports_gradient_checkpointing_no_split_modules_supports_attention_backend_supports_flash_attn_supports_sdpa_supports_flex_attn_can_compile_fullgraphrrr/r.rr1s? &*#*+"&N! %r/rceZdZdZdeffd Z ddeejdfdejdejde d e f d Z e dejd e d e d ejdejde f dZe ddeej$deejdeejdee deej&dee d ee dee dee deej$deejdeedeeeffdZxZS) OPTDecoderz Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`OPTDecoderLayer`] Args: config: OPTConfig r\c 8t|||j|_|j|_|j|_|j |_|j|_tj|j|j|j |_ t|j |j|_|j|jk7r2tj |j|jd|_nd|_|j|jk7r2tj |j|jd|_nd|_|j&r=|j(s1tj*|j|j,|_nd|_tj0t3|j4Dcgc]}t7||c}|_d|_|j=ycc}w)NFr_r)r])r*r+rG layerdrop pad_token_idrmax_position_embeddingsmax_target_positions vocab_sizerrword_embed_proj_dim embed_tokensr#raembed_positionsrl project_out project_inr_remove_final_layer_normrrr ModuleListrangenum_hidden_layersrlayersgradient_checkpointing post_init)r,r\ir-s r.r+zOPTDecoder.__init__Us  ~~ ))!..$*$B$B! ++LL):):FT]n%>>*'7 MM  ""&,,Q/ !+??AMnell;$$R(%7!; PP +')#))!, Q  KK , , 6*%%**.DD%  E*..I0CCKQZ[Kr/rrrKrc ||jdk(r|}|Stj|j}tj||f|||j }|dk7rtj |d}|tj||j |jddkDz}|ddddddfj|ddd}||j}|jd} |ddddddd| f|ddddddfj|j z} | dk(} |ddddddd| fj| ||ddddddd| f<|S) aM Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. N) fill_valuerKrr)diagonalrrIr)r5r6rrfullrtriuarangerexpandclonerrT masked_fill) r0rrrKrurrVrr mask_length padding_masks r.rz@OPTDecoder._prepare_4d_causal_attention_mask_with_cache_positions>  %.*<*<*>!*C(K*' E*..I** -0Ye\j\q\qK!##jjqA 5<< n>S>STWeWmWmnprsWtt tK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c )6 Aq!\k\12r/ input_ids head_maskrroutput_hidden_states return_dictr2rVrvc ||n|jj}||n|jj}||n|jj}| | n|jj} |du|duz r t d|j r%|jr|rtjdd}||jd|jd}||j|}|r | t}||jnd} | 2tj | | |jdz|j"} |A| |jdz}tj$|jd||j"}|j'||| ||}| 8tj(|d } | |zdz j+} | dd| df} |j-|| | }|j.|j/|}||j1|j"z}|rd nd}|rd nd}t3|gd gD]j\}}| |j5dt7|j8k7s3t d |dt7|j8d|j5ddt;|j8D]g\}}|r||fz }|jr%tj<g}||j>kr?||f|| |||nd|||| d| }|d}|s_||dfz }i|j@|jA|}|jB|jC|}|r||fz }tE||||S)a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(num_hidden_layers, num_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. for padding use -1. [What are position IDs?](../glossary#position-ids) cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. Contrarily to `position_ids`, this tensor is not affected by padding. It is used to update the cache in the correct position and to infer the complete sequence length. Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.FrIrrrr4)r2rrzThe `z` should be specified for z layers, but it is for .)r0r2rsrrrtrrulast_hidden_staterrq attentions)#r\rtrruse_return_dictrkrrNrgrhr}rrr rr6rronesrr7r8rrrTzipr|lenr enumeraterandrrrr)r,rr0rrrrrtrrr2rurVr seq_lengthr pos_embedsrqall_hidden_statesall_self_attns attn_mask mask_nameidx decoder_layerdropout_probability layer_outputss r.r9zOPTDecoder.forwardsP2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK?I   --i8M 0*nO?N?Z?99;`a  !"\\ "2]5H5H5K"KTaThThN  !)M,?,?,BBJ"ZZ (;(;A(> S`SgSghN.. M>?L]   <<A>L(>9A=CCEL'+;+<(<=L)).:JYe)f ?? & OOM:M% m6J6J(KK #7BD0d%( k]$C  Iy$>>#A&3t{{+;<$ {*DSEUDVW%NN,Q/03 #,DKK"8 6 C#!m%55!}}&+jjn#&7) *)3<3H3d."3#-  M*!,M =#3"553 66  , 11-@M    ' ,,];M -!1 1 &+++%   r/)F NNNNNNNNNNN)r:r;r<r=rr+rr6rr rr staticmethodr>rKrrrr?rrrrrr9r@rAs@r.rrMs#y#X#( BellK78BllB B  B  BH4 444{{ 4  4  44l1515,0+/59$(,0/3&*3715u E,,-u !.u ELL) u "% u   1 12 u D>u $D>u 'tnu d^u u//0u !.u -.u  u-- .u u r/rceZdZdeffd ZdZdZdZee dde e jde e jde e jd e eee j efd e e j d e ed e ed e ede ede e jde e jdeedeeeffdZxZS)OPTModelr\cdt||t||_|j yr)r*r+rdecoderrr,r\r-s r.r+zOPTModel.__init__s&  !&)  r/c.|jjSrrrr,s r.get_input_embeddingszOPTModel.get_input_embeddingss||(((r/c&||j_yrrr,rEs r.set_input_embeddingszOPTModel.set_input_embeddingss$) !r/c|jSr)rrs r. get_decoderzOPTModel.get_decoders ||r/rr0rrrrrtrrr2rurVrvc  |||n|jj}||n|jj}||n|jj}| | n|jj} |j d||| ||||||d| d | } t | j| j| j| jS)NT rr0r2rrrrrtrrrurr) r\rtrrrrrrrrqr)r,rr0rrrrrtrrr2rurVdecoder_outputss r.r9zOPTModel.forwards"2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK->(?(F"GH +   1 12 + D>+ $D>+ 'tn+ d^+ u//0+ !.+ -.+  u-- .+ + r/rc!eZdZdgZfdZdZdZdZdZe e dde e jde e jd e e jd e eee j"efd e e j"d e e jd e ede ede ede ede e jde e jdeedeeeffdZxZS)OPTForCausalLMzlm_head.weightct||t||_t j |j |jd|_|jyNFr_) r*r+rrrrlrrlm_headrrs r.r+zOPTForCausalLM.__init__sK  f% yy!;!;V=N=NUZ[  r/cB|jjjSrrrrrs r.r z#OPTForCausalLM.get_input_embeddingszz!!...r/c:||jj_yrr0r"s r.r#z#OPTForCausalLM.set_input_embeddings*/ 'r/c&||j_yrrr)r,rs r. set_decoderzOPTForCausalLM.set_decoders$ r/c.|jjSrr5rs r.r%zOPTForCausalLM.get_decoderszz!!!r/rr0rrrlabelsrrtrrr2rurVrvc  ||n|jj}| | n|jj} | | n|jj} |jj d||| |||||| d| d | }|j |dj}d}|E|j|j}|j||fd|jji| }t|||j|j|jS)an labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, OPTForCausalLM >>> model = OPTForCausalLM.from_pretrained("facebook/opt-350m") >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious. I'm just a little bit of a weirdo." ```NTr'rrlosslogitsrrqrr)r\rtrrrrr.rUrTr loss_functionrrrrqr)r,rr0rrrr8rrtrrr2rurVrr<r;s r.r9zOPTForCausalLM.forward s;R2C1N-TXT_T_TqTq$8$D $++JjJj &1%->(?(F"GH P   1 12 P ))*P D>P $D>P 'tnP d^P u//0P !.P +,P  u,, -P P r/r+a The OPT Model transformer with a sequence classification head on top (linear layer). [`OPTForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). ) custom_introcreZdZdeffd Ze ddeejdeejdeejdee e eje fdeejdeejd ee d ee d ee d ee d eejde eeffdZdZdZxZS)OPTForSequenceClassificationr\ct|||j|_t||_t j |j|jd|_|jyr-) r*r+ num_labelsrrrrlrscorerrs r.r+z%OPTForSequenceClassification.__init__nsT   ++f% YYv994??QVW  r/rr0rrrr8rrtrrr2rvc | | n|jj} |j|||| ||||| |  } | d} |j| }||jdd\}}n|jdd\}}|jj |dk7r t d|jj d}n|||jj k7j|jtj}tj|jd|jtj}||zjd}n.d}tj|jj d |tj||j |f}d}||jj"|j$dk(rd |j_nl|j$dkDrL|j&tj(k(s|j&tj*k(rd |j_nd |j_|jj"d k(rIt-}|j$dk(r&||j/|j/}n|||}n|jj"d k(r=t1}||j3d|j$|j3d}n,|jj"d k(rt5}|||}| s|f| ddz}||f|zS|St7||| j8| j:| j<S)a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). N rr0r2rrrrtrrrr(rz=Cannot handle batch sizes > 1 if no padding token is defined.rI)rrKz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r regressionsingle_label_classificationmulti_label_classificationr:)r\rrrErrrkrTrr6int32rargmaxrgrhr-r: problem_typerDrKr8r>r squeezerr}rrrrqr)r,rr0rrrr8rrtrrr2transformer_outputsrqr<rrlast_non_pad_token non_pad_mask token_indices pooled_logitsr;loss_fctoutputs r.r9z$OPTForSequenceClassification.forwardws*&1%!F!Fr!J !#    >>**+,ZZ  u||Jv}}MOaab   {{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#M$9$9$;V^^=MND#M6:D))-JJ+- 2 22t GUWY))-II,. v6#%(;AB(??F)-)9TGf$ Ev E/ /??-;;*55   r/cB|jjjSrr0rs r.r z1OPTForSequenceClassification.get_input_embeddingsr1r/c:||jj_yrr0r"s r.r#z1OPTForSequenceClassification.set_input_embeddingsr3r/r)r:r;r<rr+rrr6r?rrr)r rrrr9r r#r@rAs@r.rBrB_sLy156:15KO59-1$(,0/3&*37\ E,,-\ !!2!23\ E--. \ "%U->->(?(F"GH \   1 12 \ ))*\ D>\ $D>\ 'tn\ d^\ u//0\  u66 7\ \ |/0r/rBceZdZdeffd Ze ddeejdeejdeejdee e eje fdeejdeejd eejd ee d ee d ee d ee deejde eeffdZdZdZxZS)OPTForQuestionAnsweringr\ct||t||_t j |j d|_|jyr') r*r+rrrrlr qa_outputsrrs r.r+z OPTForQuestionAnswering.__init__s@  f% ))F$>$>B r/rr0rrrstart_positions end_positionsrrtrrr2rvc | | n|jj} |j|||| |||| | |  } | d}|j|}|j dd\}}|j dj }|j dj }d}||t|jdkDr|j d}t|jdkDr|j d}|jd}|jd|j|j}|jd|j|j}t|}|||}|||}||zdz }| s||f| ddz}||f|zS|St|||| j| j S) a Example: ```python >>> from transformers import AutoTokenizer, OPTForQuestionAnswering >>> import torch >>> torch.manual_seed(4) # doctest: +IGNORE_RESULT >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> # note: we are loading a OPTForQuestionAnswering from the hub here, >>> # so the head will be randomly initialized, hence the predictions will be random >>> model = OPTForQuestionAnswering.from_pretrained("facebook/opt-350m") >>> question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet" >>> inputs = tokenizer(question, text, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> answer_start_index = outputs.start_logits.argmax() >>> answer_end_index = outputs.end_logits.argmax() >>> answer_offset = len(tokenizer(question)[0]) >>> predict_answer_tokens = inputs.input_ids[ ... 0, answer_offset + answer_start_index : answer_offset + answer_end_index + 1 ... ] >>> predicted = tokenizer.decode(predict_answer_tokens) >>> predicted ' a nice puppet' ```NrGrrrIr4) ignore_indexr()r; start_logits end_logitsrqr)r\rrr[splitrNrUr r|clamprTrrrrqr)r,rr0rrrr\r]rrtrrr2rOrqr<r`ra total_loss ignored_indexrT start_lossend_lossrUs r.r9zOPTForQuestionAnswering.forwards`&1%)r:r;r<rr+rrr6r?rrr)r rrrr9r r#r@rAs@r.rYrYsey156:15KO596:48$(,0/3&*37_ E,,-_ !!2!23_ E--. _ "%U->->(?(F"GH _   1 12 _ "%"2"23_   0 01_ D>_ $D>_ 'tn_ d^_ u//0_  u22 3_ _ B/0r/rY)r+rrrBrY)rz)Ar=typingrrrr6torch.utils.checkpointrtorch.nnrrr activationsr cache_utilsr r generationrmodeling_attn_mask_utilsrmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrmodeling_utilsrrprocessing_utilsrutilsrrrrrconfiguration_optr!torch.nn.attention.flex_attentionr integrations.flex_attentionr! get_loggerr:rgrr#ModulerfloatrYr[rrrrr+rBrY__all__rr/r.r~s,, AA!.)>B9 G&pp( !;J   H %;BLL;H% II% <<% % << % U\\* %  %%.b)299b)Jb0bJ%%%6` #` F = != = @k 'k \  m0#5m0 m0`o00o0o0d r/