rhNddlZddlmZmZddlZddlmZmZmZm Z ddl m Z ddl m Z mZmZmZddlmZmZddlmZmZGd d ZGd d ej0j2ZGd dej0j2ZGddej0j2Z d*dedeej:deej:deedeef dZ Gddej0j2Z!Gddej0j2Z"Gddej0j2Z# d+dedeej:deej:fdZ$ d,de%d ej:d!e%d"e%d#eed$eej:d%ee%d&ed'ed(eej:fd)Z&y)-N)CallableOptional) DynamicCacheEncoderDecoderCache HybridCache StaticCache)GenerationConfig)ALL_MASK_ATTENTION_FUNCTIONS_ignore_causal_mask_sdpa#_is_torch_greater_or_equal_than_2_5prepare_padding_mask)ALL_ATTENTION_FUNCTIONSPreTrainedModel)is_torch_greater_or_equal"is_torch_greater_or_equal_than_2_3cJeZdZdZd dedefdZdZdZdZdZ d Z d d Z y )TorchExportableModuleForVLMa| A wrapper class for exporting Vision-Language Models (VLMs) like SmolVLM2 for ExecuTorch. This class handles the export of three main components: 1. Vision encoder (processes images to visual features) 2. Connector/projector (maps visual features to text embedding space) 3. Text decoder (generates text from combined visual and text tokens) max_batch_size max_cache_lenc||_||_||_|j|_|jj|_|jj |_|jj|_d|_ d|_ d|_ y)a  Initialize the exportable VLM module. Args: model: The VLM (e.g. SmolVLM) model instance max_batch_size: Maximum batch size. Always 1 for ExecuTorch max_cache_len: Maximum cache length for text generation N) modelrrconfig vision_modelvision_encoder connector text_model text_decoderexported_vision_encoderexported_connectorexported_text_decoder)selfrrrs w/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/integrations/executorch.py__init__z$TorchExportableModuleForVLM.__init__+sw ,*ll $kk66..!KK22(,$"&%)"c|jjtjddddtj}dtj j jtj j jdi}tj j |j|f|d|_|jS) z$Export the vision encoder component.idtype pixel_values)rr(Fargsdynamic_shapesstrict) revaltorchrandnfloat32exportDimAUTOr)r"r+r.s r#export_vision_encoderz1TorchExportableModuleForVLM.export_vision_encoderCs   "{{1acG  <<##((<<##(( (-||':':   ) (;( $+++r%c&|jj|jjj}|jjj }|jjj }||z}||z}tjd||tj}ddtjjjii}tjj|j|f|d|_ |jS)zExport the connector component.r'r)image_hidden_statesFr,)rr0r vision_config hidden_size image_size patch_sizer1r2r3r4r5r6r )r"vision_hidden_sizer<r=patches_per_dim num_patchesr9r.s r#export_connectorz,TorchExportableModuleForVLM.export_connector[s "[[66BB[[..99 [[..99 $ 2%7 #kk![:LTYTaTab0!U\\5E5E5J5J1KL#(,,"5"5 NN%') #6# &&&r%c"t|j|j|j|_d}t j d|ft j}t j|t j}t|j|jjj}t jjd|dz }d|id|id}|jj|||d |_|jS) z"Export the text decoder component.)rrrr(r'r)seq_length_dimmaxr input_idscache_positionF)rGrHr.r/)%TorchExportableModuleForDecoderOnlyLMrrrexportable_text_decoderr1zeroslongarangeminr text_configmax_position_embeddingsr4r5r!)r" seq_lengthrGrHmax_seq_length seq_len_dimr.s r#export_text_decoderz/TorchExportableModuleForVLM.export_text_decoderts(M##..,,( $ KKJuzzB j CT//1H1H1`1`all&&'7^a=O&P [) +.  &*%A%A%H%H)) &I& ")))r%c |jdi||jdi||jdi||j|j|j dS)z'Export all components of the VLM model.)rrr)r7rArTrr r!)r"kwargss r#r4z"TorchExportableModuleForVLM.exports`""",V,''   *6*"::00 66  r%cy)a Simplified forward pass for inference with guaranteed non-null input_ids and cache_position. Args: pixel_values: Input images [1, channels, height, width] (optional) input_ids: Text token IDs [1, seq_len] (required - won't be None) cache_position: Cache positions [seq_len] (required - won't be None) Returns: Output with logits for text generation NrV)r"r+rGrHs r#forwardz#TorchExportableModuleForVLM.forwards r%Nc y)a Simplified generate method with guaranteed non-null input_ids. Args: pixel_values: Input images [1, channels, height, width] (optional) input_ids: Initial text tokens [1, seq_len] (required - won't be None) max_new_tokens: Maximum number of tokens to generate do_sample: Whether to use sampling or greedy decoding temperature: Temperature for sampling Returns: Generated sequences NrV)r"r+rGmax_new_tokens do_sample temperaturerWs r#generatez$TorchExportableModuleForVLM.generates r%)r')NN2F?) __name__ __module__ __qualname____doc__intr$r7rArTr4rYr^rVr%r#rr!s?*c*c*0,0'2*>    be r%rceZdZdZ ddededeffd Zdejdejdejfd Z dde ejde ejd e e d e e dejjf d Z e dd ejjdedede dededededefdZxZS)rIa A recipe module designed to make a `PreTrainedModel` exportable with `torch.export`, specifically for decoder-only LM with cache. This module ensures that the exported model is compatible with further lowering and execution in `ExecuTorch`. rrrct|t|jdr|jjdur t dt|jdr*t |jddt||||_n'tjdt||||_tjdttjdtd d|jjj_y) a Initializes the exportable module with `HybridCache`. Args: model (`PreTrainedModel`): The pretrained model to wrap. max_batch_size (int): Maximum batch size for the cache. max_cache_len (int): Maximum sequence length for the cache. Raises: ValueError: If the model is configured with a unsupported cache implementation. use_cacheFz5The model must have caching enabled to be performant. layer_typessliding_windowNzmUsing `StaticCache` for export as `layer_types` is not specified or `sliding_window` is `null` in the config.sdpa_without_vmapsdpa)superr$hasattrrri ValueErrorgetattr$TorchExportableModuleWithHybridCacherlogginginfo$TorchExportableModuleWithStaticCacher registersdpa_mask_without_vmapr_attn_implementation)r"rrr __class__s r#r$z.TorchExportableModuleForDecoderOnlyLM.__init__s" u||[1U\\5K5Ku5TTU U 5<< /GELLJZ\`4a4m=e^UbcDJ LL >e^UbcDJ$--.ACYZ(()<>UV\>]^7J 4r%rGrHreturnc:|jj||S) Forward pass of the module, which is compatible with the ExecuTorch llm runner. Args: input_ids (`torch.Tensor`): Tensor representing current input token id to the module. cache_position (`torch.Tensor`): Tensor representing current input position in the cache. Returns: torch.Tensor: Logits output from the model. )rrY)r"rGrHs r#rYz-TorchExportableModuleForDecoderOnlyLM.forwardszz!!)^<v !<<.. JJ#%;<)#/6T /  r%rpromptr[r\r]top_ktop_prc |j} ||djj|} | j} d} t | j dD]E} | dd| | dzf}t j| gt j|}| ||}| dz } Gt |D]}| ddddf}t j| gt j|}| ||}|r|dkDr||z }n|}|dkDr-|t j||ddk}td ||<|d krt j|d \}}t jt j|d d }||kD}|dddfj|dddf<d|d<|jd||}td ||<t j|d }t j|d}n|j!dd }|j#dkDr|j%d}t j&| |gd } | dz } |j)|j*k(sn|j-| dd S)a Generate a sequence of tokens using an exported program. Args: exported_program (`torch.export.ExportedProgram`): The exported model being used for generate. tokenizer: The tokenizer to use. prompt (str): The input prompt. max_new_tokens (int): Maximum number of new tokens to generate. do_sample (bool): Whether to use sampling or greedy decoding. temperature (float): The temperature for sampling. top_k (int): The number of highest probability tokens to keep for top-k sampling. top_p (float): The cumulative probability for nucleus sampling. device (str): The device to use. Returns: str: The generated text. pt)return_tensorsrr'Nr).rNz-infraT) descendingdim.).r) num_samplesrkeepdimr)skip_special_tokens)modulerGtoclonerangeshaper1rrLtopkfloatsortcumsumsoftmaxscatter multinomialargmaxrsqueezecatitem eos_token_iddecode)r tokenizerrr[r\r]rrrexported_modulerG generated_ids curr_positionicurr_input_idscurr_cache_position_outputslogitsindices_to_remove sorted_logitssorted_indicescumulative_probssorted_indices_to_removeprobs next_token_ids r#r^z.TorchExportableModuleForDecoderOnlyLM.generate*s<+113fT:DDGGO ")  yq)* A&q!a!e)|4N"',, ejjY_"`  0CDA Q M ~&5 A*1bc62N"',, ejjY_"` &n6IJG?${2F$F19(.FE1J11Mm1\(\%05f F,-3;49JJvRV4W1M>',||EMM-UW4X^`'a$0@%/G,8PQTVYWYVYQY8Z8`8`8b,S!"W578,V4)A(H(H^]u(v%05f F,- f"5 % 1 1%Q G !(2t D   "Q& - 5 5b 9 "II}m&D"MM Q M!!#y'='==k5 p a 0dKKr%r'NNNN)Frar`rar)rbrcrdrerrfr$r1TensorrYrdictboolr4ExportedProgram staticmethodstrrr^ __classcell__rys@r#rIrIsk ! "K"K"K "KH=<<= =  =&-115)-!% - ELL)- !.- ! -  -  % % - ^ ! iL,,66iLiL iL  iL  iLiLiLiL iLiLr%rIc eZdZdZ ddededeffd ZdejdejfdZ e d ejjd ejd ed ejfd Z xZS)rua A recipe module designed to make a `PreTrainedModel` exportable with `torch.export`, specifically for decoder-only LM to `StaticCache`. This module ensures that the exported model is compatible with further lowering and execution in `ExecuTorch`. Note: This class is specifically designed to support export process using `torch.export` in a way that ensures the model can be further lowered and run efficiently in `ExecuTorch`. rrrc t||j,t|jj d|||dd|_|jj s t d|jjdk7r t d||_t|jj|jjjjd|jjjjd |jjjjd |jj |_ tt|jD]r}|j!d ||jj"|j$d |j!d||jj"|j&d ty)a Initializes the wrapper module with the pretrained model. Args: model (`PreTrainedModel`): The pretrained model to wrap. The model must have caching enabled and use a 'static' caching implementation. Raises: AssertionError: If the pretrained model does not have caching enabled or if it does not use a 'static' caching implementation in `model.generation_config`. Nstaticr) batch_sizerr)ricache_implementation max_length cache_configzvThe model must have caching enabled to be exported with static caching. Please set `generation_config.use_cache=True`.zThe model must use a 'static' caching implementation to be exported with static caching. Please set `generation_config.cache_implementation='static'`.rrrrrrrr* key_cache_F persistent value_cache_)rnr$generation_configr rriAssertionErrorrrr rgetr* static_cacherlenregister_bufferlayerskeysvaluesr"rrrrrys r#r$z-TorchExportableModuleWithStaticCache.__init__s"   " " *&6,,00%-("0%2# 'E #&&00 A   " " 7 78 C P   '::$$::77DDHHV**66CCGGX:://<<@@J**""  s4,,-. kA  :aS!143D3D3K3KA3N3S3S`e f  <s!3T5F5F5M5Ma5P5W5Wdi j kr%rGrHc|j\}}|jd}|j}|j|d|||d}t |dr |j S|j S)a Forward pass of the module, which is compatible with the ExecuTorch runtime. Args: input_ids (`torch.Tensor`): Tensor representing current input token id to the module. cache_position (`torch.Tensor`): Tensor representing current input position in the cache. Returns: torch.Tensor: Logits output from the model. This forward adapter serves two primary purposes: 1. **Making the Model `torch.export`-Compatible**: The adapter hides unsupported objects, such as the `Cache`, from the graph inputs and outputs, enabling the model to be exportable using `torch.export` without encountering issues. 2. **Ensuring Compatibility with `ExecuTorch` runtime**: The adapter matches the model's forward signature with that in `executorch/extension/llm/runner`, ensuring that the exported model can be executed in `ExecuTorch` out-of-the-box. rNT)rGattention_mask position_idsrHpast_key_valuesrir)r unsqueezerrrorlast_hidden_state)r"rGrHrseqlenrroutss r#rYz,TorchExportableModuleWithStaticCache.forwardsu*OO 6%//2 ++zz%)+   4 ";; )) )r%rprompt_token_idsr[rzc f|j}|jd}||z}|jD]3\}}|jds|jd}t ||}ng} t t ||D]y} |j j|dd| | dzftj| gtj|} | j|d| j{tj dddddfd j} | j| t| |kr|j jtj| ggtj|tjt| gtj|} tj| dddddfd j} | j| t| |krtj| gtj|S) a Generate a sequence of tokens using an exported program. This util function is designed to test exported models by simulating the generation process. It processes the input prompt tokens sequentially (no parallel prefill). This generate function is not intended to replace the original `generate` method, and the support for leveraging the original `generate` is potentially planned! Args: exported_program (`torch.export.ExportedProgram`): The exported program generated via `torch.export`. prompt_token_ids (`torch.Tensor`): Tensor representing the input prompt token IDs. max_new_tokens (`int`): Maximum number of new tokens to generate. Note that the total generation length is limited by both `max_new_tokens` and the model's cache size. Returns: torch.Tensor: A tensor containing the generated sequence of token IDs, including the original prompt tokens. r key_cacherNr'rrFrr)rr named_buffers startswithrNrrrYr1rrLappendrrr) rrr[rprompt_token_lenmax_generation_length buffer_namebufferrresponse_tokens input_posresult current_tokens r#r^z-TorchExportableModuleWithStaticCache.generates."((+11"5 0> A#3#A#A#C  K%%k2 & Q (+,A=(Q%   s#8:JKL JI%,,.66*1i)a-.G+GH$||YKuzzRXY7F  " "#3A#6y#A#F#F#H I  J VAr1H%52>CCE }-/"%::%,,.66,,'8 SYZ$||S-A,B%**]cd7F"LL2q)9rBGGIM  " "= 1 /"%::||_-UZZOOr%r)rbrcrdrerrfr$r1rrYrr4rr^rrs@r#rurus ! 7k7k7k 7kr&*&*u||&*P2P,,662P,,2P2P  2P2Pr%ruceZdZdZ d dededeffd Zdejdejdejfd Z xZ S) rra A recipe module designed to make a `PreTrainedModel` exportable with `torch.export`, specifically for decoder-only LM to `HybridCache`. This module ensures that the exported model is compatible with further lowering and execution in `ExecuTorch`. rrrcNt|||_|jjjs t dt |jj|||jj|jj|_ tt|jD]r}|jd||jj|jd|jd||jj|jdty)a Initializes the exportable module with `HybridCache`. Args: model (`PreTrainedModel`): The pretrained model to wrap. max_batch_size (int): Maximum batch size for the cache. max_cache_len (int): Maximum sequence length for the cache. Raises: AssertionError: If the model doesn't have the expected configuration for HybridCache. zModel must have caching enabledrrFrrN)rnr$rrrirrrr*cacherrrrrrrs r#r$z-TorchExportableModuleWithHybridCache.__init__@s"  zz  ** !BC C!::$$)'::$$**""   s4::' dA  :aS!14::3D3DQ3G3L3LY^ _  <s!3TZZ5F5Fq5I5P5P]b c dr%rGrHrzc|jd}|jdj|d}|j|d||jd|}|j S)r|rrNT)rGrrrrirH)rrexpandrrr)r"rGrHrrrs r#rYz,TorchExportableModuleWithHybridCache.forwardfsh__Q' &//299*bI **% JJ)  ~~r%r) rbrcrdrerrfr$r1rrYrrs@r#rrrr9sc ! $d$d$d $dL<<   r%rrrrrr.r/cts tdddl}tjdt t jdt dd|j_|j5||n*|jdgg|j|j}||n)|jdg|j|j}tdr0|jjt|||fi|||nd }nd|t!j"d |t!j"d |jj$j't||fd |idd }|cdddS#1swYyxYw)a Convert a `PreTrainedModel` into an exportable module and export it using `torch.export`, ensuring the exported model is compatible with `ExecuTorch`. Args: model (`PreTrainedModel`): The pretrained model to be exported. example_input_ids (`Optional[torch.Tensor]`): Example input token id used by `torch.export`. example_cache_position (`Optional[torch.Tensor]`): Example current cache position used by `torch.export`. dynamic_shapes(`Optional[dict]`): Dynamic shapes used by `torch.export`. strict(`Optional[bool]`): Flag to instruct `torch.export` to use `torchdynamo`. Returns: Exported program (`torch.export.ExportedProgram`): The exported program generated via `torch.export`. torch >= 2.3 is required.rNrlrmr'rz2.6.0TrzWDynamic shapes spec will be ignored by convert_and_export_with_cache for torch < 2.6.0.zSThe strict flag will be ignored by convert_and_export_with_cache for torch < 2.6.0.rHF)r-rW pre_dispatchr/)r ImportErrortorch.export._tracer rvrwrrrxno_gradrrLrrr4rursr_trace_export)rrrr.r/r1rs r#convert_and_export_with_cachers* .566!))*=?UV$$%8:QRX:YZ(;ELL% ' !, se5::ellK &1 #qcELLI  %W -$||224U;')?@-!'!3v 3  )m! uv %||22::4U;')(*@A" ;   O' ' ' s /C>E77Fc(eZdZdZfdZdZxZS) Seq2SeqLMEncoderExportableModulez A wrapper module designed to make a Seq2Seq LM encoder exportable with `torch.export`. This module ensures that the exported encoder model is compatible with ExecuTorch. c0t|||_y)N)rnr$encoder)r" encoder_modelrys r#r$z)Seq2SeqLMEncoderExportableModule.__init__s $ r%c:|j|jS)N)rG)rr)r"rGs r#rYz(Seq2SeqLMEncoderExportableModule.forwards||i|0BBBr%rbrcrdrer$rYrrs@r#rrs %Cr%rc(eZdZdZfdZdZxZS)/Seq2SeqLMDecoderExportableModuleWithStaticCachez A wrapper module designed to make a Seq2Seq LM decoder exportable with `torch.export`, specifically for use with static caching. This module ensures the exported decoder is compatible with ExecuTorch. cVt||j|_|j|_|j |_t |j ||dtj|_ t|jt|_ tt|jD]r}|jd||jj |j"d|jd||jj |j$dty)NrrrFrr)rnr$ get_decoderdecoderlm_headrr r1r3rrrrrrrrrr)r"rmax_static_cache_lengthrrrys r#r$z8Seq2SeqLMDecoderExportableModuleWithStaticCache.__init__s ((* }} ll (;;%1--  )):):LNK s4,,-. kA  :aS!143D3D3K3KA3N3S3S`e f  <s!3T5F5F5M5Ma5P5W5Wdi j kr%cn|j|||jd|}|j|d}|S)NT)rGencoder_hidden_statesrrirHr)r rr )r"decoder_input_idsrrHr lm_logitss r#rYz7Seq2SeqLMDecoderExportableModuleWithStaticCache.forwardsB,,'"7 JJ)  LL, r%rrs@r#r r s k. r%r c<eZdZ dfd ZdZdZddZdZxZS)Seq2SeqLMExportableModulect|||_|j|_|j |_||_td||||d|_d|_ d|_ y)NT)rr)rirrr) rnr$ full_model get_encoderrrmax_hidden_seq_lengthr rexported_encoderexported_decoder)r"rrrrmax_cache_lengthrys r#r$z"Seq2SeqLMExportableModule.__init__sm ((* ll %:"!1'!5(!1 " !% $r%ct|jj|jjj }t jjd|j}t j5t jj||fdd|iid}ddd|S#1swYSxYw)Nencoder_seq_lengthrDrGr'Tr.r/) rrrrrr0r1r4r5rr)r"encoder_input_idswrapped_encoderrSrs r#_export_encoderz)Seq2SeqLMExportableModule._export_encoder#s:4<<HKKDOOLbLbchhjll&&';A[A[&\ ]]_ $||22"3!5{UVXcTdFenr 3         s )B99Cc t|j|jjj|jjj j dj}tjjd|j}tj5tjj||||fdd|iddd}ddd|S#1swYSxYw) N)rrrrencoder_hidden_seq_lengthrDr')rrrHTr) r rrrrrrr0r1r4r5rr)r"rrrHwrapped_decoderencoder_seq_len_dimrs r#_export_decoderz)Seq2SeqLMExportableModule._export_decoder1s ;oo(,(>(>(K(K(Y(Y11>>II  RY TV $ll../JPTPjPj.k]]_ $||22"$9>J)-./1D-E&*   3      s 1-C((C2cN|jj}||n%tjdtj|}||n'tj dggtj|}||n&tj dgtj|}||nZtj |jjjd|jjftj|} |j||_|j|| ||_|S)N)r' rrr()rrr1onesrLrrKrrrrd_modelr3r!rr&r) r"rrrrHrexample_encoder_input_idsexample_decoder_input_idsrexample_encoder_hidden_statess r#r4z Seq2SeqLMExportableModule.exportNs''!, G5::fE "!, se5::fE " -8NellA3V[V`V`io>p  %0 "''44??T[[EXEXYmm &!% 4 45N O $ 4 4 %'DF\!   r%c tj5|jj|}tjdggtj }dg}t |dz D]}|jj||tj|gtj }tj|dddddfdj}|j|tj|ggtj }||jjk(sn|cdddS#1swYyxYw)Nrr)r'rr) r1rrrrrLrrrrrrr) r"rr[encoder_outputrrrr next_tokens r#r^z"Seq2SeqLMExportableModule.generatens ]]_ !;T2299;A-. 7..557%~u||QCuzz7Z #\\&B*:CHHJ $$Z0%*LL:,uzz$R!!9!99 "!5 ! ! !sD"E8EE)r'rrr_r) rbrcrdr$r!r&r4r^rrs@r#rr s os%*   :@!r%rexample_attention_maskc lts tdtjdtt jdt dd|j _tj5tjj|d||tddd}|cd d d S#1swYy xYw) a Export a model with DynamicCache using `torch.export`, ensuring the exported model is compatible with `ExecuTorch`. Args: model (`PreTrainedModel`): The pretrained model to be exported. example_input_ids (`Optional[torch.Tensor]`): Example input token id used by `torch.export`. example_attention_mask (`Optional[torch.Tensor]`): Example attention mask used by `torch.export`. Returns: Exported program (`torch.export.ExportedProgram`): The exported program generated via `torch.export`. rrlrmrVT)rGrrriF)r/N) rrr rvrwrrrxr1rr4r)rrr1rs r#export_with_dynamic_cacher3s .566!))*=?UV$$%8:QRX:YZ(;ELL%    <<..  ."8#/>!   /       s .2B**B3rrH kv_length kv_offset mask_functionr local_sizeallow_is_causal_skipallow_torch_fixrzc  &|jd} t|||} |rt| | ||rytj||j } | |z } |j dd} t| ddd}t| ddd}| | td | | k}|| | |z kD}||z}n|| |z| |zk(}||z}|ddddddfj|ddd}| || ddddddfz}ts|r|tj|dd z}|S) a Create a 4D boolean mask of shape `(batch_size, 1, query_length, kv_length)` where a value of True indicates that the element should take part in the attention computation, and False that it should not. This is similar to `masking_utils.sdpa_mask` but does not use `vmap` which is incompatible with export. Args: batch_size (`int`): The batch size of the input sequence. cache_position (`torch.Tensor`): A tensor of shape (query_length,) indicating the current indices of the input sequence elements. kv_length (`int`): The size that the key and value states will have during the attention computation. kv_offset (`int`, optional): An optional offset to indicate at which first position the key and values states will refer to. mask_function (`Callable`): The mask factory function describing the mask pattern. attention_mask (`torch.Tensor`, optional): The 2D attention mask corresponding to padded tokens of shape (batch_size, number_of_seen_tokens+q_length) local_size (`int`, optional): The size of the local attention, if we do not use full attention. This is used only if `allow_is_causal_skip=True` to try to skip mask creation if possible. allow_is_causal_skip (`bool`, optional): Whether to allow to return `None` for the mask under conditions where we can use the `is_causal` argument in `torch.sdpa` instead. Default to `True`. allow_torch_fix (`bool`, optional): Whether to update the mask in case a query is not attending to any tokens, to solve a bug in torch's older versions. We need an arg to skip it when using eager. By default `True`. rN)rrr'rrkattention_chunk_sizez;Cannot use both `sliding_window` and `attention_chunk_size`Tr) rrr r1rMrviewrqrprr all)rrHr4r5r6rr7r8r9rWq_length padding_mask kv_arangereshaped_cache_positionrk chunk_size causal_masksliding_mask_overlaychunked_mask_overlays r#rwrwspV##A&H' 9ML 8xQZ\f g Y~/D/DEI I,11"a8 VH-/?FN)+A4HJ!j&<VWW66K!(+B^+SS++  (J6:QU_:__++ dD!Q./66z2r2NK!LD$1A$BB  /?uyy+2tDD r%r)NN)rNNNTT)'rstypingrrr1 cache_utilsrrrr generation.configuration_utilsr masking_utilsr r r rmodeling_utilsrr pytorch_utilsrrrnnModulerIrurrrrrrrr rr3rfrwrVr%r#rNs% UU= FY [ [ |ULEHHOOULp_P588??_PDL588??Lb1559%)! F F  -F %U\\2F TN F TN F R Cuxx C+ehhoo+\|!|!B1559$ $  -$ %U\\2$ V(,-1 $!% VVLLVV V H% V U\\* V VVVellVr%