rh&ddlmZmZddlZddlmZddlmZddl m Z m Z m Z m Z ddlmZmZddlmZd d lmZGd d e d ZGdde ZdgZy))OptionalUnionN) BatchFeature) ImageInput)MultiModalDataProcessingKwargsProcessorMixinUnpack)PreTokenizedInput TextInput) TensorType) AutoTokenizerc8eZdZddddddejdZy)AriaProcessorKwargsF)paddingreturn_mm_token_type_ids)max_image_size split_image) text_kwargs images_kwargsreturn_tensorsN)__name__ __module__ __qualname__rPYTORCH _defaults{/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/aria/processing_aria.pyrr!s.(- "  %,, Ir!rF)totalc eZdZdZddgZdZdZ ddeee fde e de e ee e fe fffd Z dd eeeeeeefd e ed eed efd ZddZdZdZedZxZS) AriaProcessora AriaProcessor is a processor for the Aria model which wraps the Aria image preprocessor and the LLama slow tokenizer. Args: image_processor (`AriaImageProcessor`, *optional*): The AriaImageProcessor to use for image preprocessing. tokenizer (`PreTrainedTokenizerBase`, *optional*): An instance of [`PreTrainedTokenizerBase`]. This should correspond with the model's text model. The tokenizer is a required input. chat_template (`str`, *optional*): A Jinja template which will be used to convert lists of messages in a chat into a tokenizable string. size_conversion (`Dict`, *optional*): A dictionary indicating size conversions for images. image_processor tokenizerAriaImageProcessorr chat_templatesize_conversionc*|ddd}|jDcic]\}}t||c}}|_|j|_|j|_||j |j |_t|!|||ycc}}w)N)ir)r)) itemsintr* image_tokenimage_token_id pad_token unk_tokensuper__init__)selfr&r'r)r*kv __class__s r"r5zAriaProcessor.__init__Bs  "$'c2O6E6K6K6MNdaA N$00'66  Y%8%8%@"+"5"5I  )=Q OsBtextimageskwargsreturnc |jtfd|jji|}t |t r|g}n.t |t st |dt s td||j|fi|d}|j|jjd}g} |jd|z} |D]P} | j|jj|jj| z} | j| Rni}|} |djd d} |djd d } |j| fi|dd di}|j!| |d g | rUt#j$|d}t#j&|d}d|||j(k(<|j+|d<t-i||| S)a Main method to prepare for the model one or several sequences(s) and image(s). Args: text (`TextInput`, `PreTokenizedInput`, `list[TextInput]`, `list[PreTokenizedInput]`): The sequence or batch of sequences to be encoded. Each sequence can be a string or a list of strings (pretokenized string). If the sequences are provided as list of strings (pretokenized), you must set `is_split_into_words=True` (to lift the ambiguity with a batch of sequences). images (`ImageInput`): The image or batch of images to be prepared. Each image can be a PIL image, NumPy array or PyTorch tensor. Both channels-first and channels-last formats are supported. Returns: [`BatchFeature`]: A [`BatchFeature`] with the following fields: - **input_ids** -- List of token ids to be fed to a model. Returned when `text` is not `None`. - **attention_mask** -- List of indices specifying which tokens should be attended to by the model (when `return_attention_mask=True` or if *"attention_mask"* is in `self.model_input_names` and if `text` is not `None`). - **pixel_values** -- Pixel values to be fed to a model. Returned when `images` is not `None`. - **pixel_mask** -- Pixel mask to be fed to a model. Returned when `images` is not `None`. tokenizer_init_kwargsrzAInvalid input text. Please provide a string, or a list of stringsNrr num_cropsrrrFimage) modalities input_idsmm_token_type_ids)data tensor_type) _merge_kwargsrr' init_kwargs isinstancestrlist TypeErrorr&r* pixel_valuesshapepopreplacer0append_check_special_mm_tokensnparray zeros_liker1tolistr)r6r:r;audiovideosr< output_kwargs image_inputstokens_per_imageprompt_stringsr@samplerr text_inputs array_idsrEs r"__call__zAriaProcessor.__call__Ts<+**  "&.."<"<  dC 6DD$' 47C0H_` `  /4//Y-:XYL#33L4M4M4S4STU4VW N$((58HHI .(B(BDNND^D^ajDjk%%f- . L!N&}599:JDQ#0#?#C#CD^`e#f $dnn^i}]7Sidhi  %%nkwi%X #[!9:I " k+.F G BC i4+>+>> ?/@/G/G/IK+ ,!@K!@ Lr r   4D[lm n, ,,! ss $%C Cc:|jj|i|S)z This method forwards all its arguments to LlamaTokenizerFast's [`~PreTrainedTokenizer.batch_decode`]. Please refer to the docstring of this method for more information. )r' batch_decoder6argsr<s r"rozAriaProcessor.batch_decodes +t~~**D;F;;r!c:|jj|i|S)z This method forwards all its arguments to LlamaTokenizerFast's [`~PreTrainedTokenizer.decode`]. Please refer to the docstring of this method for more information. )r'decoderps r"rszAriaProcessor.decodes %t~~$$d5f55r!c|jj}|jj}|Dcgc] }|dk7s | }}ttj ||zScc}w)Nr@)r'model_input_namesr&rLdictfromkeys)r6tokenizer_input_namesimage_processor_input_namesnames r"ruzAriaProcessor.model_input_namesse $ @ @&*&:&:&L&L#9T&kW[_jWjt&k#&kDMM"7:U"UVWW'ls A#A#)NNNN)NNN)N)rrr__doc__ attributesimage_processor_classtokenizer_classrrrKrrvfloatr/r5r r rLrr rrrarmrorspropertyru __classcell__)r9s@r"r%r%/s $[1J0%O/3'+BF R+,R } R "$uUCZ'8#'=">? R*(, B^I0$y/4HYCZZ[B^$B^ ,- B^ B^H-4<6XXr!r%)typingrrnumpyrTimage_processing_utilsr image_utilsrprocessing_utilsrr r r tokenization_utilsr r utilsrautorrr%__all__r r!r"rsL*#2%XX>  *% YXNYXx  r!