rh\dZddlZddlmZmZddlZddlmZm Z ddl m Z m Z m Z ddlmZmZmZmZmZmZmZmZmZmZmZmZmZddlmZmZdd lm Z ejBe"Z# dd e$d e$d e$d e$de$de$de$fdZ%GddeZ&dgZ'y)z#Image processor class for GLM-4.1V.N)OptionalUnion)BaseImageProcessor BatchFeature)convert_to_rgbresizeto_channel_dimension_format) OPENAI_CLIP_MEANOPENAI_CLIP_STDChannelDimension ImageInputPILImageResamplingget_image_sizeinfer_channel_dimension_formatis_scaled_imagemake_flat_list_of_imagesmake_list_of_imagesto_numpy_array valid_imagesvalidate_preprocess_arguments) TensorTypelogging) VideoInput num_framesheightwidthtemporal_factorfactor min_pixels max_pixelsc||krtd|d|||ks||krtd|d|d|t||t||z dkDr%tdt||t||z t||z |z}t||z |z}t||z |z} | |z|z|kDr^t j ||z|z|z } t j || z |z |z}t j || z |z |z}||fS| |z|z|krZt j |||z|zz } t j|| z|z |z}t j|| z|z |z}||fS)Nzt:z% must be larger than temporal_factor:zheight:z or width:z must be larger than factor:z4absolute aspect ratio must be smaller than 200, got ) ValueErrormaxminroundmathsqrtfloorceil) rrrrrr r!h_barw_bart_barbetas /var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/glm4v/image_processing_glm4v.py smart_resizer12sO#2j\)NN_`aa %&.76(*UG;WX^W_`aa VU c&%0 03 6B3vuCUX[\bdiXjCjBk l   &6/ "V +E %&. !F *E *. // AE u}uz)yy*v-5CD 6D=612V; 54<&01F: %<   +yyzF':U'BCD &4-&01F: %$,/069 %<c&eZdZdZddgZddej dddddddddf d ed ee e e fd ed ed e e e fdedee e ee fdee e ee fdede de de ddffd Zddddddddddddej"dfde eefd eed ee e e fd ed eed ee deedee e ee fdee e ee fdee dee dee deedeedee e effdZddddddddddddddej"dfdeded eed ee e e fd ed eed ee deedee e ee fdee e ee fdee dee dee deedee e efdeedee e eff"dZd!de de fd ZxZS)"Glm4vImageProcessora Constructs a GLM-4V image processor that dynamically resizes images based on the original images. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to resize the image's (height, width) dimensions. size (`Dict[str, int]` *optional*, defaults to `{"shortest_edge": 112 * 112, "longest_edge": 28 * 28 * 15000}`): Size of the image's `(height, width)` dimensions after resizing. Can be overridden by the `size` parameter in the `preprocess` method. Available options are: - `{"height": int, "width": int}`: The image will be resized to the exact size `(height, width)`. Do NOT keep the aspect ratio. - `{"shortest_edge": int, "longest_edge": int}`: The image will be resized to a maximum size respecting the aspect ratio and keeping the shortest edge less or equal to `shortest_edge` and the longest edge less or equal to `longest_edge`. - `{"max_height": int, "max_width": int}`: The image will be resized to the maximum size respecting the aspect ratio and keeping the height less or equal to `max_height` and the width less or equal to `max_width`. resample (`PILImageResampling`, *optional*, defaults to `Resampling.BICUBIC`): Resampling filter to use when resizing the image. do_rescale (`bool`, *optional*, defaults to `True`): Whether to rescale the image by the specified scale `rescale_factor`. rescale_factor (`int` or `float`, *optional*, defaults to `1/255`): Scale factor to use if rescaling the image. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image. image_mean (`float` or `List[float]`, *optional*, defaults to `[0.48145466, 0.4578275, 0.40821073]`): Mean to use if normalizing the image. This is a float or list of floats for each channel in the image. image_std (`float` or `List[float]`, *optional*, defaults to `[0.26862954, 0.26130258, 0.27577711]`): Standard deviation to use if normalizing the image. This is a float or list of floats for each channel in the image. do_convert_rgb (`bool`, *optional*, defaults to `True`): Whether to convert the image to RGB. patch_size (`int`, *optional*, defaults to 14): The spatial patch size of the vision encoder. temporal_patch_size (`int`, *optional*, defaults to 2): The temporal patch size of the vision encoder. merge_size (`int`, *optional*, defaults to 2): The merge size of the vision encoder to llm encoder. pixel_valuesimage_grid_thwTNgp? do_resizesizeresample do_rescalerescale_factor do_normalize image_mean image_stddo_convert_rgb patch_sizetemporal_patch_size merge_sizereturnc  t|di| |d|vsd|vr tdddd}||_||_||_||_||_||_||nt|_ ||nt|_ | |_ | |_| |_| |_y)N shortest_edge longest_edge:size must contain 'shortest_edge' and 'longest_edge' keys.1qrGrH)super__init__r$r:r9r;r<r=r>r r?r r@rBrCrDrA)selfr9r:r;r<r=r>r?r@rArBrCrDkwargs __class__s r0rOzGlm4vImageProcessor.__init__}s "6"  !<VZ@ZYZ Z%.PD "  $,((2(>*DT&/&;$#6 $,r2images data_formatinput_data_formatc t|}| r|Dcgc] }t|}}|Dcgc] }t|}}|r#t|drtj d|t |d}t|d|\}}||}}g}|D]w}|r't| ||| | | z\}}t|||f||}|r|j|||}|r|j||| |}t|||}|j|ytj|}|t j"k(r|j%dd d d }|j&d| zdk7rXtj(|d tj*| |j&d| zz d }tj,||gd }|j&d }|j&d| z}|| z|| z}}|j/|| ||| z| | || z| | }|j%dd dddd d dd }|j/||z|z|| z| z| z}||||ffScc}wcc}w)a~ Preprocess an image or batch of images. Copy of the `preprocess` method from `CLIPImageProcessor`. Args: images (`ImageInput`): Image or batch of images to preprocess. Expects pixel values ranging from 0 to 255. If pixel values range from 0 to 1, set `do_rescale=False`. vision_info (`List[Dict]`, *optional*): Optional list of dictionaries containing additional information about vision inputs. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`Dict[str, int]`, *optional*, defaults to `self.size`): Size of the image after resizing. `shortest_edge` and `longest_edge` keys must be present. resample (`PILImageResampling`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the image. This can be one of the `PILImageResampling` enums. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Scale factor to use if rescaling the image. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. image_mean (`float` or `List[float]`, *optional*, defaults to `self.image_mean`): Mean to use if normalizing the image. Can be a float or a list of floats corresponding to the number of channels in the image. image_std (`float` or `List[float]`, *optional*, defaults to `self.image_std`): Standard deviation to use if normalizing the image. Can be a float or a list of floats corresponding to the number of channels in the image. patch_size (`int`, *optional*, defaults to `self.patch_size`): The spatial patch size of the vision encoder. temporal_patch_size (`int`, *optional*, defaults to `self.temporal_patch_size`): The temporal patch size of the vision encoder. merge_size (`int`, *optional*, defaults to `self.merge_size`): The merge size of the vision encoder to llm encoder. do_convert_rgb (`bool`, *optional*, defaults to `self.do_convert_rgb`): Whether to convert the image to RGB. data_format (`ChannelDimension`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - Unset: Use the channel dimension format of the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. rzIt looks like you are trying to rescale already rescaled images. If the input images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again.) channel_dim)rrrrr)r:r;rU)scalerU)imagemeanstdrU)input_channel_dimrr8)axis)rrrrlogger warning_oncerrr1r rescale normalizer appendnparrayr LAST transposeshaperepeatnewaxis concatenatereshape)rPrSr9r:r;r<r=r>r?r@rBrCrDrArTrUrYrrresized_height resized_widthprocessed_imagespatchesrepeatschannelgrid_tgrid_hgrid_wflatten_patchess r0 _preprocesszGlm4vImageProcessor._preprocesssz%V, 9?@nU+@F@6<vay I &vay>OP (.  +E0<2!$7% 2 1-  ?(fw U.Te fjiSd'0{VghE  # #E *- +0((+, *// /''1a3G == 1 1Q 6ii BJJ')< a@PSf@f)gnoGnngw%7a@G--"q!%88':5} 7R//    j   j    ##Aq!Q1aA>!// VOf $g0C&Cj&PS]&]  888IA=s H<Ivideosreturn_tensorscB|d|vsd|vr tdddd}||n |j}||n |j}||n |j}||n |j}||n |j }| | n |j } | | n |j} | | n |j} | | n |j} | | n |j} ||n |j}| t|}|t|s tdt||| | |||i}|gg}}|D]G}|j|||||||| | | | | ||| \}}|j!||j#|It%j&|}t%j&|}|j)||d t+|| S) a Args: images (`ImageInput`): Image to preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set `do_rescale=False`. videos (`VideoInput`): Video to preprocess. Expects a single or batch of videos with pixel values ranging from 0 to 255. If passing in videos with pixel values between 0 and 1, set `do_rescale=False`. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`Dict[str, int]`, *optional*, defaults to `self.size`): Size of the image after resizing. Shortest edge of the image is resized to size["shortest_edge"], with the longest edge resized to keep the input aspect ratio. resample (`int`, *optional*, defaults to `self.resample`): Resampling filter to use if resizing the image. This can be one of the enum `PILImageResampling`. Only has an effect if `do_resize` is set to `True`. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Rescale factor to rescale the image by if `do_rescale` is set to `True`. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. image_mean (`float` or `List[float]`, *optional*, defaults to `self.image_mean`): Image mean to use for normalization. Only has an effect if `do_normalize` is set to `True`. image_std (`float` or `List[float]`, *optional*, defaults to `self.image_std`): Image standard deviation to use for normalization. Only has an effect if `do_normalize` is set to `True`. The max pixels of the image to resize the image. patch_size (`int`, *optional*, defaults to `self.patch_size`): The spatial patch size of the vision encoder. temporal_patch_size (`int`, *optional*, defaults to `self.temporal_patch_size`): The temporal patch size of the vision encoder. merge_size (`int`, *optional*, defaults to `self.merge_size`): The merge size of the vision encoder to llm encoder. do_convert_rgb (`bool`, *optional*, defaults to `self.do_convert_rgb`): Whether to convert the image to RGB. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - Unset: Use the channel dimension format of the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. rGrHrIrJrKrLzkInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray.)r=r>r?r@r9r:r;)r9r:r;r<r=r>r?r@rBrCrDrTrArU)r5r6)data tensor_type)r$r9r;r<r=r>r?r@rBrCrDrArrrr}extendrirjrkupdater)rPrSr~r9r:r;r<r=r>r?r@rBrCrDrArrTrUrr5vision_grid_thwsrYrvr6s r0 preprocesszGlm4vImageProcessor.preprocess'sZ  !<VZ@ZYZ Z%.PD!*!6IDNN '38#-#9Zt +9+E4K^K^'3'?|TEVEV #-#9Zt !*!6IDNN #-#9Zt 5H5T1Z^ZrZr#-#9Zt +9+E4K^K^  -f5F  l6&::  &)%!   -/*L 8*.*:*:'%)#1!-)')(;) +#1&7+;+'"##G, ''7' 8(88L1L!xx(89  KKIYZ [>BBr2rrc|jd|j}|jd|j}||z}t|j||||j\}}||z||z} } | | zS)a A utility that returns number of image patches for a given image size. Args: height (`int`): Height of the input image. width (`int`): Width of the input image. images_kwargs (`dict`, *optional*) Any kwargs to override defaults of the image processor. Returns: `int`: Number of image patches per image. rBrD)rrrrr)getrBrDr1rC) rPrr images_kwargsrBrDrrsrtrzr{s r0get_number_of_image_patchesz/Glm4vImageProcessor.get_number_of_image_patchess#&&|T__E "&&|T__E j((4// 44 ) % (:5} 7Rr2)N)__name__ __module__ __qualname____doc__model_input_namesrBICUBICboolrdictstrintrfloatlistrOr FIRSTrrr}rrr __classcell__)rRs@r0r4r4Ss%N()9:)-'9'A'A,3!:>9=##$"-"-tCH~&"-% "-  "- c5j) "-"-U5$u+#567"-E%e"456"-"-"-!"-"- "-N%))-'+%)*.'+:>9=$(-1$()-2B2H2HDH!D9j*,-D9D>D9tCH~& D9 % D9 TN D9!D9tnD9U5$u+#567D9E%e"456D9SMD9&c]D9SMD9!D9./D9 $E#/?*?$@A!D9R"$()-'+%)*.'+:>9=$(-1$()-;?2B2H2HDH%MCMCMCD> MC tCH~& MC % MCTNMC!MCtnMCU5$u+#567MCE%e"456MCSMMC&c]MCSMMC!MC !sJ!78!MC"./#MC$$E#/?*?$@A%MC^#cr2r4)r8rJi)(rr(typingrrnumpyrjimage_processing_utilsrrimage_transformsrr r image_utilsr r r rrrrrrrrrrutilsrr video_utilsr get_loggerrrerr1r4__all__rMr2r0rs* "F )%   H %0       B},}@ ! !r2