rh<dZddlmZddlmZddlmZ ddZy)z9VPTQ (Vector Post-Training Quantization) integration fileN)init_empty_weights) VQuantLinearcF|sdgn|}|jD]w\}}|g}|j|dj|}|j}|j} t |t jr||vr|| vs|d|vr| j|dxs|j|dd} t5|j} |j} t| | | d| d| d| d| d | d | d | d | d dd|jdu|j|<d}|j|jddddt!t#|j%dkDrt'|||||\} }|j)dz||fS#1swYYxYw)aw Public method that recursively replaces the Linear layers of the given model with VPTQ quantized layers. `accelerate` is needed to use this method. Returns the converted model and a boolean that indicates if the conversion has been successful or not. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. quantization_config (`VptqConfig`): The quantization config object that contains the quantization parameters. modules_to_not_convert (`list[`str`]`, *optional*, defaults to `["lm_head"]`): Names of the modules to not convert in `VQuantLinear`. In practice we keep the `lm_head` in full precision for numerical stability reasons. current_key_name (`list`, *optional*): A list that contains the current key name. This is used for recursion and should not be passed by the user. has_been_replaced (`bool`, *optional*): A boolean that indicates if the conversion has been successful or not. This is used for recursion and should not be passed by the user. lm_headN. vector_lens num_centroidsnum_res_centroids group_num group_size outlier_sizeindices_as_float enable_norm enable_permTF) r r r r r rrrris_indice_packedenable_proxy_errorbiasr)quantization_configmodules_to_not_convertcurrent_key_namehas_been_replaced)named_childrenappendjoinshared_layer_configconfig_for_layers isinstancennLineargetr in_features out_featuresrr_modulesrequires_grad_lenlistchildrenreplace_with_vptq_linearpop)modelrrrrnamemodule layer_namerr layer_paramsr"r#_s q/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/integrations/vptq.pyr)r)s61Gi[Lb,,.2! f  #! %XX./ 1EE/AA vryy )"88 117G7KOb7b,00TBFYF]F] $dGL$% ;$00 %22 '3 ,] ;"."?&23F&G*;7+L9!-n!=%12D%E ,] ; ,] ;%)',D0(t$ %)!t$33E:/ ;0 tFOO%& '! +#;$7'=!1"3 $ A  R e2!f # ##E ; ;s 9BFF )NNNF)__doc__torch.nnr acceleratervptqrr)r1r8s'<)  P$r7