rh!TddlmZddlmZmZGddeZGddeZddgZy) )PretrainedConfig)CONFIG_MAPPING AutoConfigcNeZdZdZdZddddddZ d fd ZxZS) VoxtralEncoderConfiga` This is the configuration class to store the configuration of a [`VoxtralEncoder`]. It is used to instantiate a Voxtral audio encoder according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the audio encoder of the Voxtral architecture. e.g. [mistralai/Voxtral-Mini-3B-2507](https://huggingface.co/mistralai/Voxtral-Mini-3B-2507) Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: vocab_size (`int`, *optional*, defaults to 51866): Vocabulary size of the model. hidden_size (`int`, *optional*, defaults to 1280): Dimensionality of the hidden representations. intermediate_size (`int`, *optional*, defaults to 5120): Dimension of the MLP representations. num_hidden_layers (`int`, *optional*, defaults to 32): Number of hidden layers in the Transformer encoder. num_attention_heads (`int`, *optional*, defaults to 20): Number of attention heads for each attention layer in the Transformer encoder. scale_embedding (`bool`, *optional*, defaults to `False`): Scale embeddings by dividing by sqrt(hidden_size) if True. activation_function (`str`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, "gelu", num_mel_bins (`int`, *optional*, defaults to 128): Number of mel features used per input features. Should correspond to the value used in the `VoxtralProcessor` class. max_source_positions (`int`, *optional*, defaults to 1500): The maximum sequence length of log-mel filter-bank features that this model might ever be used with. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. attention_dropout (`float`, *optional*, defaults to 0.0): The dropout ratio for the attention probabilities. ```python >>> from transformers import VoxtralEncoderConfig, VoxtralEncoder >>> # Initializing a VoxtralEncoderConfig >>> configuration = VoxtralEncoderConfig() >>> # Initializing a VoxtralEncoder (with random weights) >>> model = VoxtralEncoder(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```voxtral_encoder hidden_sizenum_hidden_layersnum_attention_headsintermediate_size layerdrop)d_modelencoder_layersencoder_attention_headsencoder_ffn_dimencoder_layerdropc  t |di| ||_||_||_||_||_||_||_||_ | |_ | |_ d|_ d|_ d|_| |_y)N)super__init__ vocab_sizer r r r scale_embeddingactivation_function num_mel_binsmax_source_positionsinitializer_rangedropoutractivation_dropoutattention_dropout)selfrr r r r rrrrrr!kwargs __class__s /var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/voxtral/configuration_voxtral.pyrzVoxtralEncoderConfig.__init__Ps "6"$&!2!2#6 .#6 ($8!!2  "%!2) iii Fgeluig{Gz?r)__name__ __module__ __qualname____doc__ model_type attribute_mapr __classcell__r$s@r%rrsU/ b#J!-#8.( M"!$3$3r&rc TeZdZdZdZeedZddddddd d d d d Z dfd ZxZ S) VoxtralConfiga3 This is the configuration class to store the configuration of a [`VoxtralForConditionalGeneration`]. It is used to instantiate an Voxtral model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the Voxtral-Mini-3B. e.g. [mistralai/Voxtral-Mini-3B-2507](https://huggingface.co/mistralai/Voxtral-Mini-3B-2507) Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: audio_config (`Union[AutoConfig, dict]`, *optional*): The config object or dictionary of the audio encoder. text_config (`Union[AutoConfig, dict]`, *optional*): The config object or dictionary of the text model. audio_token_id (`int`, *optional*): The image token index to encode the image prompt. projector_hidden_act (`str`, *optional*, defaults to `"gelu"`): The activation function (function or string) in the multi-modal projector. ```python >>> from transformers import VoxtralForConditionalGeneration, VoxtralConfig >>> # Initializing a Voxtral configuration >>> configuration = VoxtralConfig(audio_token_id=24, projector_hidden_act="gelu") >>> # Initializing a 3B model with random weights >>> model = VoxtralForConditionalGeneration(configuration) >>> # Accessing the model configuration >>> configuration = model.config ```voxtral) text_config audio_configii i gh㈵>TgחAr*) rr r r num_key_value_headsmax_position_embeddings rms_norm_eps use_cache rope_thetahead_dimc t|tr'|jdd|d<t|ddi|}n| td}||_t|tr5|jdd|d<t|ddii|j |}n|tddi|j }||_|j|_|j|_||_ ||_ t|0di|y)Nr/r llamar) isinstancedictgetrr7_default_text_config_kwargsr6rr audio_token_idprojector_hidden_actrr)r"r7r6rFrGr#r$s r%rzVoxtralConfig.__init__s lD ))5)9)9,HY)ZL &),|*DEU UL  !)*;<>L( k4 ((3 g(NK %(\)BCET55EEK (1UD4T4TUK&%00&22,$8! "6"r&)NNNr)) r+r,r-r.r/r sub_configsrErr1r2s@r%r4r4wsZ BJ",jIK! #)! ## ##r&r4N)configuration_utilsrautorrrr4__all__rr&r%rLs9 4-`3+`3FO#$O#d "? 3r&