rhhdZddlZddlmZddlmZddlmZeje Z gdZ gdZ d gZ gd Zd Zd Zd ZdZeeeedZGddeZGddeZGddeZgdZy)zJukebox configurationN)Union)PretrainedConfig)logging)O block_attntranspose_block_attnprev_block_attnrrr rrr rrr rrr rrr cross_attentionrrr rrr rrr r rrr rrr rrr r rrr rrr rrr r rrr rrr rrr r rrr rrr rrr r rrr rrr rrr r )rrr dense_attention) prime_attnr dense_attnctdS)Nr)_FullDenseAttentionlayers /var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/deprecated/jukebox/configuration_jukebox.pyfull_dense_attentionrqs q !!ct|dzS)N)_RawColumnPreviousRowAttentionrs r!raw_column_previous_row_attentionrus )%!) 44rct|dzS)NO)_LARGE_ATTENTIONrs r large_separated_enc_dec_w_lyricsrys EBJ ''rcB|dzdk(r t|dzSt|dzS)Nr)_PrimePrimeDenseAttentionrrs renc_dec_with_lyricsr!}s, rzR(33 )%!) 44r)rrrr!c*eZdZdZdZdddZdddd d d dd d dddddddddd ddddddgdddddddddddgdgddd dd d f*fd! Zed$d"ee e jffd#Z xZ S)%JukeboxPriorConfiga" This is the configuration class to store the configuration of a [`JukeboxPrior`]. It is used to instantiate a `JukeboxPrior` according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the top level prior from the [openai/jukebox-1b-lyrics](https://huggingface.co/openai/jukebox -1b-lyrics) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: act_fn (`str`, *optional*, defaults to `"quick_gelu"`): Activation function. alignment_head (`int`, *optional*, defaults to 2): Head that is responsible of the alignment between lyrics and music. Only used to compute the lyric to audio alignment alignment_layer (`int`, *optional*, defaults to 68): Index of the layer that is responsible of the alignment between lyrics and music. Only used to compute the lyric to audio alignment attention_multiplier (`float`, *optional*, defaults to 0.25): Multiplier coefficient used to define the hidden dimension of the attention layers. 0.25 means that 0.25*width of the model will be used. attention_pattern (`str`, *optional*, defaults to `"enc_dec_with_lyrics"`): Which attention pattern to use for the decoder/ attn_dropout (`int`, *optional*, defaults to 0): Dropout probability for the post-attention layer dropout in the decoder. attn_res_scale (`bool`, *optional*, defaults to `False`): Whether or not to scale the residuals in the attention conditioner block. blocks (`int`, *optional*, defaults to 64): Number of blocks used in the `block_attn`. A sequence of length seq_len is factored as `[blocks, seq_len // blocks]` in the `JukeboxAttention` layer. conv_res_scale (`int`, *optional*): Whether or not to scale the residuals in the conditioner block. Since the top level prior does not have a conditioner, the default value is to None and should not be modified. num_layers (`int`, *optional*, defaults to 72): Number of layers of the transformer architecture. emb_dropout (`int`, *optional*, defaults to 0): Embedding dropout used in the lyric decoder. encoder_config (`JukeboxPriorConfig`, *optional*) : Configuration of the encoder which models the prior on the lyrics. encoder_loss_fraction (`float`, *optional*, defaults to 0.4): Multiplication factor used in front of the lyric encoder loss. hidden_size (`int`, *optional*, defaults to 2048): Hidden dimension of the attention layers. init_scale (`float`, *optional*, defaults to 0.2): Initialization scales for the prior modules. is_encoder_decoder (`bool`, *optional*, defaults to `True`): Whether or not the prior is an encoder-decoder model. In case it is not, and `nb_relevant_lyric_tokens` is greater than 0, the `encoder` args should be specified for the lyric encoding. mask (`bool`, *optional*, defaults to `False`): Whether or not to mask the previous positions in the attention. max_duration (`int`, *optional*, defaults to 600): Maximum supported duration of the generated song in seconds. max_nb_genres (`int`, *optional*, defaults to 1): Maximum number of genres that can be used to condition the model. merged_decoder (`bool`, *optional*, defaults to `True`): Whether or not the decoder and the encoder inputs are merged. This is used for the separated encoder-decoder architecture metadata_conditioning (`bool`, *optional*, defaults to `True)`: Whether or not to condition on the artist and genre metadata. metadata_dims (`List[int]`, *optional*, defaults to `[604, 7898]`): Number of genres and the number of artists that were used to train the embedding layers of the prior models. min_duration (`int`, *optional*, defaults to 0): Minimum duration of the generated audio on which the model was trained. mlp_multiplier (`float`, *optional*, defaults to 1.0): Multiplier coefficient used to define the hidden dimension of the MLP layers. 0.25 means that 0.25*width of the model will be used. music_vocab_size (`int`, *optional*, defaults to 2048): Number of different music tokens. Should be similar to the `JukeboxVQVAEConfig.nb_discrete_codes`. n_ctx (`int`, *optional*, defaults to 6144): Number of context tokens for each prior. The context tokens are the music tokens that are attended to when generating music tokens. n_heads (`int`, *optional*, defaults to 2): Number of attention heads. nb_relevant_lyric_tokens (`int`, *optional*, defaults to 384): Number of lyric tokens that are used when sampling a single window of length `n_ctx` res_conv_depth (`int`, *optional*, defaults to 3): Depth of the `JukeboxDecoderConvBock` used to upsample the previously sampled audio in the `JukeboxMusicTokenConditioner`. res_conv_width (`int`, *optional*, defaults to 128): Width of the `JukeboxDecoderConvBock` used to upsample the previously sampled audio in the `JukeboxMusicTokenConditioner`. res_convolution_multiplier (`int`, *optional*, defaults to 1): Multiplier used to scale the `hidden_dim` of the `JukeboxResConv1DBlock`. res_dilation_cycle (`int`, *optional*): Dilation cycle used to define the `JukeboxMusicTokenConditioner`. Usually similar to the ones used in the corresponding level of the VQVAE. The first prior does not use it as it is not conditioned on upper level tokens. res_dilation_growth_rate (`int`, *optional*, defaults to 1): Dilation grow rate used between each convolutionnal block of the `JukeboxMusicTokenConditioner` res_downs_t (`List[int]`, *optional*, defaults to `[3, 2, 2]`): Downsampling rates used in the audio conditioning network res_strides_t (`List[int]`, *optional*, defaults to `[2, 2, 2]`): Striding used in the audio conditioning network resid_dropout (`int`, *optional*, defaults to 0): Residual dropout used in the attention pattern. sampling_rate (`int`, *optional*, defaults to 44100): Sampling rate used for training. spread (`int`, *optional*): Spread used in the `summary_spread_attention` pattern timing_dims (`int`, *optional*, defaults to 64): Dimension of the timing embedding. zero_out (`bool`, *optional*, defaults to `False`): Whether or not to zero out convolution weights when initializing. jukebox_prior n_positionsn_head)max_position_embeddingsnum_attention_heads quick_gelurDg?r!F@NHg?皙?TPiXi\ig?iirrr*r*r*r*r*Dc+ t,|di|+||_||_||_||_||_||_||_| |_ | |_ | |_ | |_ ||_ | tdi| |_nd|_||_||_||_||_||_||_||_||_||_||_||_||_||_||_||_||_||_ | |_!|!|_"|"|_#|#|_$|$|_%|%|_&|&|_'|'|_(|(|_)|)|_*||_+|*|_,yN)-super__init__act_fnalignment_headalignment_layerattention_multiplierattention_pattern attn_dropoutattn_res_scaleblocksconv_res_scale num_layers emb_dropoutmusic_vocab_sizer#encoder_configencoder_loss_fraction init_scaleis_encoder_decoderlyric_vocab_sizelevelmask max_duration max_nb_genresmerged_decodermetadata_conditioning metadata_dims min_durationmlp_multipliern_ctxn_headsnb_relevant_lyric_tokensres_conv_depthres_conv_widthres_convolution_multiplierres_dilation_cycleres_dilation_growth_rate res_downs_t res_strides_t resid_dropout sampling_ratespread timing_dims hidden_sizezero_out)-selfr;rLr<r=r>r?r@rArBrCrDrErGrHrcrIrJrKrMrNrOrPrQrRrSrTrFrUrVrWrXrYrZr[r\r]r^r_r`rarbrdkwargs __class__s- rr:zJukeboxPriorConfig.__init__sr\ "6" ,.$8!!2(, ,$& 0  %"4"F~"FD "&D %:"$"4 0  (*,%:"*(,  (@%,,*D'"4(@%&*** &&  rpretrained_model_name_or_pathc D|j||j|fi|\}}|jddk(r|d|}d|vrGt|dr;|d|jk7r)t j d|dd|jd|j|fi|S)N model_typejukeboxprior_You are using a model of type to instantiate a model of type N. This is not supported for all configurations of models and can yield errors._set_token_in_kwargsget_config_dictgethasattrrjloggerwarning from_dict)clsrhrLrf config_dicts rfrom_pretrainedz"JukeboxPriorConfig.from_pretrained\s   (1c112OZSYZ V ??< (I 5%ug&67K ; &73 +E+VbJcgjguguJu NN0\1J0KKk>>""pr  s}}[3F33r)r)__name__ __module__ __qualname____doc__rj attribute_mapr: classmethodrstrosPathLikerz __classcell__rgs@rr#r#skZ!J#0'M!/! "Dk!$#$!"W[!z4E#r{{BR