rh*ddlmZddlmZmZmZddlZddlZddlm Z ddl m Z ddl m Z ddlmZdd lmZmZdd lmZdd lmZmZmZdd lmZd dlmZddlmZeedGddeZGdde j@Z!dZ"d@dZ#dejHde%dejHfdZ& dAde j@dejHdejHd ejHd!eejHd"e'd#e'd$eefd%Z(Gd&d'e j@Z)Gd(d)e j@Z*Gd*d+e j@Z+d,ejHd-ejHd.ejHdejHfd/Z,Gd0d1e j@Z-Gd2d3e j@Z.eGd4d5eZ/d6ejHd7e'de0ejHejHffd8Z1d9ejHd:e%d;e%dejHfd<Z2ed=Gd>d?e/Z3d5d?gZ4y)B) dataclass)CallableOptionalUnionN)nn pad_sequence)ACT2FN)FlashAttentionKwargs)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstring)can_return_tuple)AutoModelForKeypointDetection)LightGlueConfiga Base class for outputs of LightGlue keypoint matching models. Due to the nature of keypoint detection and matching, the number of keypoints is not fixed and can vary from image to image, which makes batching non-trivial. In the batch of images, the maximum number of matches is set as the dimension of the matches and matching scores. The mask tensor is used to indicate which values in the keypoints, matches, matching_scores and prune tensors are keypoint matching information. ) custom_introc^eZdZUdZdZeejed<dZ eejed<dZ eejed<dZ eejed<dZ eejed<dZeejed<dZeeejed <dZeeejed <y) LightGlueKeypointMatchingOutputa loss (`torch.FloatTensor` of shape `(1,)`, *optional*): Loss computed during training. matches (`torch.FloatTensor` of shape `(batch_size, 2, num_matches)`): Index of keypoint matched in the other image. matching_scores (`torch.FloatTensor` of shape `(batch_size, 2, num_matches)`): Scores of predicted matches. keypoints (`torch.FloatTensor` of shape `(batch_size, num_keypoints, 2)`): Absolute (x, y) coordinates of predicted keypoints in a given image. prune (`torch.IntTensor` of shape `(batch_size, num_keypoints)`): Pruning mask indicating which keypoints are removed and at which layer. mask (`torch.BoolTensor` of shape `(batch_size, num_keypoints)`): Mask indicating which values in matches, matching_scores, keypoints and prune are keypoint matching information. hidden_states (`Tuple[torch.FloatTensor, ...]`, *optional*): Tuple of `torch.FloatTensor` (one for the output of each stage) of shape `(batch_size, 2, num_channels, num_keypoints)` returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True` attentions (`Tuple[torch.FloatTensor, ...]`, *optional*): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, 2, num_heads, num_keypoints, num_keypoints)` returned when `output_attentions=True` is passed or when `config.output_attentions=True` Nlossmatchesmatching_scores keypointsprunemask hidden_states attentions)__name__ __module__ __qualname____doc__rrtorch FloatTensor__annotations__rrrr IntTensorr r!tupler"/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/transformers/models/lightglue/modeling_lightglue.pyrr&s0)-D(5$$ %,+/GXe'' (/37OXe//07-1Ix))*1'+E8EOO $+(,D(5$$ %,8projected_keypoints embeddingscosinessinesoutputs r.forwardz"LightGluePositionalEncoder.forwardXsq#nnY7(::1":E ))J' *%u% 6J*12 R\P] r-F) r#r$r%rr6r'Tensorrboolrr+rL __classcell__r=s@r.r0r0Ssbll OT  =Ed^  uU\\"E%,, *D$EE F r-r0c|ddddf}|ddddf}tj| |gdjd}|S)N.rrrArB)r'stackflatten)xx1x2rot_xs r. rotate_halfrZdsL 3!8B 319B KK"b r * 2 22 6E Lr-c6|j}|j}|j}|j|}|j|}||zt||zz}||zt||zz}|j ||j |fS)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. dtype)r]float unsqueezerZto) qkrErF position_ids unsqueeze_dimr]q_embedk_embeds r.apply_rotary_pos_embrgls( GGE  A  A -- &C -- &C3w;q>C/0G3w;q>C/0G ::E: "GJJUJ$; ;;r-r!n_repr?c|j\}}}}|dk(r|S|dddddddddfj|||||}|j|||z||S)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) rN)shapeexpandreshape)r!rhbatchnum_key_value_headsslenhead_dims r. repeat_kvrqso 2?1D1D.E h z!!Qa"23::5BUW\^bdlmM  (;e(CT8 TTr-modulequerykeyvalueattention_maskscalingdropoutkwargsc Tt||j}t||j} tj||j dd|z} |#|ddddddd|j df} | | z} t jj| dtjj|j} t jj| ||j} tj| | } | j ddj} | | fS)Nrr rSrA)rCr])ptrainingr)rqnum_key_value_groupsr'matmul transposerjr functionalsoftmaxfloat32r`r]rxr| contiguous) rrrsrtrurvrwrxry key_states value_states attn_weights causal_mask attn_outputs r.eager_attention_forwardrs3 ; ;$>&B\B\$\!}}d* !'!9!9ii    : :T]] JQWQfQf  ii    : :T]] JQWQfQf  ii    : :T]] JQWQfQf  ii  & & 68J8JQWQfQf  r-r!position_embeddingsrvencoder_hidden_statesencoder_attention_maskryr?c |jdd}g|d|j}|j|j|j dd} |du} | r|n|} | r|n|} |j | j|j dd} |j | j|j dd}||\}}t| | ||\} } t}|jjdk7rt|jj}||| | || f|jsdn |j|jd|\}}|jg|dj!}|j#|}||fS)NrArreager)rxrw)rjrprviewrrrrgrr1_attn_implementationr r|rrwrlrr)r<r!rrvrrry input_shape hidden_shape query_statesis_cross_attentioncurrent_statescurrent_attention_maskrrrErFattention_interfacerrs r.rLzLightGlueAttention.forwards$))#2. 88b8$--8 {{=166|DNNqRST 2$>2D.-;M!7Sa[[055lCMMaQRS {{>277 EOOPQSTU  **HC';L*VY[^'_ $L*(? ;; + +w 6"9$++:Z:Z"[ $7     " % $}}C$2H2HLL %  % ! \*k));;;;FFH kk+. L((r-)NNNN)r#r$r%r&rintr6r'rNrr+rr rLrPrQs@r.rrsG  3 4LP158<9= *)||*)&eELL%,,,F&GH*)!. *) ( 5 *) !) 6 *)-.*) u||Xell3XeELL>Q5RR S*)r-rc\eZdZdeffd Zdej dej fdZxZS) LightGlueMLPr1cht|||_t|j|_t j|j|j|_ t j|j|j|_ t j|jd|_ y)NT)elementwise_affine)r5r6r1r hidden_act activation_fnrr7intermediate_sizefc1rfc2 LayerNorm layer_normr;s r.r6zLightGlueMLP.__init__s}  #F$5$5699V55v7O7OP99V55v7I7IJ,,v'?'?TXYr-r!r?c|j|}|j|}|j|}|j|}|SN)rrrr)r<r!s r.rLzLightGlueMLP.forwardsB/  6 **=9 / r- r#r$r%rr6r'rNrLrPrQs@r.rrs,ZZU\\ellr-rceZdZdedeffd Z d dejdejdejdee dee d e ejee ejee ejff d Z xZ S) LightGlueTransformerLayerr1rct|t|||_t ||_t|||_t ||_yr)r5r6rself_attentionrself_mlpcross_attention cross_mlprs r.r6z"LightGlueTransformerLayer.__init__ sD 0C$V, 1&)D%f-r- descriptorsrrvr>output_attentionsr?c|rdnd}|rdnd}|r||fz}|j\}} } |j||||\} } tj|| gd} |j | }||z}|r| |f}|j dd| | j dj || | }|6|j dddd| j dj |dd| nd}|j||||\}}tj||gd}|j|}||z}|r6||f}||j || | fzz|j || | fz|z}|r || fz|fz}|||fS)Nr,)rrvrrArBrr)rrr) rjrr'catrrlfliprr)r<rrrvr>rall_hidden_statesall_attentions batch_size num_keypointsr8attention_outputself_attentionsintermediate_states output_statesself_attention_descriptorsself_attention_hidden_statesrrcross_attention_outputcross_attentionscross_intermediate_statescross_output_statescross_attention_hidden_statess r.rLz!LightGlueTransformerLayer.forwards #7BD0d  1[N B 4?4E4E1 M>-1,?,?  ))/ -@- )/ $ii6F(GRP &9: %0=%@" ,?+O ( ' . .r1m^ T T!W WZ ? )  " "2q!Q > C CA F N Nz[\^_an o 483G3G &"7#9/ 4H4 0 0 %*II/IKa.bhj$k!"nn-FG03FF -FH[,\ )!-55j-Q_`bc./&&z=.QST0 0  +.@@DTCVVN-~==r-)FF) r#r$r%rrr6r'rNrrOr+rLrPrQs@r.rrs..3.05,1 H>\\H><<H> H> 'tn H> $D> H> u||XeELL&9:HU5<r-r similarity matchability0 matchability1c|j\}}}tjj|tjj|j ddz}tjj |d}tjj |j ddj dj dd}|j||dz|dzfd} ||z|z| ddd|d|f<tjj|jd | dddddf<tjj|jd | dddddf<| S)z;create the log assignment matrix from logits and similarityrrrArSrN) rjrr logsigmoidr log_softmaxrnew_fullsqueeze) rrrrnum_keypoints_0num_keypoints_1 certaintiesscores0scores1scoress r.sigmoid_log_double_softmaxr[sS4>3C3C0J--**=9BMM--k: % T5H5HQ^QeQe(fjn(nn %--jAoq-Q_` &q!t,&q!t,#n&>&>r2&FF  << aMBDAJ((,EAJ((,66r2>E5=D#// 5;;zGWGW;X;\;\]J((5 #++J!OQ qQ %ad+%ad+,JW r-c|j|}tjj|j d}|S)z0Get matchability of descriptors as a probabilityrA)rrrsigmoidr)r<rrs r.get_matchabilityz.LightGlueMatchAssignmentLayer.get_matchabilitys7((5 }},,\:BB2F r-) r#r$r%rr6r'rNrLrrPrQs@r.rrjsRII5<<u|| 4ELLU\\r-rc\eZdZdeffd Zdej dej fdZxZS)LightGlueTokenConfidenceLayerr1clt|tj|jd|_yNr)r5r6rr7r8tokenr;s r.r6z&LightGlueTokenConfidenceLayer.__init__s& YYv44a8 r-rr?c|j|j}tjj |j d}|S)NrA)rdetachrrrr)r<rrs r.rLz%LightGlueTokenConfidenceLayer.forwards= ;--/0 %%e,44R8 r-rrQs@r.rrs*99 5<<ELLr-rc0eZdZUdZeed<dZdZdZdZ dZ y)LightGluePreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. r1 lightglue pixel_valuesFTN) r#r$r%r&rr)base_model_prefixmain_input_namesupports_gradient_checkpointing_supports_flash_attn_supports_sdpar,r-r.rrs+ #$O&+#Nr-rr thresholdc6|j\}}}|ddddddfjd}|ddddddfjd}|j}|j}tj|jd|j d}tj|jd|j d} ||j d|k(} | |j d|k(} |jj}|jd} tj| || } tj| | j d|| }| | |kDz}| |j d|z}tj||d}tj||d}tj||gjddj|dzd}tj| |gjddj|dzd}||fS)z1obtain matches from a score matrix [Bx M+1 x N+1]NrArrrr)rjmaxindicesr'arangergathervaluesexp new_tensorwhererTrrl)rrr_max0max1matches0matches1indices0indices1mutual0mutual1zeromatching_scores0matching_scores1valid0valid1rrs r.get_matches_from_scoresrs||J1 !SbS#2#+  " "1 %D !SbS#2#+  " "1 %D||H||H||HNN1-hooFtLH||HNN1-hooFtLH(//!X66G(//!X66G ;;?? D ??1 D{{7D$7{{7,<,C,CAx,PRVW (94 5F v}}Q1 1F{{68R0H{{68R0Hkk8X./99!Q?GG UVXZ[Gkk#35E"FGQQRSUVW__`jmn`nprsO O ##r-rheightwidthctj||g|j|jd}|dz }|j dj dz }||ddddfz |dz }|S)a Normalize keypoints locations based on image image_shape Args: keypoints (`torch.Tensor` of shape `(batch_size, num_keypoints, 2)`): Keypoints locations in (x, y) format. height (`int`): Image height. width (`int`): Image width. Returns: Normalized keypoints locations of shape (`torch.Tensor` of shape `(batch_size, num_keypoints, 2)`). rr]NrrA.).NN)r'rrr]rr )rrr sizeshiftscales r.normalize_keypointsr&sp << 0@0@  XY] ^D 1HE HHRL  ! #EU3a<00E/4JJI r-zV LightGlue model taking images as inputs and outputting the matching of them. ceZdZdZdeffd ZdedefdZ d"de jde jd e e de e je e je jfffd Zd e jded e jd e jde jf dZd#dZde jde jdede jfdZde jde jd e jde jde jd e jdefdZdZde jde jde jde jde e je jff dZ d$de jde jdeded e jde e d e e de e je je je e ffdZee d$de j0d e e j2de e d e e dee eff d!ZxZS)%LightGlueForKeypointMatchingan LightGlue is a model matching keypoints in images by leveraging detections from a keypoint detector such as SuperPoint. It is based on the SuperGlue architecture and is designed to be lightweight and efficient. It consists of : 1. Keypoint Encoder 2. A Graph Neural Network with self and cross attention layers 3. Matching Assignment layers The correspondence ids use -1 to indicate non-matching points. Philipp Lindenberger, Paul-Edouard Sarlin and Marc Pollefeys. LightGlue: Local Feature Matching at Light Speed. In ICCV 2023. https://arxiv.org/pdf/2306.13643.pdf r1c ,t||tj|j|j |_|jj|_|j|_ |j|_ |j|_ |j|_ |j|_|j|jk7r2tj |j|jd|_ntj$|_t'||_tj*t-|jDcgc]}t/||c}|_tj*t-|jDcgc] }t3|c}|_tj*t-|jdz Dcgc] }t7|c}|_|j;ycc}wcc}wcc}w)N)trust_remote_codeTr3)rr)r5r6r from_configkeypoint_detector_configr*keypoint_detectordescriptor_decoder_dim keypoint_detector_descriptor_dimr8num_hidden_layers num_layersfilter_thresholddepth_confidencewidth_confidencerr7input_projectionIdentityr0positional_encoder ModuleListrangertransformer_layersrmatch_assignment_layersrtoken_confidence post_init)r<r1irr=s r.r6z%LightGlueForKeypointMatching.__init__s  !>!J!J  + +v?W?W" 170O0O0f0f-$33 22 & 7 7 & 7 7 & 7 7   $"G"G G$&IId.S.SUYUhUhos$tD !$&KKMD !"c|jj}|j|}|j||}||fS)Nr>)rrr5r7)r<rrr>projected_descriptorskeypoint_encoding_outputs r._keypoint_processingz1LightGlueForKeypointMatching._keypoint_processingsO"((*557 $ 5 5k B#'#:#:9[o#:#p $&>>>r-keypoint_confidencesr num_pointsc||j\}}||jdz kru|j|dk(d}|j|dzd}|j |}d||kj j d|z z }||jkD} | Stj|tj} | S)zRevaluate whether we should stop inference based on the confidence of the keypointsrrrrAg?rBr\) rjr1rrlrCr^sumr3r'onesrO) r<rIr?r rJrrrratio_confidentearly_stopped_pairss r._get_early_stopped_image_pairsz;LightGlueForKeypointMatching._get_early_stopped_image_pairs%s A 1, ,$8#C#CDAIq#Q #7#?#? aQS#T 66{CI!%9I%E$L$L$N$R$RWX$R$Y\f$ffO"1D4I4I"I  #"#(**Zuzz"J ""r-c| ||}||}|j|||}t||j\}}||fSr)r;rr2)r<rr r? early_stopsrrrs r._get_keypoint_matchingz3LightGlueForKeypointMatching._get_keypoint_matching8sW  "%k2K $D:--k:;M#:64CXCX#Y ''r- confidencesrc\|d|jz kD}||||j|kz}|S)z#mask points which should be removedr)r4rC)r<rTrr?keeps r._get_pruning_maskz.LightGlueForKeypointMatching._get_pruning_mask@s<T2223  " K4#A#A+#NN ND r-r  prune_outputc|j\}} } |j|j|} |j|| |j |dk(t j dfd||d|d|fD\} } } }}t|D]}||||fxxdz cc<d| | | |fD\} } } }| | f}t|dd}| ||||fS) z For a given layer, prune keypoints based on the confidence of the keypoints and the matchability of the descriptors. rFc3nK|]&}t|Dcgc] \}}|| c}}(ycc}}wwr)zip).0rtr pruned_keypoints_masks r. zJLightGlueForKeypointMatching._do_layer_keypoint_pruning..[s9c %(0E$F GDQtW Gc Gs 5/5rc36K|]}t|dyw)T) batch_firstNr)r\ pruned_tensors r.r_zJLightGlueForKeypointMatching._do_layer_keypoint_pruning..cs$S  D 9 9S sTrAra padding_value) rjr;rrWrr'rr9r )r<rrr r rXrIr?rrdescriptors_matchabilitypruned_descriptorspruned_keypoints_0pruned_keypoints_1 pruned_maskpruned_indicesr>pruned_keypointsr^s @r._do_layer_keypoint_pruningz7LightGlueForKeypointMatching._do_layer_keypoint_pruningGs+',, Aq#'#?#? #L#]#]^i#j $ 6 67KMegr s 5 A A$!)U\\Z_M` ac & ! ilDY[bcc _.0BKQ_z" 4A N1-- .! 3 . 4S "46HJ\^i!jS O.0BK/0BC%n$VXY!#3^[R^^^r-ctjd||fD\}}d||fD\}}fd||||fD\}}}}||||fS)Nc38K|]}t|ddyw)TrArcNrr\rs r.r_zMLightGlueForKeypointMatching._concat_early_stopped_outputs..us$3  T D D3 c38K|]}t|ddyw)TrrcNrros r.r_zMLightGlueForKeypointMatching._concat_early_stopped_outputs..ys$>  T C C> rpc3(K|] }| ywrr,)r\rearly_stops_indicess r.r_zMLightGlueForKeypointMatching._concat_early_stopped_outputs..}s!g  & 'g s)r'rT)r<rsfinal_pruned_keypoints_indices!final_pruned_keypoints_iterationsrrs ` r._concat_early_stopped_outputsz:LightGlueForKeypointMatching._concat_early_stopped_outputsls$kk*=>3 "$BC3 //> *,MN> ::g .1 g c"@Bc./PRY[jjjr-rrrc |j\}fd|||fD\}}}|dddf}|dddf}|dddf}|dddf} |dddf} |dddf} tjdzd|fd|j|j} tj dzd|f|j|j} t dzD]}tj||dk(d||jd||jd| |d||f<tj| |dk(d||jd| |jd| |d||f<| || |d||f<| || |d||f<| | fS)Nc3JK|]}|jdzddyw)rrAN)rl)r\rrs r.r_zJLightGlueForKeypointMatching._do_final_keypoint_pruning..s'- 7=FNN:?Ar 2- s #rrrrAr")r) rjr'fullrr]zerosr9rr clamp)r<r rrrrrrrrrr_matches_matching_scoresr>rs @r._do_final_keypoint_pruningz7LightGlueForKeypointMatching._do_final_keypoint_prunings A- BI7TcAd- )/1a4=1a4=1a4=1a4=*1a40*1a40::zQ=A2gnndkdqdqr ;; 1_a /oNcNc zQ' FA*/++ r!2x{'9'9!Xa[=N=NST=N=U'V+HQ8A;& '+0++ r!2x{'9'9!Xa[=N=NST=N=U'V+HQ8A;& '3C12E Q8A;. /2B12E Q8A;. / F)))r-rr rc & (|rdnd}|rdnd} |jddk(rT|jdd} |j| dtj|j | |j | || fS|j } |j\} } }} tj |j| dd}|j| dz|d}||j| dz|nd}|j| dz||j}tj| dz| }t|||}|j||| \}}|d}|jdkD}|jdkD}g}g}g}g}g}tjd|| j| dzd}tj|}t!|j"D]3}|j%}||j'||}n&tj(| |d f|j }|j*|||||| }|\}}} |r||z}|r| | z} |r||j"dz kr+|j,||}!|j/|!||| }"n%tj(| tj0}"tj2|"r|"j5d(|(}#|j7|||( \}$}%|j9t;|#|j9t;|$|j9t;|%|r:|j9t;|(|j9t;|(||"}t=(fd||d|d||fD\}}&}'}}|&|'f}|rt=(fd||!fD\}}}!tj>|"rn$|s|jA|||||!|\}}}}}6|r4|r2|jC|||||\}}}}|jE||||\}}nE|j7|||j"dz \}}tj||j"z}|j| d|}||||| fS)Nr,rrrAr\rrBrrErS)rvr>r)rJ)rRc3*K|] }| ywrr,r\rrRs r.r_zALightGlueForKeypointMatching._match_image_pair..s"V" |,Vc3*K|] }| ywrr,rs r.r_zALightGlueForKeypointMatching._match_image_pair..s"l &#K<0lr)#rjrr'r new_zerosrrLrlr/r r&rHr3r4rk ones_liker9r1r#get_extended_attention_maskrMr:r<rPrOanyrDrSextendlistr+allrlrvr~))r<rrrr r rr>rrrjrrrinitial_num_keypointsnum_points_per_pair image_indicesrG do_early_stopdo_keypoint_pruningrsrrrtrupruned_keypoints_indicespruned_keypoints_iterationsr?rextended_attention_mask layer_outputr! attentionrIrOearly_stopped_image_indicesearly_stopped_matchesearly_stopped_matching_scores keypoints_0 keypoint_1rRs) @r._match_image_pairz.LightGlueForKeypointMatching._match_image_pairsg#7BD0d ??1  "OOCR(E""5"EII">##E*##E*!  !!2;/// A,a#ii Z(D!L%%j1n6KQO FJFVt||JN,AB\`!))*q.:OQUQvQvw  Z!^FC ' 65A 040I0I 9M1J1 - --Q/ --1 #33a7 )+&,.)#(<<3HQW#X#_#_`jmn`npr#s &+oo6N&O# 1R K%**,K*.*J*J4Q\*]'*/**j+b/5R[d[k[k*l'?422;?6%9"3 L5A 1K #$5 $E! !/)!;1!44+M4+@+@+Mk+Z(+/*M*M,k4L_+N+' +0**Zuzz*R'9901#6"G"G"JK2? 2L/KOKfKf#T;KLgLH)+H(..t4O/PQNN4(=#>?#**40M+NO*6==dC[\gCh>ij9@@FabmFnAop+>?R>R*S'PUV'2IaL)A,PTVc&dVQMKj$ "-j 9I*fkl!9 ; 4+lgc02MOc9901"33#!03,#d Y(@$HcQR h 022'25#  h *,MwXg(,'F'F.% ( $G_(,'B'B;PTVZVeVehiVi'j $G_050PSWSbSb0b -,M,U,U 0- )   -     r-rlabelsc 0d}| td||n|jj}||n|jj}|jdk7s|j ddk7r td|j \}}}} } |j|dz|| | }|j|} | dd\} }} }| j|dddj|} | j|dd|jj|} |j|dd}| j}|dddddddf| z|dddddddf<|dddddddf| z|dddddddf<|j|| | | ||| \}}}}}t|||| |||| S) Nz9LightGlue is not trainable, no labels should be provided.rrzOInput must be a 5D tensor of shape (batch_size, 2, num_channels, height, width)rAr)r rr>)rrrrrr r!r") ValueErrorr1rr>ndimr#rjrlr-r`r/clonerr)r<rrrr>rrrchannelsrr keypoint_detectionsrrr absolute_keypointsrrrr!r"s r.rLz$LightGlueForKeypointMatching.forwardZs  XY Y1B1N-TXT_T_TqTq$8$D $++JjJj     !\%6%6q%9Q%>no o1=1C1C. Ax#++JNHfeT "44\B*=bq*A' 1k4%%j!R;>>|L !))*aT=b=bcffgst ||J2.&__.);Aq!QJ)G%)O1aA:&);Aq!QJ)G&)P1aA:&EIE[E[    /!5F\F B% /+'!  r-rMr)NNN)r#r$r%r&rr6rr^rCr'rNrrOr+rHrPrSrWrlrvr~rrrr( LongTensorrrrLrPrQs@r.r(r(s @(S(U( jo? <<?49LL?X`aeXf? u||U5<<#=>> ??#$)LL#?B#JO,,#didpdp# #&(U\\5<<^afkfrfr#_\\#_<<#_ll #_  #_ ll #_$ll#_#_Jk8#*#*#* #* || #* u||U\\) * #*V",0/3k <<k \\k  k  k ll k $D>k 'tnk  u||U\\5<<E Fk Z.2,0/3 3 ''3 ))*3 $D> 3 'tn 3 u55 6 3 3 r-r(r)r)5 dataclassesrtypingrrrnumpyrAr'rtorch.nn.utils.rnnr activationsr modeling_flash_attention_utilsr modeling_utilsr rprocessing_utilsrutilsrrr utils.genericrauto.modeling_autorconfiguration_lightgluerrModuler0rZrgrNrrqr^rrrrrrrrr+rr&r(__all__r,r-r.rsU(",, +!BF&DD->4  :k :  :F"<< UU\\ U# U%,, U&% II% <<% % << % U\\* %  %%'(%4D)D)N299"P> P>f   -2\\ JO,,  \\ &BII&R BII    $ELL$U$uU\\[`[g[gMgGh$@5<<SU\\, f #;f  f R &'E Fr-