rhddlZddlZddlmZmZddlZddlmcmZddl m Z ddl m Z ddl mZmZmZmZmZmZddlmZddlmZddlmZej4eZGd d e j:ZGd d e Zy) N)AnyCallable)ir)KernelTemplate)Bufferget_free_symbolsget_symbolic_inputsgm_original_output_stridesir_node_to_tensorLayout) benchmarker)do_bench_using_profiling)Vc eZdZdZdedeedededede fdd f fd Z defd Z d ee d e jdefdZdefdZdej$fdZdeee ffdZdefdZxZS)SubgraphChoiceCallerz Represents a Subgraph Autotuning choice, and the subgraph can be any arbitrary GraphModule. Compiles the Subgraph down to a module for benchmarking. name input_nodeslayout description make_fx_graph.returnNcLt|||||g|_tj5|j D]}t t|jddk(sJt t|jddk(sJ|jj|jjt| ddd||j|_t|jt!|j |_y#1swYMxYw)NT) unbacked_onlyr)super__init__example_inputsr fake_moderlenrget_size get_stridedata freeze_layoutappendr gmr r sym_inputs)selfrrrrrinp __class__s s/var/www/html/ai-insurance-compliance-backend/venv/lib/python3.12/site-packages/torch/_inductor/codegen/subgraph.pyrzSubgraphChoiceCaller.__init__s {FK@  [[ C'' C+CLLN$OPTUUUU+CNN,.>? C Cs B"DD#c"d|jdS)NzSubgraphCaller()rr&s r)__str__zSubgraphChoiceCaller.__str__8s  1--argsoutc  ddlmcm}ddlm}||j |j tjjtjjtjjtjjtjjtjjd|j }|j D]@}||j"|j<|j$j'|jB|j Dcgc]>}t)tjj*j,j/|@c} t1 dk(rt3|j D]z\}} t5|t6j8s!t5| t6j8sJ|j:| j:k(sJ|j=| j=k(rzJt1 dk(rt3|j D]z\}} t5|t6j8s!t5| t6j8sJ|j:| j:k(sJ|j=| j=k(rzJtj>|5|jAddd5|jB|j |jE} | jF  g ddddddtjHrtK fdStMjN fd Scc}w#1swYPxYw#1swYTxYw) Nr) GraphLowering benchmark_) r$r shape_env cpp_wrapperaot_modeextern_node_serializer is_inference is_backwardrFATEN) max_autotunemax_autotune_gemmmax_autotune_gemm_backendscgSNr0bm_funcr%sr)z0SubgraphChoiceCaller.benchmark..zsG.{s9M:9M9M1Nr/)(torch._inductor.config _inductorconfigtorch._inductor.graphr3r$rrgraph _shape_envr6r7r8r9r:rr% graph_inputsgraph_input_namesr#intsizevarsr5 size_hintrzip isinstancetorchTensorshapestrideset_graph_handlerpatchruncompile_to_modulecall/profile_bandwidth_with_do_bench_using_profilingrr benchmark_gpu) r&r1r0inductor_configr3bm_graph_loweringsym_inpsym_varar example_inpmodrCr%s ` @@r) benchmarkzSubgraphChoiceCaller.benchmark;s 987)ww..gg((++WW%%#$77#A#A--++dii[)   EG;B  * *7<< 8  / / 6 6w|| D E  ??    **44W= > z?a $'tT-@-@#A ?Kb%,,/%k5<<@@@88{'8'888899;+*<*<*>>>>  ? z?a $'tT-@-@#A ?Kb%,,/%k5<<@@@88{'8'888899;+*<*<*>>>>  ? !2 3 . &&""'+1' . &!%%t':':;'99;((,*,t,- . .  A A+,QR R(()NOOS : . . . .s,2AM0N$AM5&N5M> :NN c vdj|jjdddg|jDcgc]}t |j c}|jDcgc]}t |j c}t |jjScc}wcc}w)N-_r) joinrrsplitrstrrr r$rJ)r&r's r)hash_keyzSubgraphChoiceCaller.hash_key}sxx   a(+ 151A1AB##clln%B 483C3CDC#cnn&'D DGGMM"    CDs B1 * B6 c tjjtj|j|j |j |j|jS)N)rrr$r subgraph_name) r TensorBoxcreateSubgraphBufferrrr$rrr-s r) output_nodez SubgraphChoiceCaller.output_nodesN||""   {{ ,,77#22"ii    r/c d|jdS)zRInformation returned here is logged to the autotune log file when that is enabled.subgraph)backend kernel_namer,r-s r) info_dictzSubgraphChoiceCaller.info_dicts"99  r/c d|jS)N subgraph_r,r-s r)autoheuristic_idz%SubgraphChoiceCaller.autoheuristic_ids499+&&r/)__name__ __module__ __qualname____doc__rllistrr rrrr.rSrTfloatrermrrprsdictrxr{ __classcell__)r(s@r)rrs @@&\@ @  @  S) @ @2..@PtCy@Pu||@P@PD #   R\\   4S> '#'r/rcjeZdZdZej Zdedede ffdZ de e de de d efd Zy ) SubgraphTemplatea  A template for subgraph evaluation to be used in autotuning. This class allows creating customized subgraphs that can be appended as choices during the autotuning process, enabling the selection of optimal implementations for complex operations. rr.cX|dttj|_||_y)z Initialize a subgraph template. Args: name: The name of this template graph: The FX graph rhN)nextr index_counterrr)r&rrs r)rzSubgraphTemplate.__init__s+fAd#3#A#ABCD *r/rrkwargsrc Jt|j||d|jS)a Generate a SubgraphChoiceCaller instance for autotuning. Args: input_nodes: List of input nodes to the subgraph layout: Memory layout information for the output example_inputs: Example tensor inputs used to trace and benchmark the subgraph **kwargs: Additional keyword arguments Returns: SubgraphChoiceCaller: A callable object that can be used for autotuning )rrrrr)rrr)r&rrrs r)generatezSubgraphTemplate.generates*&$#,,   r/N)r|r}r~r itertoolscountrrlrrrrrr rrrAr/r)rrsf$IOO%M + + S) + &\      r/r) rloggingtypingrrrSrFrGrHtorch._inductorrtorch._inductor.codegen.commonrtorch._inductor.irrrr r r r $torch._inductor.runtime.benchmarkingr torch._inductor.utilsrtorch._inductor.virtualizedr getLoggerr|log ChoiceCallerrrrAr/r)rse ''9=:)g!A'2??A'H3 ~3 r/