B һdK5 @sdZddlZddlZddlZddlZddlZddlZddlm Z ej ej ej ej ej ej eZej ejZdZdZddZdd Zd d Zd d ZddZddZddZdddZdddZdddZdS)zJClasses and functions that help to inspect Python source w.r.t. TF graphs.N) profilingz.py)z.pycz.pyocCstjtj|S)N)ospathnormpathabspath) file_pathrZ/var/www/html/venv/lib/python3.7/site-packages/tensorflow/python/debug/lib/source_utils.py_norm_abs_path'sr cCstj|\}}|tkS)N)rrsplitextlowerUNCOMPILED_SOURCE_SUFFIXES)r_ extensionrrr %is_extension_uncompiled_python_source+srcCstj|\}}|tkS)N)rrr r COMPILED_SOURCE_SUFFIXES)rrrrrr #is_extension_compiled_python_source0srcCs|d|dS)N:)rfind) watch_keyrrr !_convert_watch_key_to_tensor_name5srcCsTt|st|sdSt|}|ts0|toR|d oRtj dtj |kS)aGuess whether a Python source file is a part of the tensorflow library. Special cases: 1) Returns False for unit-test files in the library (*_test.py), 2) Returns False for files under python/debug/examples. Args: py_file_path: full path of the Python source file in question. Returns: (`bool`) Whether the file is inferred to be a part of the tensorflow library. Fz_test.pyz tensorflow/python/debug/examples) rrr startswith_TENSORFLOW_BASEDIR _ABSL_BASEDIRendswithrrr)Z py_file_pathrrr guess_is_tensorflow_py_library9s    rc Cs|tj|rsz(_try_load_par_source..zutf-8rN)rrr#rrelpathrrzipfileZipFilenamelistr indexr!r")r+ prefix_pathbasenameZ suffix_pathzZ norm_nameszfr-rrr r$ws r$FcCs|j}|stdt|}i}x|D]}xt||jD]\} } } } |dk rZ| |ks>|dk rl| |krlq>t| |krzq>|r||j} tt t t | } n|jg} | |kr||  | n| || <|r>Pq>Wq(W|S)aAnnotate a Python source file with a list of ops created at each line. (The annotation doesn't change the source file itself.) Args: dump: (`DebugDumpDir`) A `DebugDumpDir` object of which the Python graph has been loaded. source_file_path: (`str`) Path to the source file being annotated. do_dumped_tensors: (`str`) Whether dumped Tensors, instead of ops are to be used to annotate the source file. file_stack_top: (`bool`) Whether only the top stack trace in the specified source file is to be annotated. min_line: (`None` or `int`) The 1-based line to start annotate the source file from (inclusive). max_line: (`None` or `int`) The 1-based line number to end the annotation at (exclusive). Returns: A `dict` mapping 1-based line number to a list of op name(s) created at that line, or tensor names if `do_dumped_tensors` is True. Raises: ValueError: If the dump object does not have a Python graph set. zUCannot perform source annotation due to a lack of set Python graph in the dump objectN) python_graph ValueErrorr get_operationsreversednode_tracebackr1debug_watch_keyslistsetmaprextend)dumpr+Zdo_dumped_tensorsZfile_stack_topmin_linemax_linepy_graphZline_to_op_namesopr line_numberr watch_keysZitems_to_appendrrr annotate_sources.   rMc s"|j}|stdtt}tt}i}i|r0sz1list_source_files_against_dump..rcSs|dS)Nrr)xrrr =z0list_source_files_against_dump..)key)r<r= collections defaultdictrCrecompiler>matchr1r@r rrraddoutputsrAr#r*Z get_tensorsr&sumappendrrNsorted)rFZpath_regex_allowlistZnode_name_regex_allowlistrIZpath_to_node_namesZpath_to_tensor_namesZpath_to_first_lineZ path_regexnode_name_regexZto_skip_file_pathsrJrrKrZ output_tensorrOrLr node_nameZ output_slotZdebug_opZpath_to_num_dumpsroutputr)rPr list_source_files_against_dumpsd"           &    rcc Cst|}|rt|nd}|r(t|nd}i}x|D]} | jsBq6t| j|krRq6|dk rd| j|ks6|dk rx| j|krxq6|r|| jjsq6|r|| jsq6| j|krt | || j<q6|| j | q6W|S)aAnnotate a Python source file with profiling information at each line. (The annotation doesn't change the source file itself.) Args: profile_data: (`list` of `ProfileDatum`) A list of `ProfileDatum`. source_file_path: (`str`) Path to the source file being annotated. node_name_filter: Regular expression to filter by node name. op_type_filter: Regular expression to filter by op type. min_line: (`None` or `int`) The 1-based line to start annotate the source file from (inclusive). max_line: (`None` or `int`) The 1-based line number to end the annotation at (exclusive). Returns: A `dict` mapping 1-based line number to a the namedtuple `profiling.LineOrFuncProfileSummary`. N) r rXrYrrKrZZnode_exec_statsraZop_typerZAggregateProfiler[) Z profile_datar+Znode_name_filterZop_type_filterrGrHr`Z op_type_regexZline_to_profile_summaryZ profile_datumrrr annotate_source_against_profile@s*  rd)FFNN)NN)NNNN)__doc__rVrrXr4Zabslnumpyr'Ztensorflow.python.debug.librrdirnamerr__file__rrr rr rrrrr.r$rMrcrdrrrr s< %# ? e