B ӻda@sdZddlZddlZddlmZddlmZddlmZddlm Z ddl m Z ddl m Z dd lmZd Zd Zd d ZddZdddZdddZedgddddZdS)zqLogging tensorflow::tfprof::OpLogProto. OpLogProto is used to add extra model information for offline analysis. N)tfprof_log_pb2)context)ops) tensor_shape)gfile)flops_registry) tf_exportZ_trainable_variablesZflopsc Cs,x$|jjD]}x|jD]}|js*qy||j}Wn$tk r^}zwWdd}~XYnXt|jt|jkrvqxt |jD]\}}|j| rq|j j j}tdd|D} y$|j||j| | Wqtk r}ztjd|j|fWdd}~XYqXqWqWq W|S)zBFill Tensor shapes in 'graph' with run time shape from 'run_meta'.NcSsg|] }|jqS)size).0dr r Z/var/www/html/venv/lib/python3.7/site-packages/tensorflow/python/profiler/tfprof_logger.py 5sz-_fill_missing_graph_shape..z!Node %s incompatible shapes: %s. )Z step_statsZ dev_statsZ node_statsoutputZget_operation_by_nameZ node_nameKeyErrorlenoutputs enumerate get_shapeZis_fully_definedZtensor_descriptionshapedimrZ TensorShape set_shapeZ merge_with ValueErrorsysstderrwrite) graphrun_metaZdev_statZ node_statopeiZ node_stat_outZnode_stat_dimsZnode_stat_shaper r r _fill_missing_graph_shape"s.  ,r!cCs(||d}|dkr$t|}|||<|S)zMaps string to id.N)getr)sZ str_to_idnumr r r _str_id?s  r%Tc Cs|rt||}d}i}i}t||d<x |D]}yt||jt}Wn tk rl|d7}d}YnXt } |j | _ d} |r|j rt |j | _ d} |r$|jr xt|jD]j\} } } }| jj}| rt| |nd|_| r| nd|_| rt| |nd|_|rt||nd|_d|_qWd} | r2| || j <q2W|rxd|tjjD]R}|jj |krt } |jj | _ | jt| || j <n||jj jtqLW|dkr|stj d|||fS) aExtract trainable model parameters and FLOPs for ops from a Graph. Args: graph: tf.Graph. run_meta: RunMetadata proto used to complete shape information. add_trace: Whether to add op trace information. add_trainable_var: Whether to assign tf.compat.v1.trainable_variables() op type '_trainable_variables'. Returns: logged_ops: dict mapping from op_name to OpLogEntry. string_to_id: dict mapping from string to id. rnoneNFTz0%d ops no flops stats due to incomplete shapes. )!r!rZget_operationsrZget_stats_for_node_defZnode_defREGISTERED_FLOP_STATSrrZ OpLogEntrynamevalueint float_ops tracebackcode_deftracesaddr%Zfile_idlinenoZ function_idZline_idZfunc_start_lineZget_collectionZ GraphKeysTRAINABLE_VARIABLESrtypesappendrrr)rr add_traceadd_trainable_varZop_missing_shape logged_ops string_to_idrstatsentry add_entryfilenamer2funcnamelinetracevr r r _get_logged_opsHsV          rBc Cs&|stst}t}|s$|St||||d\}}|sN|j| ni}x|jD]} | || j <qZWx| D]v\} } | |kr|| j | j | j dkr|| j dkr| j || _ | jjr|| jjs|| j| jqv| || <qvW|j| x | D]\} } | |j| <q W|S)aMerge the tfprof default extra info with caller's op_log. Args: graph: tf.Graph. If None and eager execution is not enabled, use default graph. op_log: OpLogProto proto. run_meta: RunMetadata proto used to complete shape information. add_trace: Whether to add op trace information. add_trainable_var: Whether to assign tf.compat.v1.trainable_variables() op type '_trainable_variables'. Returns: tmp_op_log: Merged OpLogProto proto. )r6r7r)rexecuting_eagerlyrget_default_graphrZ OpLogProtorBZ log_entriesextendvaluesr*itemsr4r-r/r0Z MergeFromZ id_to_string) rop_logrr6r7Z tmp_op_logr8r9Zall_opsr;Zop_namer#r r r r merge_default_with_oplogs0    rIzprofiler.write_op_log)Zv1c CsV|stst}t||||}ttj |dd}| | WdQRXdS)aLog provided 'op_log', and add additional model information below. The API also assigns ops in tf.compat.v1.trainable_variables() an op type called '_trainable_variables'. The API also logs 'flops' statistics for ops with op.RegisterStatistics() defined. flops calculation depends on Tensor shapes defined in 'graph', which might not be complete. 'run_meta', if provided, completes the shape information with best effort. Args: graph: tf.Graph. If None and eager execution is not enabled, use default graph. log_dir: directory to write the log file. op_log: (Optional) OpLogProto proto to be written. If not provided, an new one is created. run_meta: (Optional) RunMetadata proto that helps flops computation using run time shape information. add_trace: Whether to add python code trace information. Used to support "code" view. Z tfprof_logwN) rrCrrDrIrZOpenospathjoinrZSerializeToString)rZlog_dirrHrr6logr r r write_op_logs  rO)NTT)NNTT)NNT)__doc__rKrZtensorflow.core.profilerrZtensorflow.python.eagerrZtensorflow.python.frameworkrrZtensorflow.python.platformrZ#tensorflow.python.profiler.internalrZ tensorflow.python.util.tf_exportrr3r)r!r%rBrIrOr r r r s&         D .