B лd@svdZddlZddlZddlZddlZddlZddlmZddlZ ddl m Z ddl m Z ddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZddl m!Z"ddl#m$Z$ddl#m%Z%ddl#m&Z&ddl#m'Z'ddl(m)Z)ddl(m*Z*ddl+m,Z,ddl-m.Z.dZ/e0dddgZ1ddZ2ddZ3d d!Z4d"d#Z5ddd$d%Z6d&d'Z7ded)d*Z8dfd+d,Z9d-d.Z:d/d0Z;d1d2Zdgd8d9Z?d:d;Z@dhd=d>ZAd?d@ZBdAdBZCdCdDZDdEdFZEdGdHZFdIdJZGdKdLZHdMdNZIdOdPZJdQdRZKdSdTZLdUdVZMdWdXZNdYdZZOd[d\ZPd]d^ZQd_d`ZRdadbZSeTdckrreUeSdS)izCommand-line interface to inspect and execute a graph in a SavedModel. For detailed usages and examples, please refer to: https://www.tensorflow.org/guide/saved_model#cli_to_inspect_and_execute_savedmodel N)app) example_pb2) types_pb2) config_pb2)session)local_cli_wrapper) def_function)function) meta_graph)ops) tensor_spec)file_io) tf_logging)load)loader)save)signature_constants)saved_model_aot_compile)saved_model_utils)tpu)collections_abczchttps://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/xla/debug_options_flags.ccZ WriteFileZReadFileZPrintV2cCs>t|}tdx&t|D]}tddt|qWdS)zPrints the tag-sets stored in SavedModel directory. Prints all the tag-sets for MetaGraphs stored in SavedModel directory. Args: saved_model_dir: Directory containing the SavedModel to inspect. z5The given SavedModel contains the following tag-sets:z%rz, N)rget_saved_model_tag_setsprintsortedjoin)saved_model_dirtag_setstag_setrY/var/www/html/venv/lib/python3.7/site-packages/tensorflow/python/tools/saved_model_cli.py_show_tag_sets>s r cCs8t||}tdx t|D]}td|q WdS)aPrints the keys for each SignatureDef in the SignatureDef map. Prints the list of SignatureDef keys from the SignatureDef map specified by the given tag-set and SavedModel directory. Args: saved_model_dir: Directory containing the SavedModel to inspect. tag_set: Group of tag(s) of the MetaGraphDef to get SignatureDef map from, in string format, separated by ','. For tag-set contains multiple tags, all tags must be passed in. zQThe given SavedModel MetaGraphDef contains SignatureDefs with the following keys:zSignatureDef key: "%s"N)get_signature_def_maprrkeys)rrsignature_def_mapsignature_def_keyrrr_show_signature_def_map_keysLs r%cCs6||jkr*td|dd|j|j|jS)aGets TensorInfo for all inputs of the SignatureDef. Returns a dictionary that maps each input key to its TensorInfo for the given signature_def_key in the meta_graph_def Args: meta_graph_def: MetaGraphDef protocol buffer with the SignatureDef map to look up SignatureDef key. signature_def_key: A SignatureDef key string. Returns: A dictionary that maps input tensor keys to TensorInfos. Raises: ValueError if `signature_def_key` is not found in the MetaGraphDef. zCould not find signature "z". Please choose from: z, ) signature_def ValueErrorrr"inputs)meta_graph_defr$rrr+_get_inputs_tensor_info_from_meta_graph_def_s r*cCs |j|jS)aGets TensorInfos for all outputs of the SignatureDef. Returns a dictionary that maps each output key to its TensorInfo for the given signature_def_key in the meta_graph_def. Args: meta_graph_def: MetaGraphDef protocol buffer with the SignatureDefmap to look up signature_def_key. signature_def_key: A SignatureDef key string. Returns: A dictionary that maps output tensor keys to TensorInfos. )r&outputs)r)r$rrr,_get_outputs_tensor_info_from_meta_graph_defxsr,c st||}t||}t||}d|fdd}|dx2t|D]"\}} |d|t| |dqJW|dx2t|D]"\} } |d| t| |dqW|d |j|jd S) aPrints input and output TensorInfos. Prints the details of input and output TensorInfos for the SignatureDef mapped by the given signature_def_key. Args: saved_model_dir: Directory containing the SavedModel to inspect. tag_set: Group of tag(s) of the MetaGraphDef, in string format, separated by ','. For tag-set contains multiple tags, all tags must be passed in. signature_def_key: A SignatureDef key string. indent: How far (in increments of 2 spaces) to indent each line of output. z cst|dS)N)r)s) indent_strrrin_printsz&_show_inputs_outputs..in_printzBThe given SavedModel SignatureDef contains the following input(s):z inputs['%s'] tensor_info:zCThe given SavedModel SignatureDef contains the following output(s):z outputs['%s'] tensor_info:zMethod name is: %sN) rget_meta_graph_defr*r,ritems_print_tensor_infor& method_name) rrr$indentr)inputs_tensor_infooutputs_tensor_infor/ input_keyZ input_tensorZ output_keyZ output_tensorr)r.r_show_inputs_outputss$    r9c Cslt|j}d}x|D]}||dO}qW|s4dStt|}WdQRXtdddt t | |}t |ddd }x|D]\}}g}t|tjr||nt|tjr||nqtd |t |d dd }xxt|d D]j\} } d \} } | jr| j\} } n| jr(| j} | rNtd| tdt| dd| rt| dddqWqWdS)zPrints the callable concrete and polymorphic functions of the Saved Model. Args: saved_model_dir: Directory containing the SavedModel to inspect. FZobject_graph_defNz Concrete Functions:)endcSs|jS)N)name)xrrrz)_show_defined_functions..)keyz Function Name: '%s'cSs|jS)N)r<)r=rrrr>r?r0)NNz Option #%dz Callable with:)r5zNamed Argument)rread_saved_model meta_graphsZHasFieldops_libGraphZ as_defaultrrlistrZ_AugmentedGraphViewZ list_childrenr isinstancedefunZConcreteFunctionappendrFunctionextendZ._list_all_concrete_functions_for_serialization enumerateZstructured_input_signatureZ _arg_keywords _print_args) rrCZhas_object_graph_defr)Ztrackable_objectchildrenr<childZconcrete_functionsindexZconcrete_functionargskwargsrrr_show_defined_functionssF            rSArgumentc sd|dd}dfdd }xbt|dD]R\}}|dkrN|d ||ft|trf|d |q,t|tjrt|ddd |jt|fq,t|tj rt|t s|d t |j |d ddx |D]}td||ddqWtdq,t|t r^|d t |j |dddx2| D]&\}}tdt|||fddq*Wtdq,|d t |j |dt|q,WdS)zFormats and prints the argument of the concrete functions defined in the model. Args: arguments: Arguments to format print. argument_type: Type of arguments. indent: How far (in increments of 2 spaces) to indent each line of output. z cSsdt|t}|t||S)N')rGstr)valueZ is_quotesrrr_maybe_add_quotessz&_print_args.._maybe_add_quotes cst||ddS)N)r;)r)r-r;)r.rrr/sz_print_args..in_printr0rAz%s #%dz %sz%s: %sz DType: %sz Value: [r:)r;z%sz, z]z Value: {z'%s': %sz}z Value: %sN)rY)rLrGrVr Z TensorSpecrr<reprrIterabledicttype__name__r2) argumentsZ argument_typer5rXr/rPelementrWr@r)r.rrMs2   $       " rMcsd|fdd}|dddtjD|j|jjrDd}n(dd |jjD}d |}d |d }|d ||d|jdS)zPrints details of the given tensor_info. Args: tensor_info: TensorInfo object to be printed. indent: How far (in increments of 2 spaces) to indent each line output z cst|dS)N)r)r-)r.rrr/sz$_print_tensor_info..in_printz dtype: cSsi|]\}}||qSrr).0r@rWrrr sz&_print_tensor_info.. unknown_rankcSsg|]}t|jqSr)rVsize)radimrrr sz&_print_tensor_info..z, ()z shape: z name: N) rZDataTyper2ZdtypeZ tensor_shapercrerr<)Z tensor_infor5r/shapeZdimsr)r.rr3 s    r3cCst|}xlt|D]`}tdd|d|}t||}x4t|D]$}td|dt|||ddqLWqWt|dS) aPrints tag-set, SignatureDef and Inputs/Outputs information in SavedModel. Prints all tag-set, SignatureDef and Inputs/Outputs information stored in SavedModel directory. Args: saved_model_dir: Directory containing the SavedModel to inspect. zF MetaGraphDef with tag-set: '%s' contains the following SignatureDefs:z, ,z signature_def['z']:r0)r5N) rrrrrr!r"r9rS)rrrr#r$rrr _show_all%s   rkcCs t||S)a\DEPRECATED: Use saved_model_utils.get_meta_graph_def instead. Gets MetaGraphDef from SavedModel. Returns the MetaGraphDef for the given tag-set and SavedModel directory. Args: saved_model_dir: Directory containing the SavedModel to inspect or execute. tag_set: Group of tag(s) of the MetaGraphDef to load, in string format, separated by ','. For tag-set contains multiple tags, all tags must be passed in. Raises: RuntimeError: An error when the given tag-set does not exist in the SavedModel. Returns: A MetaGraphDef corresponding to the tag-set. )rr1)rrrrrr1<sr1cCst||}|jS)aGets SignatureDef map from a MetaGraphDef in a SavedModel. Returns the SignatureDef map for the given tag-set in the SavedModel directory. Args: saved_model_dir: Directory containing the SavedModel to inspect or execute. tag_set: Group of tag(s) of the MetaGraphDef with the SignatureDef map, in string format, separated by ','. For tag-set contains multiple tags, all tags must be passed in. Returns: A SignatureDef map that maps from string keys to SignatureDefs. )rr1r&)rrr rrrr!Rs r!cCstdd|dD}|S)NcSsg|] }|r|qSrr)raoprrrrfgsz(_get_op_denylist_set..rj)setsplit) op_denylistZset_of_denylisted_opsrrr_get_op_denylist_setesrpcCsFtt|j}||@}|r0td|jj|ntd|jj|dS)zScans meta_graph_def and reports if there are ops on denylist. Print ops if they are on denylist, or print success if no denylisted ops found. Args: meta_graph_def: MetaGraphDef protocol buffer. op_denylist: set of ops to scan for. z@MetaGraph with tag set %s contains the following denylisted ops:zFMetaGraph with tag set %s does not contain the default denylisted ops:N)rmmeta_graph_libZops_used_by_graph_defZ graph_defrZ meta_info_deftags)r)roZops_in_metagraphZdenylisted_opsrrrscan_meta_graph_defks rsFc  st||} t| |x:|D].} | kr td| dddfq Wfdd|D} t| |t} fdd| D}d}|rt d t j t j j d d d }tj|t|d }|rtd|tt||d|| r t|}|j|| d}xt|D]\}}| |}td||f|r$tj|sbt|tj||d}|stj |rt!d|t"#||td||fq$WWdQRXdS)aRuns SavedModel and fetch all outputs. Runs the input dictionary through the MetaGraphDef within a SavedModel specified by the given tag_set and SignatureDef. Also save the outputs to file if outdir is not None. Args: saved_model_dir: Directory containing the SavedModel to execute. tag_set: Group of tag(s) of the MetaGraphDef with the SignatureDef map, in string format, separated by ','. For tag-set contains multiple tags, all tags must be passed in. signature_def_key: A SignatureDef key string. input_tensor_key_feed_dict: A dictionary maps input keys to numpy ndarrays. outdir: A directory to save the outputs to. If the directory doesn't exist, it will be created. overwrite_flag: A boolean flag to allow overwrite output file if file with the same name exists. worker: If provided, the session will be run on the worker. Valid worker specification is a bns or gRPC path. init_tpu: If true, the TPU system will be initialized after the session is created. use_tfrt: If true, TFRT session will be used. tf_debug: A boolean flag to use TensorFlow Debugger (TFDBG) to observe the intermediate Tensor values and runtime GraphDefs while running the SavedModel. Raises: ValueError: When any of the input tensor keys is not valid. RuntimeError: An error when output file already exists and overwrite is not enabled. zK"%s" is not a valid input key. Please choose from %s, or use --show option."z", "csi|]\}}||jqSr)r<)rar@Ztensor)r6rrrbsz2run_saved_model_with_feed_dict..csg|]}|jqSr)r<)raZ tensor_key)r7rrrfsz2run_saved_model_with_feed_dict..NzUsing TFRT session.T)use_tfrt)Z experimental)graphconfigzInitializing TPU System ...rj)Z feed_dictzResult for output key %s: %sz.npyzXOutput file %s already exists. Add "--overwrite" to overwrite the existing output files.zOutput %s is saved to %s)$rr1r*r"r'rr2r,rlogginginforZ ConfigProtoZ ExperimentalrSessionrDrErrunrZinitialize_systemrrrnrZLocalCLIDebugWrapperSessionrLospathisdirmakedirsexists RuntimeErrornpr)rrr$Zinput_tensor_key_feed_dictoutdirZoverwrite_flagworkerinit_tpurutf_debugr)Zinput_key_nameZinputs_feed_dictZoutput_tensor_keys_sortedZoutput_tensor_names_sortedrwsessr+ioutputZoutput_tensor_keyZoutput_full_pathr)r6r7rrun_saved_model_with_feed_dictsV*"        rcCsi}|d}xxtt|D]j}td|}|rN|d|df||d<qtd|}|rx|ddf||d<qtd|qW|S) aiParses input arg into dictionary that maps input to file/variable tuple. Parses input string in the format of, for example, "input1=filename1[variable_name1],input2=filename2" into a dictionary looks like {'input_key1': (filename1, variable_name1), 'input_key2': (file2, None)} , which maps input keys to a tuple of file name and variable name(None if empty). Args: inputs_str: A string that specified where to load inputs. Inputs are separated by semicolons. * For each input key: '=' or '=[]' * The optional 'variable_name' key will be set to None if not specified. Returns: A dictionary that maps input keys to a tuple of file name and variable name. Raises: RuntimeError: An error when the given input string is in a bad format. ;z!([^=]+)=([^\[\]]+)\[([^\[\]]+)\]$r0z([^=]+)=([^\[\]]+)$Nzu--inputs "%s" format is incorrect. Please follow"=", or"=[]")rnfilterboolrematchgroupr) inputs_str input_dictZ inputs_raw input_rawrrrrpreprocess_inputs_arg_strings     rTcCsi}x~tt|dD]j}d|kr.td||dd\}}|rtyt|||<Wqtd|dYqXqt|||<qW|S)aParses input arg into dictionary that maps input key to python expression. Parses input string in the format of 'input_key=' into a dictionary that maps each input_key to its python expression. Args: input_exprs_str: A string that specifies python expression for input keys. Each input is separated by semicolon. For each input key: 'input_key=' safe: Whether to evaluate the python expression as literals or allow arbitrary calls (e.g. numpy usage). Returns: A dictionary that maps input keys to their values. Raises: RuntimeError: An error when the given input string is in a bad format. r=zV--input_exprs "%s" format is incorrect. Please follow"="r0z Expression "z " is not a valid python literal.)rrrnrast literal_evaleval)input_exprs_strsaferrr8exprrrr!preprocess_input_exprs_arg_string#srcCsRt|}xD|D]8\}}t|ts8td|t|fdd|D||<qW|S)aParses input into dict that maps input keys to lists of tf.Example. Parses input string in the format of 'input_key1=[{feature_name: feature_list}];input_key2=[{feature_name:feature_list}];' into a dictionary that maps each input_key to its list of serialized tf.Example. Args: input_examples_str: A string that specifies a list of dictionaries of feature_names and their feature_lists for each input. Each input is separated by semicolon. For each input key: 'input=[{feature_name1: feature_list1, feature_name2:feature_list2}]' items in feature_list can be the type of float, int, long or str. Returns: A dictionary that maps input keys to lists of serialized tf.Example. Raises: ValueError: An error when the given tf.Example is not a list. z?tf.Example input must be a list of dictionaries, but "%s" is %scSsg|] }t|qSr)_create_example_string)raexamplerrrrfdsz8preprocess_input_examples_arg_string..)rr2rGrFr'r])input_examples_strrr8Z example_listrrr$preprocess_input_examples_arg_stringIs rcCst}x|D]\}}t|ts:td||t|ft|dtr`|jj |j j |qt|dt r|jj |jj dd|Dqt|dtr|jj |jj |qt|dtr|jj |jj |qtdt|d|dfqW|S)z7Create a serialized tf.example from feature dictionary.z0feature value must be a list, but %s: "%s" is %srcSsg|]}|dqS)utf8)encode)rafrrrrfusz*_create_example_string..z;Type %s for value %s is not supported for tf.train.Feature.)rZExampler2rGrFr'r]floatfeaturesfeatureZ float_listrWrKrVZ bytes_listbytesintZ int64_listZSerializeToString)Z example_dictrZ feature_nameZ feature_listrrrris* rcCs\i}t|}t|}t|}x|D]\}\}} tjtj|dddd} | rt| tj rvt d|| f| ||<q| | kr| | ||<qt d|| fq&t| tj jjr| j} t| dkrt d|| | d ||<q&| ||<q&Wx4|D](\}} ||krt d || ||<qWx6|D]*\}} ||krJt d || ||<q*W|S) a Parses input arg strings and create inputs feed_dict. Parses '--inputs' string for inputs to be loaded from file, and parses '--input_exprs' string for inputs to be evaluated from python expression. '--input_examples' string for inputs to be created from tf.example feature dictionary list. Args: inputs_str: A string that specified where to load inputs. Each input is separated by semicolon. * For each input key: '=' or '=[]' * The optional 'variable_name' key will be set to None if not specified. * File specified by 'filename' will be loaded using numpy.load. Inputs can be loaded from only .npy, .npz or pickle files. * The "[variable_name]" key is optional depending on the input file type as descripted in more details below. When loading from a npy file, which always contains a numpy ndarray, the content will be directly assigned to the specified input tensor. If a variable_name is specified, it will be ignored and a warning will be issued. When loading from a npz zip file, user can specify which variable within the zip file to load for the input tensor inside the square brackets. If nothing is specified, this function will check that only one file is included in the zip and load it for the specified input tensor. When loading from a pickle file, if no variable_name is specified in the square brackets, whatever that is inside the pickle file will be passed to the specified input tensor, else SavedModel CLI will assume a dictionary is stored in the pickle file and the value corresponding to the variable_name will be used. input_exprs_str: A string that specifies python expressions for inputs. * In the format of: '='. * numpy module is available as np. input_examples_str: A string that specifies tf.Example with dictionary. * In the format of: '=<[{feature:value list}]>' Returns: A dictionary that maps input tensor keys to numpy ndarrays. Raises: RuntimeError: An error when a key is specified, but the input file contains multiple numpy ndarrays, none of which matches the given key. RuntimeError: An error when no key is specified, but the input file contains more than one numpy ndarrays. rb)modeT)Z allow_picklez?Input file %s contains a single ndarray. Name key "%s" ignored.z7Input file %s does not contain variable with name "%s".r0zYInput file %s contains more than one ndarrays. Please specify the name of ndarray to use.rzrinput_key %s has been specified with both --inputs and --input_exprs options. Value in --input_exprs will be used.z\input_key %s has been specified in multiple options. Value in --input_examples will be used.)rrrr2rrr FileIOrGZndarrayrxwarnrlibZnpyioZNpzFilefileslen)rrrtensor_key_feed_dictr( input_exprsinput_examplesZinput_tensor_keyfilenameZ variable_namedataZvariable_name_listZpy_expr_evaluatedrrrr!load_inputs_from_input_arg_stringsL0        rcCsX|jrt|jnB|jdkr(t|jn,|jdkrBt|j|jnt|j|j|jdS)z_Function triggered by show command. Args: args: A namespace parsed from command line. N)allrkdirrr r&r%r9)rQrrrshows    rc Cs^|js|js|jstdt|j|j|j}t|j|j|j||j |j |j |j |j |jd dS)zFunction triggered by run command. Args: args: A namespace parsed from command line. Raises: AttributeError: An error when neither --inputs nor --input_exprs is passed to run command. zLAt least one of --inputs, --input_exprs or --input_examples must be required)rrrurN)r(rrAttributeErrorrrrrr&r overwriterrrur)rQrrrrr{s r{cCs|jr*|jr*tt|j|jt|jnl|jrHtt|j|jtnNt|j}|jr|x:|j D]}t|t|jqbWnx|j D]}t|tqWdS)z_Function triggered by scan command. Args: args: A namespace parsed from command line. N) rrorsrr1rrp _OP_DENYLISTrBrC)rQZ saved_modelr)rrrscans    rc Csddlm}|js|jj|j|j|jd}y0|jf|j |j dd| }| Wn0tk r}ztd|Wdd}~XYnX|j|jdn0|jddd |j|j|jd |j |j d|jd dS) zmFunction triggered by 'convert tensorrt' command. Args: args: A namespace parsed from command line. r) trt_convert)max_workspace_size_bytesprecision_modeminimum_segment_sizerj)input_saved_model_dirinput_saved_model_tagsz+{}. Try passing "--convert_tf1_model=True".N)output_saved_model_dirr0T)Zmax_batch_sizerrrZ is_dynamic_oprrr)Z#tensorflow.python.compiler.tensorrtrZconvert_tf1_modelZDEFAULT_TRT_CONVERSION_PARAMS_replacerrrZTrtGraphConverterV2rrrn_asdictconvert Exceptionrformatr output_dirZcreate_inference_graph)rQZtrtparams convertererrrconvert_with_tensorrt6s6     rcCsj|jptj|jd}|js"g}n |jdkr6d}n |jd}tj |t |j|j |j ||jddS)zgFunction triggered by freeze_model command. Args: args: A namespace parsed from command line. zvariables/variablesrNrj)checkpoint_pathr)r$variables_to_feed output_prefix)rr|r}rrrlowerrnr freeze_modelrr1rr$r)rQrrrrrr]s  rc Cs|jptj|jd}|js"g}n |jdkr6d}n |jd}tj |t |j|j |j ||j|j|j|j|jdkd dS)zjFunction triggered by aot_compile_cpu command. Args: args: A namespace parsed from command line. zvariables/variablesrNrj)rfalse0) rr)r$rr target_triple target_cpu cpp_classmultithreading)rr|r}rrrrrnrZaot_compile_cpu_meta_graph_defrr1rr$rrrrr)rQrrrrraot_compile_cpuvs$  rcCsnd}|jd|tjd}|jdtddd|jdd d d |jd td dd|jdtd ddd|jtdd S)zAdd parser for `show`.aUsage examples: To show all tag-sets in a SavedModel: $saved_model_cli show --dir /tmp/saved_model To show all available SignatureDef keys in a MetaGraphDef specified by its tag-set: $saved_model_cli show --dir /tmp/saved_model --tag_set serve For a MetaGraphDef with multiple tags in the tag-set, all tags must be passed in, separated by ';': $saved_model_cli show --dir /tmp/saved_model --tag_set serve,gpu To show all inputs and outputs TensorInfo for a specific SignatureDef specified by the SignatureDef key in a MetaGraph. $saved_model_cli show --dir /tmp/saved_model --tag_set serve --signature_def serving_default To show all available information in the SavedModel: $saved_model_cli show --dir /tmp/saved_model --allr) descriptionformatter_classz--dirTz.directory containing the SavedModel to inspect)r]requiredhelpz--all store_truez7if set, will output all information in given SavedModel)actionrz --tag_setNz8tag-set of graph in SavedModel to show, separated by ',')r]defaultrz--signature_defSIGNATURE_DEF_KEYz9key of SignatureDef to display input(s) and output(s) for)r]rmetavarr)func) add_parserargparseRawTextHelpFormatter add_argumentrV set_defaultsr) subparsersZshow_msgZ parser_showrrradd_show_subparsers4 rcCsd}|jd|tjd}|jdtddd|jdtdd d|jd tdd d d d}|jdtd|dd}|jdtd|dd}|jdtd|d|jdtddd|jdddd|jdddd|jdtdd d|jd!ddd"d#|jd$ddd%d#|jtd&dS)'zAdd parser for `run`.aUsage example: To run input tensors from files through a MetaGraphDef and save the output tensors to files: $saved_model_cli show --dir /tmp/saved_model --tag_set serve \ --signature_def serving_default \ --inputs input1_key=/tmp/124.npz[x],input2_key=/tmp/123.npy \ --input_exprs 'input3_key=np.ones(2)' \ --input_examples 'input4_key=[{"id":[26],"weights":[0.5, 0.5]}]' \ --outdir=/out For more information about input file format, please see: https://www.tensorflow.org/guide/saved_model_cli r{)rrz--dirTz.directory containing the SavedModel to execute)r]rrz --tag_setz8tag-set of graph in SavedModel to load, separated by ','z--signature_defrzkey of SignatureDef to run)r]rrrzLoading inputs from files, in the format of '=, or '=[]', separated by ';'. The file format can only be from .npy, .npz or pickle.z--inputsr:)r]rraGSpecifying inputs by python expressions, in the format of "=''", separated by ';'. numpy module is available as 'np'. Please note that the expression will be evaluated as-is, and is susceptible to code injection. When this is set, the value will override duplicate input keys from --inputs option.z --input_exprszSpecifying tf.Example inputs as list of dictionaries. For example: =[{feature0:value_list,feature1:value_list}]. Use ";" to separate input keys. Will override duplicate input keys from --inputs and --input_exprs option.z--input_examplesz--outdirNz?if specified, output tensor(s) will be saved to given directoryz --overwriterz=if set, output file will be overwritten if it already exists.)rrz --tf_debugzif set, will use TensorFlow Debugger (tfdbg) to watch the intermediate Tensors and runtime GraphDefs while running the SavedModel.z--workerzdif specified, a Session will be run on the worker. Valid worker specification is a bns or gRPC path.z --init_tpuz~if specified, tpu.initialize_system will be called on the Session. This option should be only used if the worker is a TPU job.)rrrz --use_tfrtz@if specified, TFRT session will be used, instead of TF1 session.)r)rrrrrVrr{)rZrun_msgZ parser_runmsgrrradd_run_subparsersl rcCsXd}|jd|tjd}|jdtddd|jdtd d |jd td d |jtd dS)zAdd parser for `scan`.aUsage example: To scan for default denylisted ops in SavedModel: $saved_model_cli scan --dir /tmp/saved_model To scan for a specific set of ops in SavedModel: $saved_model_cli scan --dir /tmp/saved_model --op_denylist OpName,OpName,OpName To scan a specific MetaGraph, pass in --tag_set r)rrz--dirTz.directory containing the SavedModel to execute)r]rrz --tag_setz8tag-set of graph in SavedModel to scan, separated by ',')r]rz --op_denylistzlist of ops to detect and report, separated by ','. If not set, default is WriteFile,ReadFile,PrintV2. For empty list, pass in '')rN)rrrrrVrr)rZscan_msgZ parser_scanrrradd_scan_subparsers& rcCsd}|jd|tjd}|jdtddd|jdtdd d|jd tdd d|jd d dd}|jddtjd}|jdtddd|jdtddd|jdtddd|jdtddd|jt dd S)!zAdd parser for `convert`.zUsage example: To convert the SavedModel to one that have TensorRT ops: $saved_model_cli convert \ --dir /tmp/saved_model \ --tag_set serve \ --output_dir /tmp/saved_model_trt \ tensorrt r)rrz--dirTz.directory containing the SavedModel to convert)r]rrz --output_dirz-output directory for the converted SavedModelz --tag_setz;tag-set of graph in SavedModel to convert, separated by ','zconversion methodszvalid conversion methodsz)the conversion to run with the SavedModel)titlerrZtensorrtz;Convert the SavedModel with Tensorflow-TensorRT integrationz--max_workspace_size_bytesi zOthe maximum GPU temporary memory which the TRT engine can use at execution time)r]rrz--precision_modeZFP32zone of FP32, FP16 and INT8z--minimum_segment_sizerzTthe minimum number of nodes required for a subgraph to be replacedin a TensorRT nodez--convert_tf1_modelFz%support TRT conversion for TF1 models)rN) rrrrrVadd_subparsersrrrr)rZ convert_msgZparser_convertZconvert_subparsersZparser_convert_with_tensorrtrrradd_convert_subparser5sb  rcCsr|jdtddd|jdtddd|jdtddd|jd ttjd d |jd td dd |jdtddd d S)z7Parse arguments shared by freeze model and aot_compile.z--dirTz.directory containing the SavedModel to convert)r]rrz--output_prefixzQoutput directory + filename prefix for the resulting header(s) and object file(s)z --tag_setz;tag-set of graph in SavedModel to convert, separated by ','z--signature_def_keyzEsignature_def key to use. default: DEFAULT_SERVING_SIGNATURE_DEF_KEY)r]rrz--checkpoint_pathNz@Custom checkpoint to use (default: use the SavedModel variables)z--variables_to_feedr:aThe names of variables that will be fed into the network. Options are: empty (default; all variables are frozen, none may be fed), 'all' (all variables may be fed), or a comma-delimited list of names of variables that may be fed. In the last case, the non-fed variables will be frozen in the graph.**NOTE** Any variables passed to `variables_to_feed` *must be set by the user*. These variables will NOT be frozen and their values will be uninitialized in the compiled object (this applies to all input arguments from the signature as well).)rrVrZ!DEFAULT_SERVING_SIGNATURE_DEF_KEY)parser_compilerrr_parse_common_freeze_and_aotrs<rcCs@dddddddg}|jd|tjd }t||jtd d S) zAdd parser for `freeze_model`.rYzUsage example:z4To freeze a SavedModel in preparation for tfcompile:z$saved_model_cli freeze_model \z --dir /tmp/saved_model \z --tag_set serve \z+ --output_prefix /tmp/saved_model_xla_aotr)rr)rN)rrrrrrr)r compile_msgrrrradd_freeze_model_subparsers rcCsdddddddddd d d d tdd dddddddg}|jd|tjd}t||jdtddd|jdtddd|jdtddd|jdtd d!d|j t d"d#S)$z!Add parser for `aot_compile_cpu`.rYzUsage example:z4To compile a SavedModel signature via (CPU) XLA AOT:z"$saved_model_cli aot_compile_cpu \z --dir /tmp/saved_model \z --tag_set serve \z( --output_dir /tmp/saved_model_xla_aotr:zFNote: Additional XLA compilation options are available by setting the zEXLA_FLAGS environment variable. See the XLA debug options flags for zall the options: z {}z5For example, to disable XLA fast math when compiling:zQXLA_FLAGS="--xla_cpu_enable_fast_math=false" $saved_model_cli aot_compile_cpu ...zSome possibly useful flags:z" --xla_cpu_enable_fast_math=falsez6 --xla_force_host_platform_device_count=z: (useful in conjunction with disabling multi threading)r)rrz--target_triplezx86_64-pc-linuxzTarget triple for LLVM during AOT compilation. Examples: x86_64-none-darwin, x86_64-apple-ios, arm64-none-ios, armv7-none-android. More examples are available in tfcompile.bzl in the tensorflow codebase.)r]rrz --target_cpuzTarget cpu name for LLVM during AOT compilation. Examples: x86_64, skylake, haswell, westmere, (unknown). For a complete list of options, run (for x86 targets): `llc -march=x86 -mcpu=help`z --cpp_classTaThe name of the generated C++ class, wrapping the generated function. The syntax of this flag is [[::],...]. This mirrors the C++ syntax for referring to a class, where multiple namespaces may precede the class name, separated by double-colons. The class will be generated in the given namespace(s), or if no namespaces are given, within the global namespace.)r]rrz--multithreadingFalsezEnable multithreading in the compiled computation. Note that if using this option, the resulting object files may have external dependencies on multithreading libraries like nsync.)rN) rr_XLA_DEBUG_OPTIONS_URLrrrrrrVrr)rrrrrradd_aot_compile_cpu_subparsers^ rcCsbtjdd}|jddddd|jdd d d }t|t|t|t|t|t ||S) zzCreates a parser that parse the command line arguments. Returns: A namespace parsed from command line arguments. z6saved_model_cli: Command-line interface for SavedModel)rz-vz --versionversionz0.1.0)rrcommandszvalid commandszadditional help)rrr) rArgumentParserrrrrrrrr)parserrrrr create_parsers rcCs<ttjt}|}t|ds.|d||dS)Nrztoo few arguments)rx set_verbosityINFOr parse_argshasattrerrorr)rrQrrrmains    r__main__)r)rTr)r)NFFF)T)V__doc__rrr|rsysZabslrnumpyrZtensorflow.core.examplerZtensorflow.core.frameworkrZtensorflow.core.protobufrZtensorflow.python.clientrZ tensorflow.python.debug.wrappersrZtensorflow.python.eagerrr rHZtensorflow.python.frameworkr rqr rDr Ztensorflow.python.lib.ior Ztensorflow.python.platformrrxZtensorflow.python.saved_modelrrrrZtensorflow.python.toolsrrZtensorflow.python.tpurZtensorflow.python.util.compatrrrmrr r%r*r,r9rSrMr3rkr1r!rprsrrrrrrrr{rrrrrrrrrrrrrr^exitrrrrs                      '0 +  h/ & g'.U=-F!