B d@sdZddlZddlZddlZddlZddl m Z ddl m Z ddl mZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZdd lm Z!ddl"m#Z#ddl$m%Z&e&dgde!'ddGdddZ(GdddZ)Gddde)Z*Gddde)Z+GdddZ,ddZ-dd Z.d!d"Z/d#d$Z0d%d&Z1d'd(Z2d)d*Z3dAd,d-Z4d.d/Z5d0d1Z6d2d3Z7d4d5fd6d7Z8dBd8d9Z9d:d;Z:e&dd?Z;dd?d@d9d;gZ.assert_dictlist_has_keysparent_ophint_input_indexfirst_child_ophint_input_indexparent_output_indexchild_output_indexchild_input_indexN)rHrIrG)rr#rXr$r$r%rJ[s     z)OpHint._validate_children_inputs_mappingscCs0t|}|j|tj|jjjdjddS)Nvalue)tensor) _opsZconvert_to_tensorr1r2r4r5node_defattrr_)rZdest_opr,r^Z tensor_valuer$r$r%_setattrxs  zOpHint._setattrcOs|jj||S)aOAdd a wrapped input argument to the hint. Args: *args: The input tensor. **kwargs: "name" label "tag" a tag to group multiple arguments that will be aggregated. I.e. a string like 'cool_input'. Basically multiple inputs can be added to the same hint for parallel operations that will eventually be combined. An example would be static_rnn which creates multiple copies of state or inputs. "aggregate" aggregation strategy that is valid only for tag non None. Acceptable values are OpHint.AGGREGATE_FIRST, OpHint.AGGREGATE_LAST, and OpHint.AGGREGATE_STACK. "index_override" The global index to use. This corresponds to the argument order in the final stub that will be generated. Returns: The wrapped input tensor. )rOr()rargsrRr$r$r% add_inputszOpHint.add_inputcOs|jj||S)aRAdd a wrapped output argument to the hint. Args: *args: The output tensor. **kwargs: "name" label "tag" a tag to group multiple arguments that will be aggregated. I.e. a string like 'cool_input'. Basically multiple inputs can be added to the same hint for parallel operations that will eventually be combined. An example would be static_rnn which creates multiple copies of state or inputs. "aggregate" aggregation strategy that is valid only for tag non None. Acceptable values are OpHint.AGGREGATE_FIRST, OpHint.AGGREGATE_LAST, and OpHint.AGGREGATE_STACK. "index_override" The global index to use. This corresponds to the argument order in the final stub that will be generated. Returns: The wrapped output tensor. )rQr()rrdrRr$r$r% add_outputszOpHint.add_outputcs:d|kr$fddt||dDSfdd|DSdS)aMAdd a sequence of inputs to the function invocation. Args: *args: List of inputs to be converted (should be Tf.Tensor). **kwargs: This allows 'names' which should be a list of names. Returns: Wrapped inputs (identity standins that have additional metadata). These are also are also tf.Tensor's. namescs g|]\}}jj||dqS))r,)rOr().0r?r,)rr$r% sz%OpHint.add_inputs..csg|]}j|qSr$)rOr()rhr?)rr$r%risN)zip)rrdrRr$)rr% add_inputss  zOpHint.add_inputscs:d|kr$fddt||dDSfdd|DSdS)aAdd a sequence of outputs to the function invocation. Args: *args: List of outputs to be converted (should be tf.Tensor). **kwargs: See Returns: Wrapped outputs (identity standins that have additional metadata). These are also tf.Tensor's. rgcs g|]\}}jj||dqS))r,)rQr()rhr?r,)rr$r%risz&OpHint.add_outputs..csg|]}j|qSr$)rQr()rhr?)rr$r%risN)rj)rrdrRr$)rr% add_outputss  zOpHint.add_outputs)rN)rBrCrDrEr3r8rNrPr=r>TFLITE_INPUT_INDICESr9r:AGGREGATE_STACKAGGREGATE_FIRSTAGGREGATE_LASTrFr&rJrcrerfrkrlr$r$r$r%rXs. %rc@s eZdZdZddZddZdS) _LiteOperandaAbstract operand for a tflite hint function._dynamic_rnn_loop. This is a base class that handles representing arguments to an OpHint. It also is able to serialize operands to the stubbed graph_def. Child classes are responsible for being able to store information about the hint identity operators. They are also responsible for knowing how to serialize to output graphdefs. Typically this will be implemented by holding one or more identity nodes that were previously discovered as hints. cCs~tddS)a6This adds the node(s) to out_graphdef and returns the input node name. Args: out_graphdef: A graphdef that is ready to have this input added. Returns: The output that the stub should use as an input for this operand. Raises: RuntimeError: if the method is not implemented. zUnimplemented abstract method.N) RuntimeError)r out_graphdefr$r$r%#aggregate_and_return_name_for_inputs z0_LiteOperand.aggregate_and_return_name_for_inputcCs~~~tddS)aAdd node(s) to graph representing output operands and returns type. Args: fused_op_name: name of the fused op stub name. output_index: Output index that we are currently processing from stub. out_graphdef: The destination graphdef we are currently building up. Returns: The datatype of this identity. Raises: RuntimeError: if the method is not implemented. zUnimplemented abstract method.N)rr)r fused_op_name output_indexrsr$r$r%$aggregate_and_return_name_for_outputsz1_LiteOperand.aggregate_and_return_name_for_outputN)rBrCrDrErtrwr$r$r$r%rqs rqc@s8eZdZdZddZddZddZdd Zd d Zd S) _LiteSingleOperandz:A simple operand that is non-aggregated (i.e. most hints).cCs t|||_t|j|_dS)N)rqr&node_tensor_name_baser,)rryr$r$r%r&s z_LiteSingleOperand.__init__cCs|jgS)N)r,)rr$r$r%flatten sz_LiteSingleOperand.flattencCs|jS)N)r,)rrsr$r$r%rt sz6_LiteSingleOperand.aggregate_and_return_name_for_inputcCsFt|j}|jdd=|jt|||j|g|jjdjS)Ntype) _copydeepcopyryinputappend_tensorflow_output_nameextendrbr.)rruindexrs output_noder$r$r%rws   z7_LiteSingleOperand.aggregate_and_return_name_for_outputcCs t|jS)N)strr,)rr$r$r%__str__sz_LiteSingleOperand.__str__N) rBrCrDrEr&r{rtrwrr$r$r$r%rxs rxc@sHeZdZdZddZddZddZdd Zd d Zd d Z ddZ dS)_LiteAggregateOperandzAn operand for a tflite hint function that is aggregated from many. For example, an LSTM is a grid of operators that are all related. Inputs going into them may need to be fused, so they should all be tracked as related arguments. cCs&t|||_i|_i|_d|_dS)N)rqr& aggregationrgnodes flattened)rrr$r$r%r&#s  z_LiteAggregateOperand.__init__cCst|j|j|<||j|<dS)N)rzr,rgr)rsortryr$r$r%r(*sz_LiteAggregateOperand.addcCs|jsdgt|j|_x |jD]\}}||j|<q$Wx|jD]}|dkrBtdqBW|jtjkrx|jdd|_n:|jtjkr|jdd|_n|jtj krnt d|j|jS)zAReturn a list of all the node protos in aggregation sorted order.NzAggregate was missing argument.rz%Invalid aggregation type %r specified) rlenritemsrrrrrorprnr')ridxrynr$r$r% flatten_nodes.s       z#_LiteAggregateOperand.flatten_nodescCsdd|DS)z=Return a list of all node names in aggregation sorted sorter.cSsg|]}t|jqSr$)rzr,)rhxr$r$r%riDsz1_LiteAggregateOperand.flatten..)r)rr$r$r%r{Bsz_LiteAggregateOperand.flattencCs|}|jtjks |jtjkr0t|dks0tt|dkrV|jtjkrVt|dj St }d|_ d|dj |_ t||j d_|dj dj|j d_x|D]}|jt|j qW|j|g|j SdS)aThis adds the nodes to out_graphdef and returns an aggregated output. In particular, if you have 4 inputs to a hint stub, this will be the node that you can use as an output. I.e. you have 4 timesteps from a static rnn, then a fused UnidirectionalLSTM will expect 1 input with all 4 time steps. So here we make a pack and return the output name of that pack. Args: out_graphdef: A graphdef that is ready to have this input added. Returns: The name of a pack that aggregates this node. rrPackzOpHintStack-%sNTN)rrrrorprrGrnrzr, _node_def_pb2NodeDefr1rbr.r|rrryr)rrsrnew_nodediscreter$r$r%rtFs   z9_LiteAggregateOperand.aggregate_and_return_name_for_inputc Cs"|}|jtjks |jtjkr0t|dks0tt|dkrb|jtjkrbt|d}| |||St }d|_ d|dj |_ t||jd_|djdj}||jd_|jt|||j|gxLt|D]@\}} t| } | jdd=| jt|j ||j| gqW|SdS)aYThis adds to `out_graphdef` all the unaggregated outputs. I.e. we are outputting from a fused stub, but we need to make it compatible with the unfused original graph so we insert an unpack. Ideally in a later stage the unpack -> pack sequences will be removed. Args: fused_op_name: The name of the stub we are in the process of fusing. output_index: The output output_index this object represents. out_graphdef: The graphdef we are in the process of buildings Returns: The type of the aggregated output (so we can finish building the stub op). rrUnpackzOpHintUnstack-%snumrN)rrrrorprrGrnrxrwrrr1r,rbr.r|rrrryr enumerater}r~) rrurvrsrZtemp_opZ stack_node output_typerrrr$r$r%rwfs.        z:_LiteAggregateOperand.aggregate_and_return_name_for_outputcCs6d|j}x&|jD]\}}|d||f7}qW|S)Nz AGGREGATE %s z %d: %s )rrg iteritems)rr-rvalr$r$r%rs z_LiteAggregateOperand.__str__N) rBrCrDrEr&r(rr{rtrwrr$r$r$r%rs ,rc@s(eZdZdZddZddZddZdS) _LiteFuncCallaRepresent a TensorFlow Lite custom function. This is uses to accumulate found hints in the graphdef into a single conceptual unit. Attributes: inputs: inputs to the op (hash from index # to argument) outputs: outputs to the op (hash from index # to argument) function_name: the tflite custom op name to use uuid: a unique call id for this particular call (i.e. multiple function calls would have the same function_name but different uuids. params: A param name to key value for op constant data. I.e. for axis on a reduction, strides on a convolution, etc. level: Level of the OpHint. children_inputs_mappings: If the Ophint has children, children inputs mappings indicate how their inputs & outputs are mapped. cCs.i|_i|_d|_d|_i|_d|_i|_dS)Nr)inputsoutputsr rAparamsr"r#)rr$r$r%r&sz_LiteFuncCall.__init__cCsdd}||j||jfS)zReturn a list of inputs and outputs in a flattened format. Returns: Tuple of (inputs, outputs). where input and output i a list of names. cSs(g}x|D]}||qW|S)N)valuesrr{)Zinput_or_output_dictZflattened_itemsitemr$r$r%_flattensz<_LiteFuncCall.flattened_inputs_and_outputs.._flatten)rr)rrr$r$r%flattened_inputs_and_outputssz*_LiteFuncCall.flattened_inputs_and_outputscCs<dd}d||j}d||j}d|j|j|j||fS)NcSs2d}x(|D]\}}|d|t|7}qW|S)Nz %d: )rr)rr-rrr$r$r% format_argssz*_LiteFuncCall.__str__..format_argsz Inputs z Outputs zAtflite function %s call %s level %d inputs: %s outputs: %s)rrr rAr")rrZ inputs_strZ outputs_strr$r$r%rs z_LiteFuncCall.__str__N)rBrCrDrEr&rrr$r$r$r%rs rc Csftt}xT|D]J}|j}tj|ks|tjjs6q|tjj}||}||_|tjj|_ |tj j |_ tj |kr|tj j nd}|dkrd}d}tj|krt|tjj}tj|krtt|tjj|_dd}tj|kr||j|tjj |||tj|kr*||j|tjj |||x2|D]*} | dr0|| j|j| dd<q0WqW|S)zLook at the all the input nodes and return a list of LiteFuncCall objs. Args: nodes: A TensorFlow graph_def to look for LiteFuncCalls. Returns: a list of `LifeFuncCall` objects in the form NrcSs>|dkrt|||<n$||kr*t|||<||||dS)z.Add a given index into the function structure.N)rxrr()stuffrroperandrr$r$r% put_operands  z-_find_all_hints_in_nodes..put_operandZ _tflite_attr_z_tflite_attr_,r) _collections defaultdictrrbrr8r-rAr3r r9r.r"r=r>r6Zas_textr:r;loadsr#rNrrPr startswithr_rreplace) rZ func_callsryrbrAZcall_defrrrar$r$r%_find_all_hints_in_nodess>            "rcCstddt|DS)Ncss |]\}}t|j|fVqdS)N)rzr,)rhrryr$r$r% sz5_extract_topology_sequence_mapping..)rIr)rr$r$r%"_extract_topology_sequence_mappingsrcCsg}xX|jD]N}x8t|jD]*\}}|j||kr||j||j|<qW|t|q Wt|j}t|}g}xR|D]F} | \}} || d} x| D]} t | || } qW|| | fq~W|j ddddd|D} | |fS)aFind children hints and all nodes inside the while loop. Args: function_def: Function def of the while loop. nodes_mapping: While loop input_arg : real node name. Returns: Ordered children hints and all re-mapped nodes inside the while loop. rcSs|dS)Nrr$)tupr$r$r%7z4_find_children_hints_in_while_loop..)rWcSsg|] }|dqS)rr$)rhrr$r$r%ri8sz6_find_children_hints_in_while_loop..) rarrrr}r~rrrrminr) function_def nodes_mapping new_nodesryr._name_to_seq_numchildren_hintsZchildren_hints_qhint output_namesseqZ output_nameZordered_children_hintsr$r$r%"_find_children_hints_in_while_loops$      rcCs~t|\}}}|\}}t||}t||}t|}g} t} | j|j| j|jt} x |j D]} | j t | gt | j} | |krn| |krn| |krn| jdks| jdkrn| jdjj}| j}x|jjD]}|jj|kr|jj}t|t|ks ti}x$t|D]\}}||||j<qWt||\}}| dd|D| || j |qWqnW| | | fS)aFind all children hints. For a given OpHint, we find all children hints inside it, we also copy all the nodes inside function defs (if applicable) to the original graph_def, they are returned in a list as well. Args: call: Parent OpHint that contains children ophints. graph_def: Original graph def. Returns: Ordered children hints inside the parent ophint; new graph def that contains nodes inside function defs (if applicable); nodes inside function defs. WhileZStatelessWhilebodycSsg|] }|jqSr$)r,)rhrr$r$r%rijsz(_find_children_hints..)r rrr _graph_pb2GraphDeflibraryCopyFromversionsryrr}r~rzr,r1rbfuncrfunction signatureZ input_argrrGrrupdate)call graph_defname_to_input_namer input_namesrreachable_by_inputreachable_by_outputoutput_nodes_setroutfunction_def_nodesryrZ body_nameZinputs_outside_looprZfunction_inputsrr.Zfunction_inputZchildren_hints_in_looprr$r$r%_find_children_hints<s@      rcCs$|dr|ddS|ddS)a#Removes the device assignment code from a tensor. e.g. _tensor_name_base("foo:3") => "foo" Args: full_tensor_name: A tensor name that is annotated with a device placement (this is what tensor flow introspection gives). Returns: A name without any device assignment. ^rN:r)rsplit)Zfull_tensor_namer$r$r%rzqs  rzcCs|dkr |Sd||fS)Nrz%s:%dr$)Z tensor_namervr$r$r%rsrcsn|g}tx\|rh|}|||krD||krDtd||f||kr|fdd||D7}qWdS)a{Checks to make sure node only connects to predecessor graph through inputs. Args: n: Node to check reachable_by_input: Nodes that are reachable by all inputs of subgraph input_nodes_set: The set of nodes that are "inputs". name_to_input_name: Maps from name to the list of inputs. Raises: TypeError: If the given node uses items past inputs directly. z)Node %s uses input %s not in input_nodes.csg|]}|kr|qSr$r$)rh input_node)visitedr$r%risz*_check_subgraph_closed..N)rpopr( TypeError)rrZinput_nodes_setr next_to_visitZ current_noder$)rr%_check_subgraph_closeds   rTcs(|dkrt}t|\}}|\}}t||}t||} t|} g} t} xf|jD]\} t| j}|| kr||kr|| kr| |qV||kr||kr| |qV|sV| |qVWt }t t |fddd}x$|D]} |j t|| gqWt |j}|t |j}|t}t}dttj|_d|_|jdtjtj j!d|jdtjt"#d gtj d gd |j |gt$|d }xLt%|D]@}||kr|j|}|&|}|j'|n|j'|jqW|jt(j)j j* ||j+|_|j,|_|j |gg}t$|d }xNt%|D]B}||krv|j|}|-|j||}n |jd j*}||qLW||jd j j.dd<d|jd_/xR| D]J}d}x ||D]}|| krd}qW|r|j t||gqW|j0|j0|j1|j1|S)aGiven a graph_def, converts `call` into a stub and returns a new graph_def. Args: call: A single function call to be converted. graph_def: A graph_def to use as input (that has call obviously). function_def_nodes: Nodes inside the function def those are not connected to the graph. is_last_run: Whether it is the last run for a given pass (for OpHint has children). Returns: A new transformed graph-def that has call as a stub (single op). Note: after this process, the graph_def can no longer be loaded into the tensorflow runtime, so all future manipulations are done in graph_def level. Ncs|S)Nr$)r)rr$r%rrz1_convert_single_op_hint_to_stub..)rWZConstZdtype)r|r^rr)r_r|Z _output_typesFZ_output_quantizedT)2rr rrryrzr,r(rrrsortedlistrr}r~rrVrrrrrrKrLrMr1rbrr4r5_dtypesZfloat32Zas_datatype_enum _tensor_utilZmake_tensor_protomaxrangertrrrmr.r rArwr|brr)rrrZ is_last_runr name_to_noderrrrrZnodes_after_fuseZnodes_deleted_by_fuseryrrZreachable_by_input_sortedZsorted_input_indicesZsorted_output_indicesrZoptional_input_nodeZ max_indexZ cur_indexrZ input_nameZ output_dtypesZmax_output_indexoutputZ output_dtypeZ should_keepr$)rr%_convert_single_op_hint_to_stubs                       rcsdt|\}}}~d}t}|j|j|j|jx |jD]}t|j}| dsj|j dsjqB|g}t t } |d} d} x|rN|d}  | |d=|| } | j d}| j d}| j dks|s|r | j dkr | |O} |fdd || D7}n4|s$|r:| j d kr:| | j| |M} nd} P | jqW| rBt | d krB|d}xV|jD]L}|jkrqpx4||jD]&}t|}|kr|krd}qWqpW| s|rB| }||jdxn|jD]d}t|j}|krt|}fd d t|||jd d D|jd d <|j|gqW|dfSqBW|dfS)zRemoves a stack->unstack pattern from in_graph_def in a returned graph. Args: in_graph_def: Graph def to use as input. Returns: Simplified tuple (graph_def, changed_something) where changed_something is true if anything was done. TZ OpHintStackrFrZ OpHintUnstackZIdentitycsg|]}|kr|qSr$r$)rhr)rr$r%riOsz7_remove_one_redundant_stack_unstack..rrcs g|]\}}|krn|qSr$r$)rhstrippedZ non_stripped) end_input pack_noder$r%ritsN)r rrrrrryrzr,rr1rr(rrrr}r~rjr)Z in_graph_defrrrZdo_generic_pack_unpackrrZ node_namerZ unpack_nodesZmatches_patternZis_hint_created_stackZcurrent_node_nameryZis_op_hint_stackZis_op_hint_unstackZno_external_dependencyZother_nZ input_tensorZinput_opendrr$)rrrr%#_remove_one_redundant_stack_unstack#sv                 rcCs$|}~d}x|rt|\}}q W|S)NT)r)rcurrZ changed_stuffr$r$r%_remove_redundant_stack_unstack}s rcCs$|dkr |}t|}|dS|S)Nr)rVr)Zoriginal_indexrZ node_indicesr$r$r%_get_correct_mappings rcCsdS)Nr$)sessrr$r$r%rrrc CsPt|j}g}x"|D]}||j|jfqW|jdddx(tt|dddD]}||\}}qXW|}~xtt|dddD]}||\}}|dkr(t |||\} }} t| dkst ||j } xt | D]\} } | dkr@x| dD]<}t |d ||j}t |d | j}||j|| j|<qWnTxR| d D]F}t |d | j}t |d | | dj}| | dj|| j|<qJW| t| dkrxJ| dD]>}t |d||j}t |d | j}||j|| j|<qWqWxJt | D]$\} } t| || | t| dk}qWqt|||}||dqWt|}|S)a-Converts a graph_def to a new graph_def where all op hints are stubbed. Args: graph_def: A graph def that we should convert. write_callback: A function pointer that can be used to write intermediate steps of graph transformation (optional). Returns: A new stubbed graph_def. cSs|dS)Nrr$)rr$r$r%rrz3_convert_op_hints_to_stubs_helper..)rWrrrrSrYrZrUr]r\rTr[initial)rryrrr"rArrrrrGr#rrrrrr)rwrite_callbackhintsZhints_qrr.r"Z hint_uuidZcurr_graph_defrrr#jZ child_hintmappingZparent_input_indexr]Z input_indexrvr[r\r$r$r%!_convert_op_hints_to_stubs_helpersd      rcCsp|dk r|dk rtdg}|dk r2t|jj}n|dk rDt|j}x&|D]}|\}}||qNW|S)aFind all Ophints output nodes in the graph. This is used to get all the output nodes those are ophinted, it is important for operation like convert_variables_to_constants keep all ophints structure. Note: only one of session or graph_def should be used, not both. Why this can be useful? Some TensorFlow ops (e.g. bidirectional rnn), can generate multiple outputs for unfused subgraph. If not all output nodes are consumed, graph optimization can potentially drop the unused nodes and cause ophints in an invalid states (due to missing ophinted output nodes). So it's important for us to find all those hinted output nodes and make sure they're not discarded away. Args: session: A TensorFlow session that contains the graph to convert. graph_def: A graph def that we should convert. Returns: A list of OpHints output nodes. Raises: ValueError: If both session and graph_def are provided. Nz*Provide only one of session and graph_def.)r'rrryrrr)sessionrZhinted_outputs_nodesrrrZ output_nodesr$r$r%find_all_hinted_output_nodess  rcCs>|dkrtdd}x$|jD]}|j}tj|krd}PqW|S)NzMust provide the graph_def.FT)r'ryrbrrN)rZophint_convertedryrbr$r$r%is_ophint_converteds  rz+lite.experimental.convert_op_hints_to_stubscCsdS)Nr$)rcommentsr$r$r%rrcCsJ|dk r|dk rtd|dk r,t|j|S|dk r>t||StddS)aConverts a graphdef with LiteOp hints into stub operations. This is used to prepare for toco conversion of complex intrinsic usages. Note: only one of session or graph_def should be used, not both. Args: session: A TensorFlow session that contains the graph to convert. graph_def: A graph def that we should convert. write_callback: A function pointer that can be used to write intermediate steps of graph transformation (optional). Returns: A new graphdef with all ops contained in OpHints being replaced by a single op call with the right parameters. Raises: ValueError: If both session and graph_def are provided. Nz*Provide only one of session and graph_def.z+Must specify session or graph_def as input.)r'rr)rrrr$r$r%convert_op_hints_to_stubs s  rZconvert_op_hints_to_stubs_new)NT)NN)>rE collectionsrcopyr}jsonr;rArKZtensorflow.core.frameworkrr4rrrrZtensorflow.python.frameworkrrrr`rrZ+tensorflow.python.framework.graph_util_implrr Ztensorflow.python.opsr r/Ztensorflow.python.utilr r6r Z _deprecationZtensorflow.python.util.all_utilr Z tensorflow.python.util.tf_exportrZ _tf_export deprecatedrrqrxrrrrrrrzrrrrrrrrrrZ_allowed_symbolsrBr$r$r$r%Bsn              y/~<A!5 |Z  L #