B ӻdB|@sdZddlmZddlmZddlmZddlmZddlm Z ddlm Z ddlm Z dd lm Z dd l mZdd l mZdd lmZdd lmZddlmZddlmZddlmZddlmZddlmZddlmZddlmZddl m!Z!e j"Z"e j#Z#e j$Z$ej%Z%ej&Z&ej'Z'ej(Z(ej)Z)ddZ*ddZ+ddZ,ddZ-de+fdd Z.d!d"Z/d#d$Z0de+fd%d&Z1e!d'd7d(d)Z2d*d+Z3d,d-Z4e!d.gd/d0d1Z5e!d2gd/d8d5d6Z6dS)9z7Code for model cloning, plus model-related API entries.)ops)backend)metrics) optimizer_v1) functional) sequential)training) training_v1) AddMetric)Layer)Input) InputLayer) model_config)save) generic_utils) version_utils)CustomObjectScope) tf_logging)nest) keras_exportcCs|S)N)layerrrP/var/www/html/venv/lib/python3.7/site-packages/tensorflow/python/keras/models.py share_weights4srcCs|j|S)N) __class__ from_config get_config)rrrr _clone_layer8srcsJdd|D}|jfddddd|D|}|j|t|ddS) z>Inserts ancillary layers into the model with the proper order.cSsg|]}t|tr|qSr) isinstancer ).0rrrr @sz,_insert_ancillary_layers..cs |jS)N)indexZ metric_name)r) metrics_namesrrBz*_insert_ancillary_layers..)keycSsg|]}t|ts|qSr)rr )rrrrrr Ds)Zrelevant_nodesN)sortZ_insert_layerslist)modelancillary_layersr" new_nodesZ metric_layersr)r"r_insert_ancillary_layers<s  r+cs&t}t|}|jddx|D]}||}x|D]}|j} | |kr`|| } | || <| } n|| } t| trtq8tfddt |j Dr8t fdd|j } t fdd|j } | | | } t | d}|| j|jjx.tt |jt | D]\}}||<qWq8Wq&W|S) aUses the layers in `layer_map` to make new nodes based on `nodes_by_depth`. Args: nodes_by_depth: Provides structure information to create new nodes. layer_fn: Function to clone layers. layer_map: Map from layers in `model` to new layers. tensor_map: Map from tensors in `model` to newly compute tensors. Returns: A set of new nodes. `layer_map` and `tensor_map` are updated. T)reversec3s|]}|kVqdS)Nr)rtensor) tensor_maprr nsz"_make_new_nodes..cs ||S)N)get)t)r.rrr#pr$z!_make_new_nodes..cs ||S)N)r0)r1)r.rrr#rr$r)setr'keysr&outbound_layerrr allrflatten input_tensorsZ map_structure call_argsZ call_kwargsaddZ_inbound_nodes_keras_historyZ node_indexzipoutput_tensors)Znodes_by_depthlayer_fn layer_mapr.r*Z depth_keysdepthnodesnoderZ new_layerargskwargsr<Zfirst_output_tensorxyr)r.r_make_new_nodesIs8       rFNcs6ttstdttr(tdjs6tdi}|dk rt|}xXt|D]L\}}j|}t |s|j }t |d|d}|j j}|||<qV|||<qVWt|stdt||\} } tj| | d\}} } j} t|| j d fd d | D} | r2td d | D}t| | |S) a^Clone a functional `Model` instance. Model cloning is similar to calling a model on new inputs, except that it creates new layers (and thus new weights) instead of sharing the weights of the existing layers. Input layers are always cloned. Args: model: Instance of `Model`. input_tensors: optional list of input tensors to build the model upon. If not provided, placeholders will be created. layer_fn: callable to be applied on non-input layers in the model. By default it clones the layer. Another example is to preserve the layer to share the weights. This is required when we create a per-replica copy of the model with distribution strategy; we want the weights to be shared but still feed inputs separately so we create new input layers. Returns: An instance of `Model` reproducing the behavior of the original model, on top of new inputs tensors, using newly instantiated weights. Raises: ValueError: in case of invalid `model` argument value or `layer_fn` argument value. z8Expected `model` argument to be a `Model` instance, got zcExpected `model` argument to be a functional `Model` instance, got a `Sequential` instance instead:z`Expected `model` argument to be a functional `Model` instance, but got a subclass model instead.Ninput_wrapper_for_)r-namez.Expected `layer_fn` argument to be a callable.)created_layers)rHcsg|]}|jkr|qSr)layers)rr)r(rrr sz+_clone_functional_model..cSs*g|]"}t|r |jddn|jqS)N)rZ_should_skip_first_nodeZ inbound_nodes)rrrrrr s)rModel ValueError Sequential_is_graph_networkrr6 enumerate _input_layersris_keras_tensorrHr r:rcallable_clone_layers_and_model_configrZreconstruct_from_configr"valuesr+)r(r7r=Znew_input_layersi input_tensorZoriginal_input_layerrHZnewly_created_input_layerZ model_configsrIr<r"r)r*r)r(r_clone_functional_modelsF        rXcs,ifdd}tj|d}|fS)adClones all layers, and returns the model config without serializing layers. This function ensures that only the node graph is retrieved when getting the model config. The `layer_fn` used to clone layers might not rely on `layer.get_config()`, so some custom layers do not define `get_config`. Trying to retrieve the config results in errors. Args: model: A Functional model. input_layers: Dictionary mapping input layers in `model` to new input layers layer_fn: Function used to clone all non-input layers. Returns: Model config object, and a dictionary of newly created layers. csJ|kr||j<n.|jkr8tf||j<n||j<iS)N)rHrQr r)r)rI input_layersr=r(rr _copy_layers  z3_clone_layers_and_model_config.._copy_layer)Zserialize_layer_fn)rZget_network_config)r(rYr=rZconfigr)rIrYr=r(rrTs   rTcszg|js|fSdd|jD}|jddx0|D](}x"|j|D]}||jqHWq8Wfdd|DfS)aRemoves and returns any ancillary layers from `layers` based on `model`. Ancillary layers are part of the model topology but not used to compute the model outputs, e.g., layers from `add_loss` and `add_metric`. Args: model: A Keras Model. layer_map: A map to from layers in the `model` to those in `layers`. layers: A list of all layers. Returns: Two lists of layers: (1) `layers` with the ancillary layers removed, and (2) the ancillary layers. cSsg|]}|dkr|qS)rr)rr?rrrr sz,_remove_ancillary_layers..T)r,csg|]}|kr|qSrr)rl)r)rrr s)rO_nodes_by_depthr3r&appendr4)r(r>rJZdepthsr?rAr)r)r_remove_ancillary_layerss  r_cCst|tstd|t|s$tdg}i}xV|jdddD]D}t|trT|dk rTqvsz+_clone_sequential_model..)rrNrMrS_flatten_layersr rr^r_rHlenrZto_listtupler'rrRr:rr strr]itemsr;r<rPrFr+r")r(r7r=rJr>rZ cloned_layerr)Z cloned_modelrDZ origin_layerrWZ input_layerr.r?Z cloned_nodesr@Z cloned_noderAjZ output_tensorr*rrr_clone_sequential_models^            rizkeras.models.clone_modelc CsJt8|dkrt}t|tr.t|||dSt|||dSWdQRXdS)a Clone a Functional or Sequential `Model` instance. Model cloning is similar to calling a model on new inputs, except that it creates new layers (and thus new weights) instead of sharing the weights of the existing layers. Note that `clone_model` will not preserve the uniqueness of shared objects within the model (e.g. a single variable attached to two distinct layers will be restored as two separate variables). Args: model: Instance of `Model` (could be a Functional model or a Sequential model). input_tensors: optional list of input tensors or InputLayer objects to build the model upon. If not provided, new `Input` objects will be created. clone_function: Callable to be used to clone each layer in the target model (except `InputLayer` instances). It takes as argument the layer instance to be cloned, and returns the corresponding layer instance to be used in the model copy. If unspecified, this callable defaults to the following serialization/deserialization function: `lambda layer: layer.__class__.from_config(layer.get_config())`. By passing a custom callable, you can customize your copy of the model, e.g. by wrapping certain layers of interest (you might want to replace all `LSTM` instances with equivalent `Bidirectional(LSTM(...))` instances, for example). Returns: An instance of `Model` reproducing the behavior of the original model, on top of new inputs tensors, using newly instantiated weights. The cloned model may behave differently from the original model if a custom `clone_function` modifies the layer. Example: ```python # Create a test Sequential model. model = keras.Sequential([ keras.Input(shape=(728,)), keras.layers.Dense(32, activation='relu'), keras.layers.Dense(1, activation='sigmoid'), ]) # Create a copy of the test model (with freshly initialized weights). new_model = clone_model(model) ``` Note that subclassed models cannot be cloned, since their internal layer structure is not known. To achieve equivalent functionality as `clone_model` in the case of a subclassed model, simply make sure that the model class implements `get_config()` (and optionally `from_config()`), and call: ```python new_model = model.__class__.from_config(model.get_config()) ``` N)r7r=)rZDisableSharedObjectScoperrrNrirX)r(r7Zclone_functionrrr clone_models<   rjc Cs|jr tt|jtjtjt i}xt |D]}|dks2|dkrHq2yt ||}Wnt t tfk rrw2YnXt|tr|||<||jkstt|dr|jrt d|q2t|ttfr2|dkr2|r2tdd|Dr2t d|q2Wd d |D}t|jd d d }|j}d |_g|_xd|D]\}|}t|tjrX|jsXt d ||j|} ||}t||| |j| q*Wt|dr|jdkr|jrddddddddddddg } x| D]}t ||||<qW||_t |||_dS)aSubstitute for model cloning that works for subclassed models. Subclassed models cannot be cloned because their topology is not serializable. To "instantiate" an identical model in a new TF graph, we reuse the original model object, but we clear its state. After calling this function on a model instance, you can use the model instance as if it were a model clone (in particular you can use it in a new graph). This method clears the state of the input model. It is thus destructive. However the original state can be restored fully by calling `_in_place_subclassed_model_state_restoration`. Args: model: Instance of a Keras model created via subclassing. Raises: ValueError: In case the model uses a subclassed model as inner layer. Z submodules_self_tracked_trackablesrJzgWe do not support the use of nested layers in `model_to_estimator` at this time. Found nested layer: %s)rJZ_layersrZ_compile_metric_functionsZ_output_loss_metricscss|]}t|tVqdS)N)rr )rvalrrrr/sz3_in_place_subclassed_model_reset..zWe do not support the use of list-of-layers attributes in subclassed models used with `model_to_estimator` at this time. Found list model: %scSsi|]\}}||qSrr)rr%valuerrrrbsz4_in_place_subclassed_model_reset..F)r`razrWe do not support the use of nested subclassed models in `model_to_estimator` at this time. Found nested model: %s_original_attributes_cacheNinputsoutputsZ total_loss optimizerZtrain_functionZ test_functionZpredict_functionZ_training_endpointsZ_collected_trainable_weightsZ _feed_inputsZ_feed_input_namesZ_feed_input_shapes)!rOAssertionErrorrZ swap_classrrrLr r#executing_eagerly_outside_functionsdirgetattrAttributeErrorrM TypeErrorrr rJhasattrr'rer5rgrc_setattr_trackingrkrrsetattrr^rnbuilt_reset_build_compile_trackers) r(Zattributes_cacherHrmZlayers_to_namesZoriginal_layerssetattr_trackingrr[Z fresh_layerZattributes_to_cacherrr _in_place_subclassed_model_resetsr           r~cCs0d|_d|_d|_d|_ts&d|_d|_dS)aReset state trackers for model. Note that we do not actually zero out attributes such as optimizer, but instead rely on the expectation that all of the attrs will be over-written on calling build/compile/etc. This is somewhat fragile, insofar as we check elsewhere for the presence of these attributes as evidence of having been built/compiled/etc. Pending a better way to do this, we reset key attributes here to allow building and compiling. Args: model: the model that is being reset FN)r{rorpZ _is_compiledrrsZ_v1_compile_was_calledrq)r(rrrr|-sr|zEkeras.__internal__.models.in_place_subclassed_model_state_restoration)Zv1cCs|jr tt|drx|jdk rx|j}d|_g|_x8|jD]*\}}t|||t|t r<|j |q.)r7Nr)dtypezvThis model is a subclassed model. Please implement `get_config` and `from_config` to better support cloning the model.aJThis model is a subclassed model. Such a model cannot be cloned, but there is a workaround where the model is reset in-place. To use this, please set the argument `in_place_reset` to `True`. This will reset the attributes in the original model. To restore the attributes, call `in_place_subclassed_model_state_restoration(model)`.rKcSsg|]}|j|qSr)rrr)roptrrrr sz)clone_and_build_model..cSsg|]\}}|j|qSr)rr)rrZ opt_configrrrr srqtarget_tensorsrZweighted_metrics)$rqrMZ_get_compile_argsrrOrjrrNZ_build_input_shaperrsbuildZ _set_inputsr placeholderrorrrrNotImplementedErrorloggingwarningr~r'rerdrZ TFOptimizerZtrack_tf_optimizerdictr;Z iterationsmetrics_moduleZ clone_metricscompile) r(r7rZcustom_objectsZ compile_cloneZin_place_resetZoptimizer_iterationsZoptimizer_configZorig_optimizerclonerqrr)rrclone_and_build_modelhsv/              r)NN)NNNTFNN)7__doc__Ztensorflow.python.frameworkrZtensorflow.python.kerasrrrrZtensorflow.python.keras.enginerrrr Z)tensorflow.python.keras.engine.base_layerr r Z*tensorflow.python.keras.engine.input_layerr r Ztensorflow.python.keras.savingrrZtensorflow.python.keras.utilsrrZ+tensorflow.python.keras.utils.generic_utilsrZtensorflow.python.platformrrZtensorflow.python.utilrZ tensorflow.python.util.tf_exportrrLrNZ FunctionalZ save_modelZ load_modelZmodel_from_configZmodel_from_yamlZmodel_from_jsonrrr+rFrXrTr_rirjr~r|rrrrrrs^                     8X!i Hd!