B ӻd@sdZddlZddlmZddlmZddlmZddlmZddlmZddl m Z dd l m Z dd l mZdd l mZdd lmZdd lmZddlmZddlmZddlmZGdddeZGdddee ZedGdddeZdS)zConvolutional-recurrent layers.N) activations)backend) constraints) initializers) regularizers)Layer) InputSpec)DropoutRNNCellMixin)RNN) conv_utils) generic_utils)tf_utils) array_ops) keras_exportcsZeZdZdZdfdd ZejddZejddZd d Z dd d Z dddZ Z S) ConvRNN2Da]Base class for convolutional-recurrent layers. Args: cell: A RNN cell instance. A RNN cell is a class that has: - a `call(input_at_t, states_at_t)` method, returning `(output_at_t, states_at_t_plus_1)`. The call method of the cell can also take the optional argument `constants`, see section "Note on passing external constants" below. - a `state_size` attribute. This can be a single integer (single state) in which case it is the number of channels of the recurrent state (which should be the same as the number of channels of the cell output). This can also be a list/tuple of integers (one size per state). In this case, the first entry (`state_size[0]`) should be the same as the size of the cell output. return_sequences: Boolean. Whether to return the last output. in the output sequence, or the full sequence. return_state: Boolean. Whether to return the last state in addition to the output. go_backwards: Boolean (default False). If True, process the input sequence backwards and return the reversed sequence. stateful: Boolean (default False). If True, the last state for each sample at index i in a batch will be used as initial state for the sample of index i in the following batch. input_shape: Use this argument to specify the shape of the input when this layer is the first one in a model. Call arguments: inputs: A 5D tensor. mask: Binary tensor of shape `(samples, timesteps)` indicating whether a given timestep should be masked. training: Python boolean indicating whether the layer should behave in training mode or in inference mode. This argument is passed to the cell when calling it. This is for use with cells that use dropout. initial_state: List of initial state tensors to be passed to the first call of the cell. constants: List of constant tensors to be passed to the cell at each timestep. Input shape: 5D tensor with shape: `(samples, timesteps, channels, rows, cols)` if data_format='channels_first' or 5D tensor with shape: `(samples, timesteps, rows, cols, channels)` if data_format='channels_last'. Output shape: - If `return_state`: a list of tensors. The first tensor is the output. The remaining tensors are the last states, each 4D tensor with shape: `(samples, filters, new_rows, new_cols)` if data_format='channels_first' or 4D tensor with shape: `(samples, new_rows, new_cols, filters)` if data_format='channels_last'. `rows` and `cols` values might have changed due to padding. - If `return_sequences`: 5D tensor with shape: `(samples, timesteps, filters, new_rows, new_cols)` if data_format='channels_first' or 5D tensor with shape: `(samples, timesteps, new_rows, new_cols, filters)` if data_format='channels_last'. - Else, 4D tensor with shape: `(samples, filters, new_rows, new_cols)` if data_format='channels_first' or 4D tensor with shape: `(samples, new_rows, new_cols, filters)` if data_format='channels_last'. Masking: This layer supports masking for input data with a variable number of timesteps. Note on using statefulness in RNNs: You can set RNN layers to be 'stateful', which means that the states computed for the samples in one batch will be reused as initial states for the samples in the next batch. This assumes a one-to-one mapping between samples in different successive batches. To enable statefulness: - Specify `stateful=True` in the layer constructor. - Specify a fixed batch size for your model, by passing - If sequential model: `batch_input_shape=(...)` to the first layer in your model. - If functional model with 1 or more Input layers: `batch_shape=(...)` to all the first layers in your model. This is the expected shape of your inputs *including the batch size*. It should be a tuple of integers, e.g. `(32, 10, 100, 100, 32)`. Note that the number of rows and columns should be specified too. - Specify `shuffle=False` when calling fit(). To reset the states of your model, call `.reset_states()` on either a specific layer, or on your entire model. Note on specifying the initial state of RNNs: You can specify the initial state of RNN layers symbolically by calling them with the keyword argument `initial_state`. The value of `initial_state` should be a tensor or list of tensors representing the initial state of the RNN layer. You can specify the initial state of RNN layers numerically by calling `reset_states` with the keyword argument `states`. The value of `states` should be a numpy array or list of numpy arrays representing the initial state of the RNN layer. Note on passing external constants to RNNs: You can pass "external" constants to the cell using the `constants` keyword argument of `RNN.__call__` (as well as `RNN.call`) method. This requires that the `cell.call` method accepts the same keyword argument `constants`. Such constants can be used to condition the cell transformation on additional static inputs (not changing over time), a.k.a. an attention mechanism. Fc s^|r tdt|ttfr"tdtt|j||||||f|tddg|_d|_ d|_ dS)Nz1Unrolling isn't possible with convolutional RNNs.z=It is not possible at the moment tostack convolutional cells.)ndim) TypeError isinstancelisttuplesuperr__init__r input_specstates_num_constants)selfcellreturn_sequences return_state go_backwardsstatefulZunrollkwargs) __class__h/var/www/html/venv/lib/python3.7/site-packages/tensorflow/python/keras/layers/convolutional_recurrent.pyrs zConvRNN2D.__init__cspttrd|jjdkr4ddnjdkrNddtjjdjjdj ddtjjdjjdj ddjdkrˆddj f}n"jdkrddj f}|j s|dd|dd}|j rl|g}jdkr@|fd d t dD7}n,jdkrl|fd d t dD7}|S) Nrchannels_first channels_last)paddingZstrideZdilationcsg|]}djfqS)r)filters).0_)rcols input_shaperowsr$r% sz2ConvRNN2D.compute_output_shape..csg|]}djfqS)r)r-)r.r/)rr0r1r2r$r%r3s)rrr data_formatr Zconv_output_length kernel_sizer+strides dilation_rater-rrrange)rr1Z output_shaper$)rr0r1r2r%compute_output_shapesD        zConvRNN2D.compute_output_shapecs|jdk r||j d}nd}t|tr2|d}|jr@|dnd}t|df|ddd|jd<t|jtr|df|dd}|dk r|j|g|n |j|t |jj drt|jj }n |jj g}|j dk r@|jj dkrdn|jj dkrd fd d |j D|kr~t d d d |j D|jj n>|jj dkr`dd |D|_ n|jj dkr~dd |D|_ |jr|d|_dS)Nrr*r)shape__len__r&r,r)r'csg|]}|jqSr$)r:)r.spec)ch_dimr$r%r3sz#ConvRNN2D.build..zAn initial_state was passed that is not compatible with `cell.state_size`. Received `state_spec`={}; However `cell.state_size` is {}cSsg|] }|jqSr$)r:)r.r<r$r$r%r3scSsg|]}td|ddfdqS)N)r:)r)r.dimr$r$r%r3 scSsg|]}tddd|fdqS)N)r:)r)r.r>r$r$r%r3 sT)rrrr!rrrrbuildhasattr state_sizeZ state_specr4 ValueErrorformat reset_statesbuilt)rr1Zconstants_shapeZ batch_sizeZstep_input_shaperAr$)r=r%r?sD          zConvRNN2D.buildcst|tjddt|jj}|jj|d<|jjt t |j |jj dt |jjdrxfdd|jjDSgSdS)Nr,)axis)r+r;csg|]}qSr$r$)r.r/) initial_stater$r%r3sz/ConvRNN2D.get_initial_state..)rZ zeros_likesumrr kernel_shaper- input_convrzerosrdtyper+r@rA)rinputsr:r$)rHr%get_initial_states     zConvRNN2D.get_initial_stateNc s|||\}}}t|tr&|d}t|d}itjjdrP|d<|r|tjjdslt dfdd}nfdd}tj ||||j ||d \}} } j rd d t j| D} | jr| } n|} jrt| ttfs| g} nt| } | g| S| SdS) Nrr,training constantsz#RNN cell does not support constantscs:|j d}|dj }jj||fd|iS)NrQ)rrcall)rNrrQ)r"rr$r%step:szConvRNN2D.call..stepcsjj||fS)N)rrR)rNr)r"rr$r%rS?s)rQr maskZ input_lengthcSsg|]\}}t||qSr$)rupdate)r.Z self_statestater$r$r%r3Ksz"ConvRNN2D.call..)Z_process_inputsrrrZ int_shaper has_argrrRrBZrnnr r!ziprZ add_updaterrr) rrNrTrPrHrQZ timestepsrSZ last_outputoutputsrZupdatesoutputr$)r"rr%rR#sB   zConvRNN2D.callcs@jstdjdj}|jr2djrRdddddkrbtdfddj ddkrt j j drfd d j j D_ nt j j g_ n||dkr4t j j drxTtj j j D]\}}t |t|qWn t j dtj j nt|ttfsJ|g}t|tj krtd jd ttj d dtt|dt|xtt|j D]\}\}}t j j drڈj j |}nj j }|j|kr*tdt|djdt|dt|jt ||qWdS)NzLayer must be stateful.rr,r*aIf a RNN is stateful, it needs to know its batch size. Specify the batch size of your input tensors: - If using a Sequential model, specify the batch size by passing a `batch_input_shape` argument to your first layer. - If using the functional API, specify the time dimension by passing a `batch_shape` argument to your Input layer. The same thing goes for the number of rows and columns.cs@t}jjdkr||d<njjdkr4||d<ntt|S)Nr&r,r)r')rrr4KeyErrorr)Z nb_channelsresult)r state_shaper$r%get_tuple_shapevs    z/ConvRNN2D.reset_states..get_tuple_shaper;csg|]}t|qSr$)rrL)r.r>)r^r$r%r3sz*ConvRNN2D.reset_states..zLayer z expects z states, zbut it received z state values. Input received: zState z is incompatible with layer z: expected shape=z, found shape=)r!AttributeErrorrr:r9rr concatenaterBrr@rrArrLrX set_valuenprrrlennamestr enumerate)rrr1rVr>indexvaluer$)r^rr]r%rD^sF      >6zConvRNN2D.reset_states)FFFFF)NNNN)N) __name__ __module__ __qualname____doc__rr Zshape_type_conversionr9r?rOrRrD __classcell__r$r$)r#r%r%ss)5 6rcsReZdZdZdfd d ZddZdddZdddZddZfddZ Z S)ConvLSTM2DCella Cell class for the ConvLSTM2D layer. Args: filters: Integer, the dimensionality of the output space (i.e. the number of output filters in the convolution). kernel_size: An integer or tuple/list of n integers, specifying the dimensions of the convolution window. strides: An integer or tuple/list of n integers, specifying the strides of the convolution. Specifying any stride value != 1 is incompatible with specifying any `dilation_rate` value != 1. padding: One of `"valid"` or `"same"` (case-insensitive). `"valid"` means no padding. `"same"` results in padding evenly to the left/right or up/down of the input such that output has the same height/width dimension as the input. data_format: A string, one of `channels_last` (default) or `channels_first`. It defaults to the `image_data_format` value found in your Keras config file at `~/.keras/keras.json`. If you never set it, then it will be "channels_last". dilation_rate: An integer or tuple/list of n integers, specifying the dilation rate to use for dilated convolution. Currently, specifying any `dilation_rate` value != 1 is incompatible with specifying any `strides` value != 1. activation: Activation function to use. If you don't specify anything, no activation is applied (ie. "linear" activation: `a(x) = x`). recurrent_activation: Activation function to use for the recurrent step. use_bias: Boolean, whether the layer uses a bias vector. kernel_initializer: Initializer for the `kernel` weights matrix, used for the linear transformation of the inputs. recurrent_initializer: Initializer for the `recurrent_kernel` weights matrix, used for the linear transformation of the recurrent state. bias_initializer: Initializer for the bias vector. unit_forget_bias: Boolean. If True, add 1 to the bias of the forget gate at initialization. Use in combination with `bias_initializer="zeros"`. This is recommended in [Jozefowicz et al., 2015]( http://www.jmlr.org/proceedings/papers/v37/jozefowicz15.pdf) kernel_regularizer: Regularizer function applied to the `kernel` weights matrix. recurrent_regularizer: Regularizer function applied to the `recurrent_kernel` weights matrix. bias_regularizer: Regularizer function applied to the bias vector. kernel_constraint: Constraint function applied to the `kernel` weights matrix. recurrent_constraint: Constraint function applied to the `recurrent_kernel` weights matrix. bias_constraint: Constraint function applied to the bias vector. dropout: Float between 0 and 1. Fraction of the units to drop for the linear transformation of the inputs. recurrent_dropout: Float between 0 and 1. Fraction of the units to drop for the linear transformation of the recurrent state. Call arguments: inputs: A 4D tensor. states: List of state tensors corresponding to the previous timestep. training: Python boolean indicating whether the layer should behave in training mode or in inference mode. Only relevant when `dropout` or `recurrent_dropout` is used. r,r,validNtanh hard_sigmoidTglorot_uniform orthogonalrLc s&tt|jf|||_t|dd|_t|dd|_t||_ t ||_ t|dd|_ t ||_t ||_| |_t| |_t| |_t| |_| |_t||_t||_t||_t||_t||_t||_tdt d||_!tdt d||_"|j|jf|_#dS)Nr*r5r6r7g?g)$rrnrr-r Znormalize_tupler5r6Znormalize_paddingr+Znormalize_data_formatr4r7rget activationrecurrent_activationuse_biasrkernel_initializerrecurrent_initializerbias_initializerunit_forget_biasrkernel_regularizerrecurrent_regularizerbias_regularizerrkernel_constraintrecurrent_constraintbias_constraintminmaxdropoutrecurrent_dropoutrA)rr-r5r6r+r4r7rwrxryrzr{r|r}r~rrrrrrrr")r#r$r%rs0             zConvLSTM2DCell.__init__csjdkrd}nd}||dkr(td||}j|jdf}|_jjjdf}j|jdjjd_ j|j dj j d_ jr܈jrfd d }nj}jjdfd |jjd _nd_d _dS)Nr&r,rGzDThe channel dimension of the inputs should be defined. Found `None`.r(kernel)r: initializerrd regularizer constraintrecurrent_kernelcsNtjjff||tdjff||jjdff||gS)NZonesr*)rr`r|r-rrv)r/argsr")rr$r%r|9sz.ConvLSTM2DCell.build..bias_initializerbias)r:rdrrrT)r4rBr5r-rJZ add_weightrzr~rrr{rrrryr}r|rrrrE)rr1Z channel_axisZ input_dimrJZrecurrent_kernel_shaper|r$)rr%r?s@     zConvLSTM2DCell.buildc)Cs&|d}|d}|j||dd}|j||dd}d|jkrFdkr|nn2||d}||d} ||d} ||d} n|}|} |} |} d|jkrdkrnn2||d} ||d} ||d}||d}n|} |} |}|}tj|jddd\}}}}tj|jddd\}}}}|jrZZS)A ConvLSTM2DaN2D Convolutional LSTM layer. A convolutional LSTM is similar to an LSTM, but the input transformations and recurrent transformations are both convolutional. This layer is typically used to process timeseries of images (i.e. video-like data). It is known to perform well for weather data forecasting, using inputs that are timeseries of 2D grids of sensor values. It isn't usually applied to regular video data, due to its high computational cost. Args: filters: Integer, the dimensionality of the output space (i.e. the number of output filters in the convolution). kernel_size: An integer or tuple/list of n integers, specifying the dimensions of the convolution window. strides: An integer or tuple/list of n integers, specifying the strides of the convolution. Specifying any stride value != 1 is incompatible with specifying any `dilation_rate` value != 1. padding: One of `"valid"` or `"same"` (case-insensitive). `"valid"` means no padding. `"same"` results in padding evenly to the left/right or up/down of the input such that output has the same height/width dimension as the input. data_format: A string, one of `channels_last` (default) or `channels_first`. The ordering of the dimensions in the inputs. `channels_last` corresponds to inputs with shape `(batch, time, ..., channels)` while `channels_first` corresponds to inputs with shape `(batch, time, channels, ...)`. It defaults to the `image_data_format` value found in your Keras config file at `~/.keras/keras.json`. If you never set it, then it will be "channels_last". dilation_rate: An integer or tuple/list of n integers, specifying the dilation rate to use for dilated convolution. Currently, specifying any `dilation_rate` value != 1 is incompatible with specifying any `strides` value != 1. activation: Activation function to use. By default hyperbolic tangent activation function is applied (`tanh(x)`). recurrent_activation: Activation function to use for the recurrent step. use_bias: Boolean, whether the layer uses a bias vector. kernel_initializer: Initializer for the `kernel` weights matrix, used for the linear transformation of the inputs. recurrent_initializer: Initializer for the `recurrent_kernel` weights matrix, used for the linear transformation of the recurrent state. bias_initializer: Initializer for the bias vector. unit_forget_bias: Boolean. If True, add 1 to the bias of the forget gate at initialization. Use in combination with `bias_initializer="zeros"`. This is recommended in [Jozefowicz et al., 2015]( http://www.jmlr.org/proceedings/papers/v37/jozefowicz15.pdf) kernel_regularizer: Regularizer function applied to the `kernel` weights matrix. recurrent_regularizer: Regularizer function applied to the `recurrent_kernel` weights matrix. bias_regularizer: Regularizer function applied to the bias vector. activity_regularizer: Regularizer function applied to. kernel_constraint: Constraint function applied to the `kernel` weights matrix. recurrent_constraint: Constraint function applied to the `recurrent_kernel` weights matrix. bias_constraint: Constraint function applied to the bias vector. return_sequences: Boolean. Whether to return the last output in the output sequence, or the full sequence. (default False) return_state: Boolean Whether to return the last state in addition to the output. (default False) go_backwards: Boolean (default False). If True, process the input sequence backwards. stateful: Boolean (default False). If True, the last state for each sample at index i in a batch will be used as initial state for the sample of index i in the following batch. dropout: Float between 0 and 1. Fraction of the units to drop for the linear transformation of the inputs. recurrent_dropout: Float between 0 and 1. Fraction of the units to drop for the linear transformation of the recurrent state. Call arguments: inputs: A 5D float tensor (see input shape description below). mask: Binary tensor of shape `(samples, timesteps)` indicating whether a given timestep should be masked. training: Python boolean indicating whether the layer should behave in training mode or in inference mode. This argument is passed to the cell when calling it. This is only relevant if `dropout` or `recurrent_dropout` are set. initial_state: List of initial state tensors to be passed to the first call of the cell. Input shape: - If data_format='channels_first' 5D tensor with shape: `(samples, time, channels, rows, cols)` - If data_format='channels_last' 5D tensor with shape: `(samples, time, rows, cols, channels)` Output shape: - If `return_state`: a list of tensors. The first tensor is the output. The remaining tensors are the last states, each 4D tensor with shape: `(samples, filters, new_rows, new_cols)` if data_format='channels_first' or 4D tensor with shape: `(samples, new_rows, new_cols, filters)` if data_format='channels_last'. `rows` and `cols` values might have changed due to padding. - If `return_sequences`: 5D tensor with shape: `(samples, timesteps, filters, new_rows, new_cols)` if data_format='channels_first' or 5D tensor with shape: `(samples, timesteps, new_rows, new_cols, filters)` if data_format='channels_last'. - Else, 4D tensor with shape: `(samples, filters, new_rows, new_cols)` if data_format='channels_first' or 4D tensor with shape: `(samples, new_rows, new_cols, filters)` if data_format='channels_last'. Raises: ValueError: in case of invalid constructor arguments. References: - [Shi et al., 2015](http://arxiv.org/abs/1506.04214v1) (the current implementation does not include the feedback loop on the cells output). Example: ```python steps = 10 height = 32 width = 32 input_channels = 3 output_channels = 6 inputs = tf.keras.Input(shape=(steps, height, width, input_channels)) layer = tf.keras.layers.ConvLSTM2D(filters=output_channels, kernel_size=3) outputs = layer(inputs) ``` r,r,rpNrqrrTrsrtrLFc slt||||||||| | | | | |||||||||dd}tt|j|f||||d|t||_dS)NrM)r-r5r6r+r4r7rwrxryrzr{r|r}r~rrrrrrrrM)rrr r!)rnrvrrrractivity_regularizer)rr-r5r6r+r4r7rwrxryrzr{r|r}r~rrrrrrrrr r!rrr"r)r#r$r%rMs:zConvLSTM2D.__init__cstt|j||||dS)N)rTrPrH)rrrR)rrNrTrPrH)r#r$r%rRs zConvLSTM2D.callcCs|jjS)N)rr-)rr$r$r%r-szConvLSTM2D.filterscCs|jjS)N)rr5)rr$r$r%r5szConvLSTM2D.kernel_sizecCs|jjS)N)rr6)rr$r$r%r6szConvLSTM2D.stridescCs|jjS)N)rr+)rr$r$r%r+szConvLSTM2D.paddingcCs|jjS)N)rr4)rr$r$r%r4szConvLSTM2D.data_formatcCs|jjS)N)rr7)rr$r$r%r7szConvLSTM2D.dilation_ratecCs|jjS)N)rrw)rr$r$r%rwszConvLSTM2D.activationcCs|jjS)N)rrx)rr$r$r%rxszConvLSTM2D.recurrent_activationcCs|jjS)N)rry)rr$r$r%ryszConvLSTM2D.use_biascCs|jjS)N)rrz)rr$r$r%rzszConvLSTM2D.kernel_initializercCs|jjS)N)rr{)rr$r$r%r{sz ConvLSTM2D.recurrent_initializercCs|jjS)N)rr|)rr$r$r%r|szConvLSTM2D.bias_initializercCs|jjS)N)rr})rr$r$r%r}szConvLSTM2D.unit_forget_biascCs|jjS)N)rr~)rr$r$r%r~szConvLSTM2D.kernel_regularizercCs|jjS)N)rr)rr$r$r%rsz ConvLSTM2D.recurrent_regularizercCs|jjS)N)rr)rr$r$r%rszConvLSTM2D.bias_regularizercCs|jjS)N)rr)rr$r$r%rszConvLSTM2D.kernel_constraintcCs|jjS)N)rr)rr$r$r%rszConvLSTM2D.recurrent_constraintcCs|jjS)N)rr)rr$r$r%rszConvLSTM2D.bias_constraintcCs|jjS)N)rr)rr$r$r%rszConvLSTM2D.dropoutcCs|jjS)N)rr)rr$r$r%rszConvLSTM2D.recurrent_dropoutcs|j|j|j|j|j|jt|jt|j |j t |j t |j t |j|jt|jt|jt|jt|jt|jt|jt|j|j|jd}tt|}|d=tt| t| S)N)r-r5r6r+r4r7rwrxryrzr{r|r}r~rrrrrrrrr)!r-r5r6r+r4r7rrrwrxryrrzr{r|r}rr~rrrrrrrrrrrrrrr)rrr)r#r$r%rsB     zConvLSTM2D.get_configcCs |f|S)Nr$)clsrr$r$r% from_configszConvLSTM2D.from_config)rrpNrrqrrTrsrtrLTNNNNNNNFFFFrr)NNN) rirjrkrlrrRpropertyr-r5r6r+r4r7rwrxryrzr{r|r}r~rrrrrrrr classmethodrrmr$r$)r#r%rsd                       #r)rlnumpyrbZtensorflow.python.kerasrrrrrZ)tensorflow.python.keras.engine.base_layerrZ)tensorflow.python.keras.engine.input_specrZ(tensorflow.python.keras.layers.recurrentr r Ztensorflow.python.keras.utilsr r r Ztensorflow.python.opsrZ tensorflow.python.util.tf_exportrrrnrr$r$r$r%s0