B ӻdvQ@sdZddlZddlmZddlmZddlmZddlmZddlmZddl m Z dd l m Z dd l mZdd lmZdd lmZdd lmZddlmZGdddeZedgdGdddeZedgdGdddeZdS)z!Recurrent layers backed by cuDNN.N) constant_op)backend) constraints) initializers) regularizers) InputSpec) recurrent_v2)RNN) array_ops)gen_cudnn_rnn_ops) state_ops) keras_exportcszeZdZdZdfdd ZdddZfdd Zed d Ze d d Z e ddZ e fddZ dfdd Z ZS) _CuDNNRNNa-Private base class for CuDNNGRU and CuDNNLSTM layers. Args: return_sequences: Boolean. Whether to return the last output in the output sequence, or the full sequence. return_state: Boolean. Whether to return the last state in addition to the output. go_backwards: Boolean (default False). If True, process the input sequence backwards and return the reversed sequence. stateful: Boolean (default False). If True, the last state for each sample at index i in a batch will be used as initial state for the sample of index i in the following batch. time_major: Boolean (default False). If true, the inputs and outputs will be in shape `(timesteps, batch, ...)`, whereas in the False case, it will be `(batch, timesteps, ...)`. Fc stt|jf|||_||_||_||_||_d|_t ddg|_ t |j j dr\|j j }n |j j g}dd|D|_d|_d|_d|_tdg|_dS) NF)ndim__len__cSsg|]}td|fdqS)N)shape)r).0dimr`/var/www/html/venv/lib/python3.7/site-packages/tensorflow/python/keras/layers/cudnn_recurrent.py Isz&_CuDNNRNN.__init__..r)superr __init__return_sequences return_state go_backwardsstateful time_majorZsupports_maskingrZ input_spechasattrcell state_sizeZ state_specZconstants_spec_statesZ_num_constantsrZconstant _vector_shape)selfrrrrrkwargsr") __class__rrr4s   z_CuDNNRNN.__init__NcCst|tr|d}|dk r"tdt|trB|dd}|d}n"|dk rLn|jrZ|j}n ||}t|t|jkrtdtt|jdtt|d|jrt |d}| ||\}}|jrddt |j|D}| ||jr|g|S|SdS) Nrz(Masking is not supported for CuDNN RNNs.z Layer has z states but was passed z initial states.cSsg|]\}}t||qSr)r Zassign)rZ self_statestaterrrrnsz"_CuDNNRNN.call..) isinstancelist ValueErrorrstatesZget_initial_statelenstrrrreverse_process_batchzipZ add_updater)r%inputsmaskZtraining initial_stateoutputr-ZupdatesrrrcallOs4         z_CuDNNRNN.callcsD|j|j|j|j|jd}tt|}tt | t | S)N)rrrrr) rrrrrrr get_configdictr+items)r%config base_config)r'rrr8xs  z_CuDNNRNN.get_configcCs |f|S)Nr)clsr;rrr from_configsz_CuDNNRNN.from_configcCs |jr|jr|j|j|jgSgS)N) trainablebuiltkernelrecurrent_kernelbias)r%rrrtrainable_weightss z_CuDNNRNN.trainable_weightscCs |js|jr|j|j|jgSgS)N)r?r@rArBrC)r%rrrnon_trainable_weightss z_CuDNNRNN.non_trainable_weightscs tt|jS)N)rr losses)r%)r'rrrFsz_CuDNNRNN.lossescstt|j|dS)N)r3)rr get_losses_for)r%r3)r'rrrGsz_CuDNNRNN.get_losses_for)FFFFF)NNN)N)__name__ __module__ __qualname____doc__rr7r8 classmethodr>propertyrDrErFrG __classcell__rr)r'rr!s )   rzkeras.layers.CuDNNGRU)Zv1csNeZdZdZdfdd Zed d Zfd d Zd dZfddZ Z S)CuDNNGRUaFast GRU implementation backed by cuDNN. More information about cuDNN can be found on the [NVIDIA developer website](https://developer.nvidia.com/cudnn). Can only be run on GPU. Args: units: Positive integer, dimensionality of the output space. kernel_initializer: Initializer for the `kernel` weights matrix, used for the linear transformation of the inputs. recurrent_initializer: Initializer for the `recurrent_kernel` weights matrix, used for the linear transformation of the recurrent state. bias_initializer: Initializer for the bias vector. kernel_regularizer: Regularizer function applied to the `kernel` weights matrix. recurrent_regularizer: Regularizer function applied to the `recurrent_kernel` weights matrix. bias_regularizer: Regularizer function applied to the bias vector. activity_regularizer: Regularizer function applied to the output of the layer (its "activation"). kernel_constraint: Constraint function applied to the `kernel` weights matrix. recurrent_constraint: Constraint function applied to the `recurrent_kernel` weights matrix. bias_constraint: Constraint function applied to the bias vector. return_sequences: Boolean. Whether to return the last output in the output sequence, or the full sequence. return_state: Boolean. Whether to return the last state in addition to the output. go_backwards: Boolean (default False). If True, process the input sequence backwards and return the reversed sequence. stateful: Boolean (default False). If True, the last state for each sample at index i in a batch will be used as initial state for the sample of index i in the following batch. glorot_uniform orthogonalzerosNFc s||_tdd}||jd|_tt|jf| | ||d|t||_ t||_ t||_ t ||_ t ||_t ||_t ||_t| |_t| |_t| |_dS)Nr!r")r")rrrr)units collections namedtuple_cellrrOrrgetkernel_initializerrecurrent_initializerbias_initializerrkernel_regularizerrecurrent_regularizerbias_regularizeractivity_regularizerrkernel_constraintrecurrent_constraintbias_constraint)r%rSrXrYrZr[r\r]r^r_r`rarrrrr& cell_spec)r'rrrs&           zCuDNNGRU.__init__cCs|jS)N)rV)r%rrrr!sz CuDNNGRU.cellcstt||t|tr"|d}t|d}|j||jdfd|j|j |j d|_ |j|j|jdfd|j |j |jd|_|j|jdfd|j|j|jd|_d |_dS) NrrrrA)rname initializer regularizer constraintrBrCT)rrObuildr*r+int add_weightrSrXr[r_rArYr\r`rBrZr]rarCr@)r% input_shape input_dim)r'rrrhs.       zCuDNNGRU.buildc Cs|jstj|dd}|d}tj|dd}tj|jdd|j|jdf|jddd|jf|jdd|jddf|jdd|j|jdf|jddd|jf|jdd|jddfg|j |j|jd|j d|j|j |jd|jd|j |jd|jd|j |jd|jd|j |jddg|j d }||d|d d d }t j f|\}}}}}|j s|jr|d}|jr|jr|} ntj|dd} n|d } | |gfS)N)r(r)permr)axisrmr)weightsbiasesrTZgru)inputinput_hinput_cparams is_trainingZrnn_moder)rr transpose expand_dimsr_canonical_to_paramsrArSrBrCr$r CudnnRNNV2rrr) r%r3r5rurwargsoutputsh_r6rrrr1 sB zCuDNNGRU._process_batchc s|jt|jt|jt|jt|jt|jt|j t|j t |j t |j t |jd }tt|}tt|t|S)N) rSrXrYrZr[r\r]r^r_r`ra)rSr serializerXrYrZrr[r\r]r^rr_r`rarrOr8r9r+r:)r%r;r<)r'rrr8;s         zCuDNNGRU.get_config)rPrQrRNNNNNNNFFFF) rHrIrJrKrrMr!rhr1r8rNrr)r'rrOs&$  /rOzkeras.layers.CuDNNLSTMcsNeZdZdZdfdd Zed d Zfd d ZddZfddZ Z S) CuDNNLSTMa3Fast LSTM implementation backed by cuDNN. More information about cuDNN can be found on the [NVIDIA developer website](https://developer.nvidia.com/cudnn). Can only be run on GPU. Args: units: Positive integer, dimensionality of the output space. kernel_initializer: Initializer for the `kernel` weights matrix, used for the linear transformation of the inputs. unit_forget_bias: Boolean. If True, add 1 to the bias of the forget gate at initialization. Setting it to true will also force `bias_initializer="zeros"`. This is recommended in [Jozefowicz et al.](http://www.jmlr.org/proceedings/papers/v37/jozefowicz15.pdf) recurrent_initializer: Initializer for the `recurrent_kernel` weights matrix, used for the linear transformation of the recurrent state. bias_initializer: Initializer for the bias vector. kernel_regularizer: Regularizer function applied to the `kernel` weights matrix. recurrent_regularizer: Regularizer function applied to the `recurrent_kernel` weights matrix. bias_regularizer: Regularizer function applied to the bias vector. activity_regularizer: Regularizer function applied to the output of the layer (its "activation"). kernel_constraint: Constraint function applied to the `kernel` weights matrix. recurrent_constraint: Constraint function applied to the `recurrent_kernel` weights matrix. bias_constraint: Constraint function applied to the bias vector. return_sequences: Boolean. Whether to return the last output. in the output sequence, or the full sequence. return_state: Boolean. Whether to return the last state in addition to the output. go_backwards: Boolean (default False). If True, process the input sequence backwards and return the reversed sequence. stateful: Boolean (default False). If True, the last state for each sample at index i in a batch will be used as initial state for the sample of index i in the following batch. rPrQrRTNFc s||_tdd}||j|jfd|_tt|jf| |||d|t||_ t||_ t||_ ||_ t ||_t ||_t ||_t | |_t| |_t| |_t| |_dS)Nr!r")r")rrrr)rSrTrUrVrrrrrWrXrYrZunit_forget_biasrr[r\r]r^rr_r`ra)r%rSrXrYrZrr[r\r]r^r_r`rarrrrr&rb)r'rrr{s(           zCuDNNLSTM.__init__cCs|jS)N)rV)r%rrrr!szCuDNNLSTM.cellcstt|t|tr"|d}t|d}j|jdfdjj j d_ jjjdfdj j jd_jrfdd}nj}jjd fd |jjd_d _dS) NrrrprA)rrcrdrerfrBcsTtjjjdff||tjff||jjdff||gddS)Nrqrmr)ro)r concatrZrSrZOnes)rr}r&)r%rrrZs z)CuDNNLSTM.build..bias_initializerrCT)rrrhr*r+rirjrSrXr[r_rArYr\r`rBrrZr]rarCr@)r%rkrlrZ)r')r%rrhs4       zCuDNNLSTM.buildc CsV|jstj|dd}|d}|d}tj|dd}tj|dd}tj|jddd|jf|jdd|j|jdf|jdd|jd|jdf|jdd|jddf|jddd|jf|jdd|j|jdf|jdd|jd|jdf|jdd|jddfg|j d|j|j |j|jd|j |jd|jd|j |jd|jd|j |jd|jd |j |jd |jd |j |jd |jd |j |jd dg|j d }||||d d}t j f|\}}} } } |j s |jr|d}| d} |jrB|jr2|} ntj|dd} n|d} | || gfS)N)r(rrm)rnrr()rormrrprqrg)rrrsrT)rtrurvrwrxr)rr ryrzrr{rArSrBrCr$r r|rrr) r%r3r5rurvrwr}r~rcrr6rrrr1sN   zCuDNNLSTM._process_batchcs|jt|jt|jt|j|jt|jt|j t|j t|j t |j t |jt |jd }tt|}tt|t|S)N) rSrXrYrZrr[r\r]r^r_r`ra)rSrrrXrYrZrrr[r\r]r^rr_r`rarrr8r9r+r:)r%r;r<)r'rrr8s         zCuDNNLSTM.get_config)rPrQrRTNNNNNNNFFFF) rHrIrJrKrrMr!rhr1r8rNrr)r'rrQs((  '5r)rKrTZtensorflow.python.frameworkrZtensorflow.python.kerasrrrrZ)tensorflow.python.keras.engine.input_specrZtensorflow.python.keras.layersrZ(tensorflow.python.keras.layers.recurrentr Ztensorflow.python.opsr r r Z tensorflow.python.util.tf_exportr rrOrrrrrs&            |4