B ӻd7@s dZddlmZddlmZddlmZddlmZddlmZddlm Z ddl m Z dd l m Z dd lmZdd lmZd d ZedGddde ZedGddde ZedGddde ZedGddde ZddZedGddde ZedGd d!d!e Zd"S)#z(Layers that act as activation functions.)dtypes)backend) constraints) initializers) regularizers)Layer) InputSpec)tf_utils)math_ops) keras_exportcCstS)N)globalsr r e/var/www/html/venv/lib/python3.7/site-packages/tensorflow/python/keras/layers/advanced_activations.py get_globalssrzkeras.layers.LeakyReLUcsDeZdZdZd fdd ZddZfddZejd d Z Z S) LeakyReLUaLeaky version of a Rectified Linear Unit. It allows a small gradient when the unit is not active: ``` f(x) = alpha * x if x < 0 f(x) = x if x >= 0 ``` Usage: >>> layer = tf.keras.layers.LeakyReLU() >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [-0.9, -0.3, 0.0, 2.0] >>> layer = tf.keras.layers.LeakyReLU(alpha=0.1) >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [-0.3, -0.1, 0.0, 2.0] Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the batch axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: alpha: Float >= 0. Negative slope coefficient. Default to 0.3. 333333?c s<tt|jf||dkr&td|d|_t||_dS)NzKThe alpha value of a Leaky ReLU layer cannot be None, needs a float. Got %sT)superr__init__ ValueErrorsupports_maskingrcast_to_floatxalpha)selfrkwargs) __class__r rrEs zLeakyReLU.__init__cCstj||jdS)N)r)rrelur)rinputsr r rcallNszLeakyReLU.callcs8dt|ji}tt|}tt|t|S)Nr)floatrrr get_configdictlistitems)rconfig base_config)rr rrQszLeakyReLU.get_configcCs|S)Nr )r input_shaper r rcompute_output_shapeVszLeakyReLU.compute_output_shape)r) __name__ __module__ __qualname____doc__rrrr shape_type_conversionr& __classcell__r r )rrr"s !  rzkeras.layers.PReLUcsReZdZdZdfdd ZejddZdd Zfd d Z ejd d Z Z S)PReLUaParametric Rectified Linear Unit. It follows: ``` f(x) = alpha * x for x < 0 f(x) = x for x >= 0 ``` where `alpha` is a learned array with the same shape as x. Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: alpha_initializer: Initializer function for the weights. alpha_regularizer: Regularizer for the weights. alpha_constraint: Constraint for the weights. shared_axes: The axes along which to share learnable parameters for the activation function. For example, if the incoming feature maps are from a 2D convolution with output shape `(batch, height, width, channels)`, and you wish to share parameters across space so that each filter only has one set of parameters, set `shared_axes=[1, 2]`. zerosNc srtt|jf|d|_t||_t||_t ||_ |dkrLd|_ n"t |t tfsd|g|_ n t ||_ dS)NT)rr-rrrgetalpha_initializerralpha_regularizerralpha_constraint shared_axes isinstancer!tuple)rr0r1r2r3r)rr rr~s    zPReLU.__init__cCst|dd}|jdk r6x|jD]}d||d<q"W|j|d|j|j|jd|_i}|jrx,tdt|D]}||jkrl||||<qlWt t||d|_ d|_ dS)Nr)shapenameZ initializerZ regularizer constraint)ndimaxesT) r!r3Z add_weightr0r1r2rrangelenrZ input_specZbuilt)rr%Z param_shapeir;r r rbuilds"    z PReLU.buildcCs&t|}|j t| }||S)N)rrr)rrposnegr r rrs z PReLU.callcsRt|jt|jt|j|jd}tt | }t t | t | S)N)r0r1r2r3)r serializer0rr1rr2r3rr-rr r!r")rr#r$)rr rrs     zPReLU.get_configcCs|S)Nr )rr%r r rr&szPReLU.compute_output_shape)r.NNN) r'r(r)r*rr r+r?rrr&r,r r )rrr-[s! r-zkeras.layers.ELUcsDeZdZdZd fdd ZddZfddZejd d Z Z S) ELUaExponential Linear Unit. It follows: ``` f(x) = alpha * (exp(x) - 1.) for x < 0 f(x) = x for x >= 0 ``` Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: alpha: Scale for the negative factor. ?c s<tt|jf||dkr&td|d|_t||_dS)Nz>Alpha of an ELU layer cannot be None, requires a float. Got %sT)rrCrrrrrr)rrr)rr rrs z ELU.__init__cCst||jS)N)rZelur)rrr r rrszELU.callcs8dt|ji}tt|}tt|t|S)Nr)rrrrCrr r!r")rr#r$)rr rrszELU.get_configcCs|S)Nr )rr%r r rr&szELU.compute_output_shape)rD) r'r(r)r*rrrr r+r&r,r r )rrrCs  rCzkeras.layers.ThresholdedReLUcsDeZdZdZd fdd ZddZfddZejd d Z Z S) ThresholdedReLUaThresholded Rectified Linear Unit. It follows: ``` f(x) = x for x > theta f(x) = 0 otherwise` ``` Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: theta: Float >= 0. Threshold location of activation. ?c sPtt|jf||dkr&td||dkr:td|d|_t||_dS)NzJTheta of a Thresholded ReLU layer cannot be None, requires a float. Got %srzAThe theta value of a Thresholded ReLU layer should be >=0, got %sT)rrErrrrrtheta)rrGr)rr rrszThresholdedReLU.__init__cCs*t|j|j}|tt|||jS)N)r castrGdtypeZgreater)rrrGr r rrszThresholdedReLU.callcs8dt|ji}tt|}tt|t|S)NrG)rrGrrErr r!r")rr#r$)rr rr szThresholdedReLU.get_configcCs|S)Nr )rr%r r rr&sz$ThresholdedReLU.compute_output_shape)rF) r'r(r)r*rrrr r+r&r,r r )rrrEs   rEcCs|tjkrtjjSdS)a Large negative number as Tensor. This function is necessary because the standard value for epsilon in this module (-1e9) cannot be represented using tf.float16 Args: tensor_type: a dtype to determine the type. Returns: a large negative number. ge)rZfloat16min)Z tensor_typer r r_large_compatible_negatives rKzkeras.layers.SoftmaxcsFeZdZdZd fdd Zd ddZfdd Zejd d Z Z S)SoftmaxaSoftmax activation function. Example without mask: >>> inp = np.asarray([1., 2., 1.]) >>> layer = tf.keras.layers.Softmax() >>> layer(inp).numpy() array([0.21194157, 0.5761169 , 0.21194157], dtype=float32) >>> mask = np.asarray([True, False, True], dtype=bool) >>> layer(inp, mask).numpy() array([0.5, 0. , 0.5], dtype=float32) Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the samples axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: axis: Integer, or list of Integers, axis along which the softmax normalization is applied. Call arguments: inputs: The inputs, or logits to the softmax layer. mask: A boolean mask of the same shape as `inputs`. Defaults to `None`. The mask specifies 1 to keep and 0 to mask. Returns: softmaxed output with the same shape as `inputs`. c s"tt|jf|d|_||_dS)NT)rrLrraxis)rrNr)rr rrHszSoftmax.__init__NcCs|dk r,dt||jt|j}||7}t|jttfrzt|jdkrft |tj ||jddSt j ||jddSt j ||jdS)Ng?r6T)rNZkeepdimsr)rN) r rHrIrKr4rNr5r!r=expZreduce_logsumexprZsoftmax)rrmaskZadderr r rrMs  z Softmax.callcs4d|ji}tt|}tt|t|S)NrN)rNrrLrr r!r")rr#r$)rr rr`s zSoftmax.get_configcCs|S)Nr )rr%r r rr&eszSoftmax.compute_output_shape)rM)N) r'r(r)r*rrrr r+r&r,r r )rrrL&s    rLzkeras.layers.ReLUcsDeZdZdZd fdd ZddZfdd Zejd d Z Z S) ReLUaYRectified Linear Unit activation function. With default values, it returns element-wise `max(x, 0)`. Otherwise, it follows: ``` f(x) = max_value if x >= max_value f(x) = x if threshold <= x < max_value f(x) = negative_slope * (x - threshold) otherwise ``` Usage: >>> layer = tf.keras.layers.ReLU() >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [0.0, 0.0, 0.0, 2.0] >>> layer = tf.keras.layers.ReLU(max_value=1.0) >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [0.0, 0.0, 0.0, 1.0] >>> layer = tf.keras.layers.ReLU(negative_slope=1.0) >>> output = layer([-3.0, -1.0, 0.0, 2.0]) >>> list(output.numpy()) [-3.0, -1.0, 0.0, 2.0] >>> layer = tf.keras.layers.ReLU(threshold=1.5) >>> output = layer([-3.0, -1.0, 1.0, 2.0]) >>> list(output.numpy()) [0.0, 0.0, 0.0, 2.0] Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of integers, does not include the batch axis) when using this layer as the first layer in a model. Output shape: Same shape as the input. Args: max_value: Float >= 0. Maximum activation value. Default to None, which means unlimited. negative_slope: Float >= 0. Negative slope coefficient. Default to 0. threshold: Float >= 0. Threshold value for thresholded activation. Default to 0. Nrc stt|jf||dk r.|dkr.td||dks>|dkrJtd||dksZ|dkrftd|d|_|dk r~t|}||_t||_t||_ dS)Ngz=max_value of a ReLU layer cannot be a negative value. Got: %szBnegative_slope of a ReLU layer cannot be a negative value. Got: %sz=threshold of a ReLU layer cannot be a negative value. Got: %sT) rrQrrrrr max_valuenegative_slope threshold)rrRrSrTr)rr rrs   z ReLU.__init__cCstj||j|j|jdS)N)rrRrT)rrrSrRrT)rrr r rrsz ReLU.callcs<|j|j|jd}tt|}tt|t|S)N)rRrSrT) rRrSrTrrQrr r!r")rr#r$)rr rrs  zReLU.get_configcCs|S)Nr )rr%r r rr&szReLU.compute_output_shape)Nrr) r'r(r)r*rrrr r+r&r,r r )rrrQjs / rQN)r*Ztensorflow.python.frameworkrZtensorflow.python.kerasrrrrZ)tensorflow.python.keras.engine.base_layerrZ)tensorflow.python.keras.engine.input_specrZtensorflow.python.keras.utilsr Ztensorflow.python.opsr Z tensorflow.python.util.tf_exportr rrr-rCrErKrLrQr r r rs0          8]+/C