B ӻdV#@sdZddlmZddlmZddlmZddlmZddlmZddlm Z ddl m Z dd l m Z dd l mZdd lmZed Gd dde ZdS)zEmbedding layer.)backend) constraints) initializers) regularizers)base_layer_utils)Layer)tf_utils) embedding_ops)math_ops) keras_exportzkeras.layers.Embeddingcs^eZdZdZdfdd ZejdddZdd d Zejd d Z d dZ fddZ Z S) Embeddinga Turns positive integers (indexes) into dense vectors of fixed size. e.g. `[[4], [20]] -> [[0.25, 0.1], [0.6, -0.2]]` This layer can only be used as the first layer in a model. Example: >>> model = tf.keras.Sequential() >>> model.add(tf.keras.layers.Embedding(1000, 64, input_length=10)) >>> # The model will take as input an integer matrix of size (batch, >>> # input_length), and the largest integer (i.e. word index) in the input >>> # should be no larger than 999 (vocabulary size). >>> # Now model.output_shape is (None, 10, 64), where `None` is the batch >>> # dimension. >>> input_array = np.random.randint(1000, size=(32, 10)) >>> model.compile('rmsprop', 'mse') >>> output_array = model.predict(input_array) >>> print(output_array.shape) (32, 10, 64) Args: input_dim: Integer. Size of the vocabulary, i.e. maximum integer index + 1. output_dim: Integer. Dimension of the dense embedding. embeddings_initializer: Initializer for the `embeddings` matrix (see `keras.initializers`). embeddings_regularizer: Regularizer function applied to the `embeddings` matrix (see `keras.regularizers`). embeddings_constraint: Constraint function applied to the `embeddings` matrix (see `keras.constraints`). mask_zero: Boolean, whether or not the input value 0 is a special "padding" value that should be masked out. This is useful when using recurrent layers which may take variable length input. If this is `True`, then all subsequent layers in the model need to support masking or an exception will be raised. If mask_zero is set to True, as a consequence, index 0 cannot be used in the vocabulary (input_dim should equal size of vocabulary + 1). input_length: Length of input sequences, when it is constant. This argument is required if you are going to connect `Flatten` then `Dense` layers upstream (without it, the shape of the dense outputs cannot be computed). Input shape: 2D tensor with shape: `(batch_size, input_length)`. Output shape: 3D tensor with shape: `(batch_size, input_length, output_dim)`. **Note on variable placement:** By default, if a GPU is available, the embedding matrix will be placed on the GPU. This achieves the best performance, but it might cause issues: - You may be using an optimizer that does not support sparse GPU kernels. In this case you will see an error upon training your model. - Your embedding matrix may be too large to fit on your GPU. In this case you will see an Out Of Memory (OOM) error. In such cases, you should place the embedding matrix on the CPU memory. You can do so with a device scope, as such: ```python with tf.device('cpu:0'): embedding_layer = Embedding(...) embedding_layer.build() ``` The pre-built `embedding_layer` instance can then be added to a `Sequential` model (e.g. `model.add(embedding_layer)`), called in a Functional model (e.g. `x = embedding_layer(x)`), or used in a subclassed model. uniformNFc  sd| kr |r|f| d<nd| d<|dks0|dkr@td||ts\d| kr\t| d<d| d<tt|jf| ||_ ||_ t ||_ t ||_t ||_t ||_||_||_||_dS)N input_shape)NrzZBoth `input_dim` and `output_dim` should be positive, found input_dim {} and output_dim {}dtypeFZautocast) ValueErrorformatrZv2_dtype_behavior_enabledrZfloatxsuperr __init__ input_dim output_dimrgetembeddings_initializerrembeddings_regularizeractivity_regularizerrembeddings_constraint mask_zeroZsupports_masking input_length) selfrrrrrrrrkwargs) __class__[/var/www/html/venv/lib/python3.7/site-packages/tensorflow/python/keras/layers/embeddings.pyrjs*        zEmbedding.__init__cCs0|j|j|jf|jd|j|jdd|_d|_dS)N embeddingsF)shapeZ initializernameZ regularizer constraintZexperimental_autocastT)Z add_weightrrrrrr"Zbuilt)rrr r r!builds  zEmbedding.buildcCs|js dSt|dS)Nr)rr not_equal)rinputsmaskr r r! compute_maskszEmbedding.compute_maskcCs|jdkr||jfSt|jttfr2t|j}n|jg}t|t|dkrjtdt|jt|fnlxjtt ||ddD]P\}\}}|dk r|dk r||krtdt|jt|fq|dkr|||<qW|dft||jfSdS)Nz5"input_length" is %s, but received input has shape %sr) rr isinstancelisttuplelenrstr enumeratezip)rrZin_lensis1s2r r r!compute_output_shapes"   $ zEmbedding.compute_output_shapecCsXt|}|dkr&|dkr&t|d}t|j|}|jj|jj krTt||jj}|S)NZint32Zint64) rrr castr Zembedding_lookup_v2r"Z _dtype_policyZ compute_dtypeZvariable_dtype)rr(routr r r!calls  zEmbedding.callc sh|j|jt|jt|jt|jt|j |j |j d}t t |}tt|t|S)N)rrrrrrrr)rrr serializerrrrrrrrrr get_configdictr-items)rconfigZ base_config)rr r!r;s     zEmbedding.get_config)r NNNFN)N)N) __name__ __module__ __qualname____doc__rrZshape_type_conversionr&r*r6r9r; __classcell__r r )rr!r sJ!  r N)rBZtensorflow.python.kerasrrrrZtensorflow.python.keras.enginerZ)tensorflow.python.keras.engine.base_layerrZtensorflow.python.keras.utilsrZtensorflow.python.opsr r Z tensorflow.python.util.tf_exportr r r r r r!s