B ٻd@sdZddlmZddlmZddlmZddlmZddlmZddl m Z ddl m Z dd l mZed e d d Gd ddZd S)z$Python wrappers for tf.data writers.) dataset_ops)convert)dtypes)ops) tensor_spec)gen_experimental_dataset_ops) deprecation) tf_exportz data.experimental.TFRecordWriterNzTo write TFRecords to disk, use `tf.io.TFRecordWriter`. To save and load the contents of a dataset, use `tf.data.experimental.save` and `tf.data.experimental.load`c@s"eZdZdZdddZddZdS)TFRecordWriteraWrites a dataset to a TFRecord file. The elements of the dataset must be scalar strings. To serialize dataset elements as strings, you can use the `tf.io.serialize_tensor` function. ```python dataset = tf.data.Dataset.range(3) dataset = dataset.map(tf.io.serialize_tensor) writer = tf.data.experimental.TFRecordWriter("/path/to/file.tfrecord") writer.write(dataset) ``` To read back the elements, use `TFRecordDataset`. ```python dataset = tf.data.TFRecordDataset("/path/to/file.tfrecord") dataset = dataset.map(lambda x: tf.io.parse_tensor(x, tf.int64)) ``` To shard a `dataset` across multiple TFRecord files: ```python dataset = ... # dataset to be written def reduce_func(key, dataset): filename = tf.strings.join([PATH_PREFIX, tf.strings.as_string(key)]) writer = tf.data.experimental.TFRecordWriter(filename) writer.write(dataset.map(lambda _, x: x)) return tf.data.Dataset.from_tensors(filename) dataset = dataset.enumerate() dataset = dataset.apply(tf.data.experimental.group_by_window( lambda i, _: i % NUM_SHARDS, reduce_func, tf.int64.max )) # Iterate through the dataset to trigger data writing. for _ in dataset: pass ``` NcCs.tj|tjdd|_tjd|dtjd|_dS)a^Initializes a `TFRecordWriter`. Args: filename: a string path indicating where to write the TFRecord data. compression_type: (Optional.) a string indicating what type of compression to use when writing the file. See `tf.io.TFRecordCompressionType` for what types of compression are available. Defaults to `None`. filename)namecompression_type)argument_defaultZargument_dtypeN)rZconvert_to_tensorrstring _filenamerZoptional_param_to_tensor_compression_type)selfr r ra/var/www/html/venv/lib/python3.7/site-packages/tensorflow/python/data/experimental/ops/writers.py__init__Is zTFRecordWriter.__init__cCsxt|tjs tdt|dt|tgt j s\tdt |dt |d| }t|j|j|jS)aWrites a dataset to a TFRecord file. An operation that writes the content of the specified dataset to the file specified in the constructor. If the file exists, it will be overwritten. Args: dataset: a `tf.data.Dataset` whose elements are to be written to a file Returns: In graph mode, this returns an operation which when executed performs the write. In eager mode, the write is performed by the method itself and there is no return value. Raises TypeError: if `dataset` is not a `tf.data.Dataset`. TypeError: if the elements produced by the dataset are not scalar strings. z?Invalid `dataset.` Expected a `tf.data.Dataset` object but got .zInvalid `dataset`. Expected a`dataset` that produces scalar `tf.string` elements, but got a dataset which produces elements with shapes z and types ) isinstancerZ DatasetV2 TypeErrortypeZ get_structureZis_compatible_withrZ TensorSpecrrZget_legacy_output_shapesZget_legacy_output_typesZ_apply_debug_optionsrZdataset_to_tf_recordZ_variant_tensorrr)rZdatasetrrrwriteZs   zTFRecordWriter.write)N)__name__ __module__ __qualname____doc__rrrrrrr s- r )rZtensorflow.python.data.opsrZtensorflow.python.data.utilrZtensorflow.python.frameworkrrrZtensorflow.python.opsrZtensorflow.python.utilrZ tensorflow.python.util.tf_exportr deprecatedr rrrrs