B ٻd$@sdZddlZddlmZddlmZddlmZddlm Z ddlm Z ddlm Z dd l m Zdd d ZGd ddejZdS)z0The implementation of `tf.data.Dataset.rebatch`.N) dataset_ops)nest)dtypes)ops) tensor_shape) tensor_util)gen_experimental_dataset_opsFcCst||||S)N)RebatchDataset) input_datasetZ batch_sizedrop_remaindernamer W/var/www/html/venv/lib/python3.7/site-packages/tensorflow/python/data/ops/rebatch_op.pyrebatchsrcs>eZdZdZd fdd ZddZdd Zed d ZZ S) r adA `Dataset` that rebatches elements from its input into new batch sizes. `RebatchDataset(input_dataset, batch_sizes)` is functionally equivalent to `input_dataset.unbatch().batch(N)`, where the value of N cycles through the `batch_sizes` input list. The elements produced by this dataset have the same rank as the elements of the input dataset. FNcs||_tj|tjdd|_tj|tjdd|_||_| t fddt ||_t |}tj|jf||d|j}tt|||dS)z"See `Dataset.rebatch` for details. batch_sizes)Zdtyper r cs|S)N)Z_unbatchZ_batch)ts) new_batch_dimr r9z)RebatchDataset.__init__..)rr N)_input_datasetrZconvert_to_tensorrZint64 _batch_sizesbool_drop_remainder_name_compute_static_batch_dimrZ map_structurer get_structure _element_specZnormalize_to_denseged_opsZrebatch_dataset_v2Z_variant_tensorZ_flat_structuresuperr __init__)selfr rr r Zvariant_tensor) __class__)rrr)s"   zRebatchDataset.__init__cCst|j}|dkrdSt|tjrxt|jdkrTt||dkrN|d}qxdSn$t|jdkrxt dt|jd| |rdS|S)a0Computes the static batch dimension of a dataset if it can be determined. Given the RebatchDataset parameters, determines the batch dimension of this dataset statically. Returns None if this cannot be determined or is variable. Returns: An integer representing the batch dimension of the dataset. If it cannot be determined statically, returns None. Raises: ValueError: The batch_sizes parameter is malformed, input_dataset is not batched, or input_dataset batch sizes are incompatible with each other. NrziInvalid `batch_sizes`. Expected `batch_sizes` to be a scalar or a vector. Received `batch_sizes` of rank .) rconstant_valuer isinstancenpZndarraylenshapeall ValueError_may_form_partial_batches)r rr r rrIs    z(RebatchDataset._compute_static_batch_dimcst|jrdSddfddtt|jD}dd|D}|sNdSt |}t ||dkst d |dd |d |d|dkS) z6Returns whether this dataset may form partial batches.FcSs`y |}Wntk r dSXt|tjs2dS|jdkr@dSt|dkrTtd|jdj S)Nr"zInvalid `batch_sizes`. Expected dataset with rank of >= 1 but found a dataset with scalar elements. Fix the issue by adding the `batch` transformation to the dataset.r) Z_to_legacy_output_shapesNotImplementedErrorr%rZ TensorShapeZrankr'r*Zdimsvalue)Z type_specr(r r r get_batch_dimss    z?RebatchDataset._may_form_partial_batches..get_batch_dimcsg|] }|qSr r ).0r)r.r r sz.cSsg|]}|dk r|qS)Nr )r/dr r rr0sTrz?Invalid `input_dataset.` The batch dimension of component 0 is z., while the batch dimension of component i is r#) rr$rrflattenrrrr&Zasarrayr)r*)r Zdesired_batch_sizeZinput_batch_dimsZknown_input_batch_dimsr )r.rr+ns   z(RebatchDataset._may_form_partial_batchescCs|jS)N)r)r r r r element_specszRebatchDataset.element_spec)FN) __name__ __module__ __qualname____doc__rrr+propertyr3 __classcell__r r )r!rr s %'r )FN)r7numpyr&Ztensorflow.python.data.opsrZtensorflow.python.data.utilrZtensorflow.python.frameworkrrrrZtensorflow.python.opsrrrZ UnaryDatasetr r r r rs