Element <iterate>

Purpose: Traverses records from a required source and exposes each current row to child contexts without owning an output target.

Why: Use iterate to traverse a source row and make it available to child contexts without exporting it directly.

Example

<generate name="customers" source="data/customers.ent.csv"/>

Decision guide

Business value: Makes source traversal explicit while child contexts consume the current source row.

• Use when

  • Child elements need fields from each current source row through their parent context.

• Choose another approach when

  • The operation creates or exports a product directly.

• Prerequisites

  • Provide a source and an explicit bound where the source contract requires one.

• Alternatives

  • Use generate when the operation owns an output target or creates a product. (See: <generate>)

Complete examples

Traverse an in-memory product and export an enriched child product

Use iterate when child contexts consume each current source row; let a nested generate own the enriched output product and target.

<setup>
    <generate name="source_rows" count="4" target="mem">
        <key name="n" generator="IncrementGenerator"/>
    </generate>
    <iterate name="source_row"
             source="mem"
             type="source_rows"
             count="4"
             distribution="ordered">
        <generate name="enriched_rows" count="1" target="LogExporter">
            <key name="n" script="parent.n"/>
            <key name="doubled" script="n * 2"/>
            <nestedKey name="metrics" type="dict">
                <variable name="n" script="parent.n"/>
                <key name="squared" script="n * n"/>
            </nestedKey>
        </generate>
    </iterate>
</setup>

Rules and invalid combinations

Source-selection options require source.

Attributes: source, selector, separator, sourceScripted, cyclic, weightColumn, stratifyBy

Why: selector, separator, sourceScripted, cyclic, weightColumn, and stratifyBy only modify a source read.

<generate name="customers" source="data/customers.ent.csv" count="10"/>

<generate name="customers" count="10" selector="select * from customers"/>

A source may specify type or selector, but not both.

Attributes: source, type, selector

Why: selector owns the query shape, while type identifies an unselected source entity.

<generate name="customers" source="customerDb" type="Customer" count="10"/>

<generate name="customers" source="customerDb" type="Customer" selector="select * from customers" count="10"/>

Use count or the randomized minCount/maxCount strategy, never both.

Attributes: count, minCount, maxCount

Why: Both strategies own the number of rows, so combining them would make execution ambiguous.

<generate name="customers" count="10"/>

<generate name="customers" count="10" minCount="2"/>

minCount must be less than or equal to maxCount.

Attributes: minCount, maxCount

Why: The randomized count is drawn from the inclusive interval between the two bounds.

<generate name="customers" minCount="2" maxCount="10"/>

<generate name="customers" minCount="10" maxCount="2"/>

minCount/maxCount cannot be combined with start/end/interval.

Attributes: minCount, maxCount, start, end, interval

Why: Time-series length is derived from its window and interval, not from a randomized row count.

<generate name="events" start="2026-01-01T00:00:00Z" end="2026-01-02T00:00:00Z" interval="PT1H"/>

<generate name="events" minCount="2" start="2026-01-01T00:00:00Z" end="2026-01-02T00:00:00Z" interval="PT1H"/>

offset requires source.

Attributes: offset, source

Why: An offset skips source rows and therefore has no meaning for purely synthetic generation.

<generate name="customers" source="data/customers.ent.csv" offset="2" count="10"/>

<generate name="customers" count="10" offset="2"/>

resume='group' requires resumeGroup.

Attributes: resume, resumeGroup

Why: The runtime needs the shared group name to resolve continuation state.

<generate name="customers" count="10" resume="group" resumeGroup="customer-import"/>

<generate name="customers" count="10" resume="group"/>

resumeGroup requires resume='group'.

Attributes: resume, resumeGroup

Why: The group name has no meaning for statement- or table-scoped continuation.

<generate name="customers" count="10" resume="group" resumeGroup="customer-import"/>

<generate name="customers" count="10" resume="stmt" resumeGroup="customer-import"/>

unique requires a finite source.

Attributes: unique, source

Why: Distinct sampling without replacement needs a source pool.

<generate name="customers" source="data/customers.ent.csv" unique="true" count="10"/>

<generate name="customers" count="10" unique="true"/>

unique cannot be combined with cyclic.

Attributes: unique, cyclic

Why: Unique selection consumes without replacement, while cyclic repeats exhausted rows.

<generate name="customers" source="data/customers.ent.csv" unique="true" count="10"/>

<generate name="customers" source="data/customers.ent.csv" count="10" unique="true" cyclic="true"/>

unique combines only with an explicit distribution='random'.

Attributes: unique, distribution

Why: Ordered or weighted selection would conflict with unique random sampling without replacement.

<generate name="customers" source="data/customers.ent.csv" unique="true" distribution="random" count="10"/>

<generate name="customers" source="data/customers.ent.csv" count="10" unique="true" distribution="ordered"/>

cyclic requires a distribution that supports repetition.

Attributes: cyclic, distribution

Why: Full-pool weighted and stratified draws do not define a stable exhaustion boundary to restart.

<generate name="customers" source="data/customers.ent.csv" cyclic="true" distribution="ordered" count="10"/>

<generate name="customers" source="data/customers.ent.csv" count="10" cyclic="true" distribution="weighted" weightColumn="weight"/>

distribution='weighted' requires weightColumn.

Attributes: distribution, weightColumn

Why: The selector needs a non-negative numeric column to calculate sampling probabilities.

<generate name="customers" source="data/customers.ent.csv" distribution="weighted" weightColumn="weight" count="10"/>

<generate name="customers" source="data/customers.ent.csv" count="10" distribution="weighted"/>

distribution='stratified' requires stratifyBy.

Attributes: distribution, stratifyBy

Why: The selector needs a source column that identifies each stratum.

<generate name="customers" source="data/customers.ent.csv" distribution="stratified" stratifyBy="segment" count="10"/>

<generate name="customers" source="data/customers.ent.csv" count="10" distribution="stratified"/>

A semantic project-file source requires its catalogued metadata attribute.

Attributes: source, weightColumn

Why: Weighted entity rows need weightColumn so selection can consume and then remove the metadata column.

<generate name="customers" source="data/customers.wgt.ent.csv" distribution="weighted" weightColumn="weight" count="10"/>

<generate name="customers" source="data/customers.wgt.ent.csv" count="10"/>

A semantic project-file source requires its catalogued distribution.

Attributes: source, distribution

Why: The file suffix declares selection semantics that an explicit conflicting distribution cannot override.

<generate name="customers" source="data/customers.wgt.ent.csv" distribution="weighted" weightColumn="weight" count="10"/>

<generate name="customers" source="data/customers.wgt.ent.csv" count="10" weightColumn="weight" distribution="ordered"/>

A weighted entity project source requires explicit count.

Attributes: source, count

Why: Weighted selection does not infer a bounded output size from source length.

<generate name="customers" source="data/customers.wgt.ent.csv" distribution="weighted" weightColumn="weight" count="10"/>

<generate name="customers" source="data/customers.wgt.ent.csv" weightColumn="weight"/>

W004 — Deprecated XML Attribute Ignored

<{element}> attribute '{attribute}' in descriptor '{descriptor}' is deprecated and ignored; execution continues. {migration}

Why: The descriptor uses a known legacy attribute that no longer controls execution.

Resolution: Follow the migration hint when updating the model. Removing the attribute is not required to run it.

Full rule

I883 — Time-Series Bad Start

Invalid time-series configuration: {detail}

Why: The 'start' attribute is not a valid ISO 8601 datetime.

Resolution: Set start to an ISO 8601 datetime, e.g. '2026-01-01T00:00:00Z'.

Full rule

I884 — Time-Series Bad End

Invalid time-series configuration: {detail}

Why: The 'end' attribute is not a valid ISO 8601 datetime.

Resolution: Set end to an ISO 8601 datetime, e.g. '2026-01-02T00:00:00Z'.

Full rule

I885 — Time-Series Bad Interval

Invalid time-series configuration: {detail}

Why: The 'interval' attribute is not a valid ISO 8601 duration.

Resolution: Set interval to an ISO 8601 duration, e.g. 'PT1H'.

Full rule

I886 — Time-Series End Not After Start

Invalid time-series configuration: {detail}

Why: The 'end' attribute is not strictly after the 'start' attribute.

Resolution: Set end to a datetime strictly after start.

Full rule

I887 — Time-Series Interval Too Fine

Invalid time-series configuration: {detail}

Why: The 'interval' duration is non-positive, sub-microsecond, or a months/years Duration.

Resolution: Set interval to a positive constant-length duration of at least 1us, e.g. 'PT1S'.

Full rule

I888 — Time-Series Namespace Collision

is not allowed inside a time-series : 'ts' is reserved for the time-iterator namespace (ts.now/ts.step/ts.series). Rename the variable, e.g. 'ts_meta'.

Why: A would shadow the reserved time-iterator namespace.

Resolution: Rename the variable to something other than 'ts', e.g. 'ts_meta'.

Full rule

I889 — Time-Series Incomplete Config

Time-series attributes start/end/interval must be set together; missing: {missing}

Why: Only some of the time-series attributes start/end/interval were provided.

Resolution: Provide all three attributes (start, end, interval) or none of them.

Full rule

I949 — Source ML Model Option Unsupported

ML model source '{source}' does not support {option}={value}. Supported behavior: {supported}.

Why: The requested source-selection option cannot be preserved by generated ml:// model samples.

Resolution: Remove the unsupported option and use bounded ordered ML model generation.

Full rule

I192 — Source Reference Identifier Empty

Explicit source URI for family '{family}' requires a non-empty identifier

Why: A recognized source-family URI was declared without the identifier needed to resolve its source.

Resolution: Add the source identifier after the URI scheme and retry.

Full rule

I194 — Source Reference Client Family Mismatch

Explicit source family '{family}' does not match configured client '{client_id}' of type '{actual_client_type}'

Why: The referenced client exists but does not implement the source family declared by the URI.

Resolution: Use the URI scheme matching the configured client or reference a client of the declared family.

Full rule

I195 — Source Reference Identifier Conflict

Explicit source identifier '{identifier}' conflicts with {attribute}='{configured_identifier}'

Why: Two source attributes select different identities for the same explicit source reference.

Resolution: Remove the legacy override or make it equal to the identifier in source.

Full rule

Allowed parents / Allowed children

Allowed parents: else, else-if, generate, if, iterate, setup, while

Allowed children:

array, assert, condition, echo, generate, id, include, iterate, key, list, mapping, nestedKey, reference, rule, sourceConstraints, targetConstraints, variable, while

Related concepts and use cases

• First DATAMIMIC Model — Builds the smallest model and then composes preparation and generation stages.
• Migrate Benerator Models to DATAMIMIC — Maps Benerator concepts to the current generated grammar without freezing another support matrix.
• Choose a source and target — Guides users and agents from an input ownership boundary and intended side effect to one bounded source and target contract.
• Assemble a complex deterministic model — Provides the requirement-to-model sequence for products, field dependencies, relationships, targets, and acceptance evidence.
• Determinism, parallelism, paging, and distribution — Separates seed, distribution, worker policy, paging, and streaming so topology choices do not become false determinism claims.

Attributes

Show all 46 attributes

bucket

Bucket name for the external source or target.

optional; string; Default: null.

container

Container name for the external source or target.

optional; string; Default: null.

converter

Converter for element data transformation.

optional; string; Default: null.

count

Number of records to generate.

optional; string; Default: null.

cyclic

Enable or disable cyclic generation.

optional; boolean; Default: null.

device

Computation device to use.

optional; string; Default: null.

distribution

Distribution type for data generation.

optional; string; Default: null; Values: ordered, random, weighted, stratified, round_robin, reservoir, cumulated.

encoding

Override encoding for this generate task.

optional; string; Default: null.

end

Time-series window end (ISO 8601 datetime).

optional; string; Default: null.

fairness

Fairness configuration (JSON).

optional; string; Default: null.

generationBatchSize

Batch size during generation.

optional; integer; Default: null.

imputation

Imputation configuration (JSON).

optional; string; Default: null.

interval

Time-series tick interval (ISO 8601 duration).

optional; string; Default: null.

iterationSelector

Selector evaluated per iteration.

optional; string; Default: null.

maxCount

Maximum count for randomized generate/iterate length (mutually exclusive with 'count').

optional; integer; Default: null.

minCount

Minimum count for randomized generate/iterate length (mutually exclusive with 'count').

optional; integer; Default: null.

mpPlatform

Multiprocessing platform override.

optional; string; Default: null; Values: multiprocessing, fork, spawn, forkserver.

multiprocessing

Deprecated: accepted with a warning and ignored. Use numProcess and mpPlatform instead.

optional; string; Default: null.

name

Name of the generation task.

required; string.

numProcess

Specify the number of processes to use.

optional; integer; Default: null.

offset

Skip the first N rows of the source.

optional; integer; Default: null.

pageSize

Page size for processing data.

optional; integer; Default: null.

page_bytes_cap

Hard cap on page size in bytes.

optional; integer; Default: null.

page_memory_cap_mb

Cap in megabytes for page memory usage.

optional; integer; Default: null.

rareCategoryReplacementMethod

Method for handling rare categories.

optional; string; Default: null; Values: constant, sample.

rebalancing

Class/feature rebalancing configuration (JSON).

optional; string; Default: null.

resume

Optional per-statement resume scope override.

optional; string; Default: null; Values: stmt, table, group.

resumeGroup

Logical resume group key used when resume='group'.

optional; string; Default: null.

samplingTemperature

Sampling temperature used for generation (0-2).

optional; number; Default: null.

samplingTopP

Top-p (nucleus) sampling threshold (0-1).

optional; number; Default: null.

script

Script driving generation logic.

optional; string; Default: null.

selector

Selector for data generation.

optional; string; Default: null.

separator

Separator for generated data.

optional; string; Default: null.

source

Canonical source URI (for example file://data/orders.csv, database://sourceDb, or ml://customer_model); legacy raw source values remain accepted.

required; string; Default: null.

sourceClient

Override the client used to read when source is ambiguous.

optional; string; Default: null.

sourceEntity

Explicit physical entity to read (table/collection/product). Overrides selector or inferred name.

optional; string; Default: null.

sourceScripted

Enable or disable scripted sources.

optional; boolean; Default: null.

sourceUri

URI backing the source (file/object storage).

optional; string; Default: null.

start

Time-series window start (ISO 8601 datetime).

optional; string; Default: null.

storageId

Object storage client id.

optional; string; Default: null.

stratifyBy

Stratum column for distribution='stratified' source generation.

optional; string; Default: null.

type

Type of data generation.

optional; string; Default: null.

unique

Emit each source row at most once (distinct selection without replacement).

optional; boolean; Default: null.

variablePrefix

Prefix before field's name for query select data in selector element

optional; string; Default: null.

variableSuffix

Suffix after field's name for query select data in selector element

optional; string; Default: null.

weightColumn

Weight column for distribution='weighted' source generation; required for a .wgt.ent.csv source.

optional; string; Default: null.