Model relationships and correlated data

A relationship states where a related record is owned and how long it remains available. Choose the construct by the required output shape and lifetime, not by similar-looking syntax.

Intent	Use	Result
Separate child rows per parent	nested <generate>	A child product with explicit per-parent cardinality
Embedded objects or lists inside one row	<nestedKey>	Structured data inside the current product; no separate child product
Copy one or several values from one existing source row	<reference>	One lookup event whose mapped fields remain correlated
Reuse an earlier product in a later top-level stage	memstore target and source	Run-local product lineage across separate generation stages

Parent and child products

Nest <generate> when every parent row owns a bounded number of separate child rows. The child count is evaluated per immediate parent, and parent.<field> copies a value from that parent into the child row.

<setup rngSeed="42">
    <generate name="customers" count="4">
        <key name="id" type="int" min="1" max="99" distribution="shuffle"/>
        <key name="region" values="('north', 'south', 'east', 'west')"/>
        <generate name="orders" count="2">
            <key name="customer_id" script="parent.id"/>
            <key name="amount" type="float" min="10" max="500"/>
        </generate>
    </generate>
</setup>

This produces two orders rows for each customers row. Use this.parent.<field> when the explicit scope improves readability. At deeper nesting levels, each parent still means only the immediate parent; copy every required key deliberately.

Use <nestedKey> instead when the repeated objects must remain embedded in the same output row rather than becoming a separately addressable product.

Correlated reference values

Use one <reference> with <field> children when several output values must come from the same selected source row. Independent references or generators can select different rows and break the correlation.

<setup numProcess="1">
    <generate name="orders" count="2" target="LogExporter">
        <id name="order_id" generator="IncrementGenerator"/>
        <reference name="customer"
                   source="data/customers.wgt.ent.csv"
                   weightColumn="weight">
            <field target="customer_id" sourceKey="id"/>
            <field target="customer_country" sourceKey="country"/>
        </reference>
    </generate>
</setup>

The two mapped fields are selected together, preserving the customer ID/country pair. A reference enriches the current row from an existing relational, MongoDB, or weighted project-file source; it does not create a child product by itself. The complete <reference> specification documents single-field mapping, ordered/cyclic selection, uniqueness, weighting, and supported combinations.

Multi-stage product lineage

Use a memstore target and source when a later top-level product consumes an earlier product from the same run. The producer's targetEntity and the consumer's memstore:// source id must match. This is a stage boundary, not parent scope: later expressions read the current source row through this.<field>. See the validated memstore model.

Review checklist

• The required result is a child product, an embedded value, a lookup, or a later stage—not an accidental mix.
• Parent keys are copied explicitly, and per-parent cardinality is bounded.
• Correlated reference fields share one <reference> selection.
• The source family and selection mode are supported by the referenced element specification.
• Referential integrity and cardinality are asserted in the acceptance checks for the model.

For agent-generated models, the Authoring reference provides validated intent examples and acceptance contracts; the XML above remains the user-facing execution model.