DateTimeGenerator

Generates datetimes for keys and variables with fixed or random values, strict min/max bounds, optional weighted sampling, native offset arithmetic, deterministic seeding, and epoch conversions. This document covers capabilities, API and usage for EE.

Modes

• Custom value: value='<date string>' parsed with input_format.
• Random window: random=True (or random_mode=True) with optional min/max.
• Current time: omit the above → uses the current system time.
• Offset arithmetic: offset=... with optional max_offset and reference.

Defaults when no bounds are provided: start=1970-01-01 00:00:00, end=2020-01-01 00:00:00 (50-year span).

Core Parameters

• min / max (str): Optional bounds parsed using input_format (default %Y-%m-%d %H:%M:%S).
• value (str): Fixed datetime value using input_format.
• random / random_mode (bool): Enable random sampling.
• input_format (str): Parse format for min, max, value.
• seed (int): Deterministic RNG seed; same seed + same params → same sequence.

Offset Arithmetic

• offset (str): Relative duration applied to the base datetime.
• max_offset (str): Optional upper bound for a sampled offset window.
• reference (str): Optional lookup-only reference resolved against the current record. Supported forms are bare identifiers like issue_at_raw, this.<field>, this.parent.<field>, and root.<field>.

Offset examples:

• 30d
• -2h
• 1y3mo
• 2w3d
• 1h30m

Offset contract:

• offset only: resolve one datetime relative to the base.
• offset + max_offset: sample uniformly between the two resolved datetimes.
• reference omitted: use the injected setup clock/current datetime as the base.
• reference='issue_at_raw': resolve against the current scope before each generate() call.
• reference='this.parent.issued_at': resolve against the direct parent frame in nested generates.
• reference='root.issued_at': resolve against the flattened root frame.
• Bare parent.<field>, calls, subscripts, and arbitrary expressions are rejected as invalid datetime references.

Examples:

<key name="expires_at" generator="DateTimeGenerator(offset='30d')"/>
<key name="recent_event" generator="DateTimeGenerator(offset='-2h')"/>
<key name="future_window" generator="DateTimeGenerator(offset='1d', max_offset='90d')"/>
<key name="due_at" generator="DateTimeGenerator(reference='issue_at_raw', offset='30d')" outDateFormat="%Y-%m-%d"/>

Nested lookup examples:

<key name="due_at" generator="DateTimeGenerator(reference='this.parent.issued_at', offset='30d')" outDateFormat="%Y-%m-%d"/>
<key name="renewal_at" generator="DateTimeGenerator(reference='root.created_at', offset='1y')" outDateFormat="%Y-%m-%d"/>

For broader nested lookup rules in script=... expressions, see Variable Scoping in Nested Generates. DateTimeGenerator(reference=...) intentionally supports only the lookup-only subset above.

Raw Datetime Rule

Datetime arithmetic is raw-only.

• Use variables or unformatted datetime values for downstream arithmetic.
• Once outDateFormat has been applied, the downstream value is a string.
• DATAMIMIC does not auto-parse formatted strings back into datetime values.
• If you accidentally call datetime helper methods on a formatted value, the runtime raises a typed script error and points back to the raw-value rule.

Preferred pattern:

<variable name="issue_at_raw" generator="DateTimeGenerator(value='2024-01-15 10:30:00')"/>
<key name="issue_date" script="issue_at_raw" outDateFormat="%Y-%m-%d"/>
<key name="due_date" script="issue_at_raw.add_days(30)" outDateFormat="%Y-%m-%d"/>

Bounds are respected down to the second. When the sampled day equals the min/max day, hour/minute/second are clamped into the legal sub‑range.

Time Weights (intra‑day)

• hour_weights (list[24] of float)
• minute_weights (list[60] of float)
• second_weights (list[60] of float)

Rules: - Length must match (24/60/60). Values non-negative and sum > 0 when provided. - If omitted and no other day-level weighting is used, distribution within the day is uniform. - Generator constructor parsing preserves quoted weight expressions and sugar strings as written, so declarations such as month_weights='[0,0,1]+[0]*9', minute_weights='[1 if m in (0,15,30,45) else 0 for m in range(60)]', and months='3,7-9' remain valid.

Day Selection Weights (inter‑day)

• month_weights (list[12] of float)
• weekday_weights (list[7] of float) – Monday=0, Sunday=6
• dom_weights (list[31] of float) – Day‑of‑month 1..31

How it works: - The generator enumerates all eligible days in the [min, max] window. - Per‑day weight = month_weight × weekday_weight × dom_weight. - A month is drawn proportional to the sum of its days’ weights; then a day is drawn using that month’s day weights.

Validation: - Correct lengths, non‑negative numbers, and non‑zero total weight are enforced. - If no day is eligible → clear ValueError.

Sugar (friendly shortcuts)

Only applies when explicit weights for that dimension are not provided.

• months / months_exclude: restrict months.
• Examples: months='3,7-9', months_exclude='2,4'.
• Quarters: months='Q1' (Jan–Mar), months='Q2,Q4' (Apr–Jun, Oct–Dec).
• weekdays: names, ranges, presets (Mon=0, Sun=6).
• Names/ranges: weekdays='Mon-Fri', weekdays='Fri-Mon' (wrap‑around supported).
• Presets: weekdays='Weekend', weekdays='Weekdays'/Business/Workdays, weekdays='All'.
• dom: day‑of‑month filter.
• Examples: dom='1-5,15,31', dom='last' (last day of each month).
• hours_preset: office (09–17 high), night (22–05 high), flat (no bias).
• minute_granularity / second_granularity (int): keep only ticks divisible by granularity (e.g., 15 → 0,15,30,45).

Precedence: explicit weights (*_weights) override sugar for that dimension.

Epoch Support

There are two ways to work with epoch time:

1) Via key formatting (preferred in XML): - inDateFormat="epoch" converts a constant epoch (seconds/millis/micros/nanos) to datetime. - outDateFormat="epoch|epoch_millis|epoch_micros|epoch_nanos|%Nf" converts a datetime to the requested format.

2) Via Python API (generator options): - as_epoch=True with epoch_unit='seconds'|'milliseconds' returns integer epoch from .generate(); .generate_date() returns date based on the epoch.

Determinism & Performance

• Deterministic sampling uses a private RNG with seed.
• Fast path (Rust) is used when there is no seed and no weights/sugar. A small buffer is prefilled for throughput.
• Runtime flags: rust_fast_generators (default true), rust_fast_buffer_size (default 8192).
• Weighted/sugar/deterministic cases run in Python with O(1) per sample after a one‑time day distribution build.

Error Handling

• Invalid lengths, negative values, or all‑zero weights → ValueError.
• Offset durations that would exceed Python's supported datetime range → ValueError.
• Sugar that yields no eligible days in the window (e.g., months='2' over a January range) → ValueError.
• Invalid granularity (≤0 or >60) → ValueError.
• Unknown hours_preset → ValueError.

XML Examples

Formatting and epoch conversions:

<generate name="InOutDatetimeFormat" count="10">
  <key name="datetime_with_in" constant="2021-02-01 12:08:04.376188" inDateFormat="%Y-%m-%d %H:%M:%S.%f"/>
  <key name="datetime_with_out" generator="DateTimeGenerator" outDateFormat="%d.%m.%Y %H:%M:%S.%f"/>
  <key name="date_with_in_out" constant="2022-02-01" inDateFormat="%Y-%m-%d" outDateFormat="%d.%m.%Y"/>
  <key name="day_out" generator="DateTimeGenerator" outDateFormat="%d"/>
  <key name="epoch_in_seconds" constant="1692967891" inDateFormat="epoch" outDateFormat="%Y-%m-%d %H:%M:%S"/>
  <key name="to_epoch" generator="DateTimeGenerator" outDateFormat="epoch"/>
  <key name="to_epoch_millis" generator="DateTimeGenerator" outDateFormat="epoch_millis"/>
</generate>

Uniform window:

<setup>
  <generate name="uniform" count="3">
    <key name="dt" generator="DateTimeGenerator(min='2024-01-01 00:00:00', max='2024-12-31 23:59:59', random=True)"/>
  </generate>
</setup>

Time sugar and granularity:

<key name="office_15_10"
     generator="DateTimeGenerator(min='2024-04-01 00:00:00', max='2024-04-30 23:59:59', random=True,
                                   hours_preset='office', minute_granularity=15, second_granularity=10, seed=7)"/>

Weighted month/weekday/dom selection:

<key name="march_mondays_dom4"
     generator="DateTimeGenerator(min='2024-01-01 00:00:00', max='2024-12-31 23:59:59', random=True,
                                   month_weights='[0,0,1]+[0]*9',
                                   weekday_weights='[1,0,0,0,0,0,0]',
                                   dom_weights='[0,0,0,1]+[0]*27', seed=7)"/>

Sugar filters:

<key name="biz_days" generator="DateTimeGenerator(min='2024-04-01 00:00:00', max='2024-04-30 23:59:59', random=True, weekdays='Mon-Fri')"/>
<key name="q1_only"  generator="DateTimeGenerator(min='2024-01-01 00:00:00', max='2024-12-31 23:59:59', random=True, months='Q1')"/>
<key name="dom_last" generator="DateTimeGenerator(min='2024-01-01 00:00:00', max='2024-12-31 23:59:59', random=True, dom='last')"/>

Offset and reference arithmetic:

<setup>
  <generate name="invoice" count="1">
    <variable name="issue_at_raw" generator="DateTimeGenerator(value='2024-01-15 10:30:00')"/>
    <key name="issue_date" script="issue_at_raw" outDateFormat="%Y-%m-%d"/>
    <key name="due_date" script="issue_at_raw.add_days(30)" outDateFormat="%Y-%m-%d"/>
    <key name="due_at_generator" generator="DateTimeGenerator(reference='issue_at_raw', offset='30d')" outDateFormat="%Y-%m-%d"/>
  </generate>
</setup>

Biased intra‑day and sampled seconds/minutes:

<key name="biased"
     generator="DateTimeGenerator(
       min='2020-02-01 0:0:0', max='2025-07-31 0:0:0',
       month_weights='[0.5] * 2 + [1] * 3 + [0.8] * 3 + [1] * 3 + [0.5]',
       hour_weights='[0.6]*6 + [0.3]*16 + [0.1]*2',
       minute_weights='[1 if m in (0,15,30,45) else 0 for m in range(60)]',
       second_weights='[1]+[0]*59', seed=123
     )"/>

Notes on XML weight expressions: - Due to XML parsing, pass list expressions as strings (quote the expression) when they contain commas. - Simple patterns like [0]*12 work quoted or unquoted. Complex list comprehensions must be quoted, as above.

Python API Examples

from datamimic_ee.domains.common.literal_generators.datetime_generator import DateTimeGenerator

# Deterministic weekends in January 2024
gen = DateTimeGenerator(
    min='2024-01-01 00:00:00', max='2024-01-31 23:59:59', random=True,
    weekdays='Weekend', seed=42,
)
print(gen.generate())

# Time weights only (uniform dates)
gen = DateTimeGenerator(random=True, hour_weights=[0.2]*6 + [0.02]*18, seed=1)
print(gen.generate())

# Epoch via Python API
gen = DateTimeGenerator(random=True, as_epoch=True, epoch_unit='milliseconds')
print(gen.generate())  # -> int millis

# Relative window from a fixed base
gen = DateTimeGenerator(
    base_datetime=datetime(2024, 1, 15, 10, 30, 0),
    offset="1d",
    max_offset="3d",
    seed=7,
)
print(gen.generate())

Edge Cases & Tips

• Leap years: DOM weights and dom='last' handle February 29 correctly.
• Weekday wrap: ranges like Fri-Mon wrap across week boundaries.
• Bounds + granularity: When granularity excludes most ticks, sampling remains within the min/max time range of the chosen day.
• Precedence: Explicit weights override sugar. Combine month/weekday/dom carefully to avoid eliminating all days.
• random_mode is an alias of random for compatibility.