Skip to content

DateTimeGenerator

Generates datetimes for keys and variables with fixed or random values, strict min/max bounds, optional weighted sampling, sugar filters, deterministic seeding, and epoch conversions. This document covers capabilities, API and usage for EE.

Modes

  • Custom value: value='<date string>' parsed with input_format.
  • Random window: random=True (or random_mode=True) with optional min/max.
  • Current time: omit the above → uses the current system time.

Defaults when no bounds are provided: start=1970-01-01 00:00:00, end=2020-01-01 00:00:00 (50-year span).

Core Parameters

  • min / max (str): Optional bounds parsed using input_format (default %Y-%m-%d %H:%M:%S).
  • value (str): Fixed datetime value using input_format.
  • random / random_mode (bool): Enable random sampling.
  • input_format (str): Parse format for min, max, value.
  • seed (int): Deterministic RNG seed; same seed + same params → same sequence.

Bounds are respected down to the second. When the sampled day equals the min/max day, hour/minute/second are clamped into the legal sub‑range.

Time Weights (intra‑day)

  • hour_weights (list[24] of float)
  • minute_weights (list[60] of float)
  • second_weights (list[60] of float)

Rules: - Length must match (24/60/60). Values non‑negative and sum > 0 when provided. - If omitted and no other day‑level weighting is used, distribution within the day is uniform.

Day Selection Weights (inter‑day)

  • month_weights (list[12] of float)
  • weekday_weights (list[7] of float) – Monday=0, Sunday=6
  • dom_weights (list[31] of float) – Day‑of‑month 1..31

How it works: - The generator enumerates all eligible days in the [min, max] window. - Per‑day weight = month_weight × weekday_weight × dom_weight. - A month is drawn proportional to the sum of its days’ weights; then a day is drawn using that month’s day weights.

Validation: - Correct lengths, non‑negative numbers, and non‑zero total weight are enforced. - If no day is eligible → clear ValueError.

Sugar (friendly shortcuts)

Only applies when explicit weights for that dimension are not provided.

  • months / months_exclude: restrict months.
  • Examples: months='3,7-9', months_exclude='2,4'.
  • Quarters: months='Q1' (Jan–Mar), months='Q2,Q4' (Apr–Jun, Oct–Dec).
  • weekdays: names, ranges, presets (Mon=0, Sun=6).
  • Names/ranges: weekdays='Mon-Fri', weekdays='Fri-Mon' (wrap‑around supported).
  • Presets: weekdays='Weekend', weekdays='Weekdays'/Business/Workdays, weekdays='All'.
  • dom: day‑of‑month filter.
  • Examples: dom='1-5,15,31', dom='last' (last day of each month).
  • hours_preset: office (09–17 high), night (22–05 high), flat (no bias).
  • minute_granularity / second_granularity (int): keep only ticks divisible by granularity (e.g., 15 → 0,15,30,45).

Precedence: explicit weights (*_weights) override sugar for that dimension.

Epoch Support

There are two ways to work with epoch time:

1) Via key formatting (preferred in XML): - inDateFormat="epoch" converts a constant epoch (seconds/millis/micros/nanos) to datetime. - outDateFormat="epoch|epoch_millis|epoch_micros|epoch_nanos|%Nf" converts a datetime to the requested format.

2) Via Python API (generator options): - as_epoch=True with epoch_unit='seconds'|'milliseconds' returns integer epoch from .generate(); .generate_date() returns date based on the epoch.

Determinism & Performance

  • Deterministic sampling uses a private RNG with seed.
  • Fast path (Rust) is used when there is no seed and no weights/sugar. A small buffer is prefilled for throughput.
  • Runtime flags: rust_fast_generators (default true), rust_fast_buffer_size (default 8192).
  • Weighted/sugar/deterministic cases run in Python with O(1) per sample after a one‑time day distribution build.

Error Handling

  • Invalid lengths, negative values, or all‑zero weights → ValueError.
  • Sugar that yields no eligible days in the window (e.g., months='2' over a January range) → ValueError.
  • Invalid granularity (≤0 or >60) → ValueError.
  • Unknown hours_preset → ValueError.

XML Examples

Formatting and epoch conversions:

1
2
3
4
5
6
7
8
<generate name="InOutDatetimeFormat" count="10">
  <key name="datetime_with_in" constant="2021-02-01 12:08:04.376188" inDateFormat="%Y-%m-%d %H:%M:%S.%f"/>
  <key name="datetime_with_out" generator="DateTimeGenerator" outDateFormat="%d.%m.%Y %H:%M:%S.%f"/>
  <key name="date_with_in_out" constant="2022-02-01" inDateFormat="%Y-%m-%d" outDateFormat="%d.%m.%Y"/>
  <key name="day_out" generator="DateTimeGenerator" outDateFormat="%d"/>
  <key name="epoch_in_seconds" constant="1692967891" inDateFormat="epoch" outDateFormat="%Y-%m-%d %H:%M:%S"/>
  <key name="to_epoch" generator="DateTimeGenerator" outDateFormat="epoch"/>
  <key name="to_epoch_millis" generator="DateTimeGenerator" outDateFormat="epoch_millis"/>

Uniform window:

1
2
3
4
5
<setup>
  <generate name="uniform" count="3">
    <key name="dt" generator="DateTimeGenerator(min='2024-01-01 00:00:00', max='2024-12-31 23:59:59', random=True)"/>
  </generate>
</setup>

Time sugar and granularity:

1
2
3
<key name="office_15_10"
     generator="DateTimeGenerator(min='2024-04-01 00:00:00', max='2024-04-30 23:59:59', random=True,
                                   hours_preset='office', minute_granularity=15, second_granularity=10, seed=7)"/>

Weighted month/weekday/dom selection:

1
2
3
4
5
<key name="march_mondays_dom4"
     generator="DateTimeGenerator(min='2024-01-01 00:00:00', max='2024-12-31 23:59:59', random=True,
                                   month_weights='[0,0,1]+[0]*9',
                                   weekday_weights='[1,0,0,0,0,0,0]',
                                   dom_weights='[0,0,0,1]+[0]*27', seed=7)"/>

Sugar filters:

1
2
3
<key name="biz_days" generator="DateTimeGenerator(min='2024-04-01 00:00:00', max='2024-04-30 23:59:59', random=True, weekdays='Mon-Fri')"/>
<key name="q1_only"  generator="DateTimeGenerator(min='2024-01-01 00:00:00', max='2024-12-31 23:59:59', random=True, months='Q1')"/>
<key name="dom_last" generator="DateTimeGenerator(min='2024-01-01 00:00:00', max='2024-12-31 23:59:59', random=True, dom='last')"/>

Biased intra‑day and sampled seconds/minutes:

1
2
3
4
5
6
7
8
<key name="biased"
     generator="DateTimeGenerator(
       min='2020-02-01 0:0:0', max='2025-07-31 0:0:0',
       month_weights='[0.5] * 2 + [1] * 3 + [0.8] * 3 + [1] * 3 + [0.5]',
       hour_weights='[0.6]*6 + [0.3]*16 + [0.1]*2',
       minute_weights='[1 if m in (0,15,30,45) else 0 for m in range(60)]',
       second_weights='[1]+[0]*59', seed=123
     )"/>

Notes on XML weight expressions: - Due to XML parsing, pass list expressions as strings (quote the expression) when they contain commas. - Simple patterns like [0]*12 work quoted or unquoted. Complex list comprehensions must be quoted, as above.

Python API Examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
from datamimic_ee.domains.common.literal_generators.datetime_generator import DateTimeGenerator

# Deterministic weekends in January 2024
gen = DateTimeGenerator(
    min='2024-01-01 00:00:00', max='2024-01-31 23:59:59', random=True,
    weekdays='Weekend', seed=42,
)
print(gen.generate())

# Time weights only (uniform dates)
gen = DateTimeGenerator(random=True, hour_weights=[0.2]*6 + [0.02]*18, seed=1)
print(gen.generate())

# Epoch via Python API
gen = DateTimeGenerator(random=True, as_epoch=True, epoch_unit='milliseconds')
print(gen.generate())  # -> int millis

Edge Cases & Tips

  • Leap years: DOM weights and dom='last' handle February 29 correctly.
  • Weekday wrap: ranges like Fri-Mon wrap across week boundaries.
  • Bounds + granularity: When granularity excludes most ticks, sampling remains within the min/max time range of the chosen day.
  • Precedence: Explicit weights override sugar. Combine month/weekday/dom carefully to avoid eliminating all days.
  • random_mode is an alias of random for compatibility.