Skip to content

Element <object-storage>

Purpose: Declare a named S3-compatible object-storage client for project reads and writes.

Why: Use it when project sources or generated file artifacts live in S3-compatible storage.

Example

1
<object-storage id="project-storage"/>

Decision guide

Business value: Keeps project inputs and generated file artifacts in one named S3-compatible storage contract.

  • Use when

    • Sources or file-export targets live in S3, MinIO, or another supported object store.

  • Choose another approach when

    • The data belongs in a database, message broker, or descriptor-local filesystem.

  • Prerequisites

    • Provide provider, authentication mode, bucket or container, and permitted object paths.

Complete examples

Read and write project artifacts through S3-compatible storage

Use object-storage when source files and generated artifacts belong in a brokered S3 or MinIO bucket rather than the local project filesystem; keep credentials in the deployment's provider chain.

object-storage-artifact/datamimic.xml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
<setup defaultSeparator="|">
    <object-storage id="project_storage" system="s3"/>
    <generate name="audit_rows"
              source="project_storage"
              sourceUri="incoming/audit.ent.csv"
              count="3"
              distribution="ordered"
              storageId="project_storage"
              exportUri="out/audit"
              target="LogExporter,PARQUET"/>
</setup>

Rules and invalid combinations

No additional element-specific rules. Required attributes, types, and supported values are shown below.

Allowed parents / Allowed children

Allowed parents: else, else-if, if, setup, while

Allowed children:

None

  • Choose a source and target — Guides users and agents from an input ownership boundary and intended side effect to one bounded source and target contract.

Attributes

Show all 16 attributes

accessKey

Access key / client id.

optional; string; Default: null.

authMode

Authentication mode for AWS object storage.

optional; string; Default: null; Values: static, default_chain, iam, iam_env.

bucket

Default bucket name for this storage.

optional; string; Default: null.

connString

Connection string / URI for the storage service.

optional; string; Default: null.

container

Default container name (e.g., Azure Blob).

optional; string; Default: null.

endpointUrl

Optional custom S3 endpoint URL (S3-compatible or VPC endpoint).

optional; string; Default: null.

environment

Optional environment label for this storage config.

optional; string; Default: null.

host

Hostname or endpoint for the storage service.

optional; string; Default: null.

id

Object storage ID for accessing the object storage environment configuration.

required; string.

port

Port for the storage service.

optional; integer; Default: null.

region

Region for cloud object storage.

optional; string; Default: null.

secretKey

Secret key / client secret.

optional; string; Default: null.

secure

Use HTTPS for this storage endpoint. Defaults to false for MinIO, which is commonly reached over plain HTTP inside a cluster, and true for AWS.

optional; boolean; Default: null.

sessionToken

Optional AWS session token for temporary credentials.

optional; string; Default: null.

system

Optional platform system identifier used to resolve environment properties.

optional; string; Default: null.

type

Object storage type/adapter.

optional; string; Values: minio, aws, azure.