loom.etl.runtime¶

Runtime contracts used by executor and backend implementations.

class loom.etl.runtime.AgentBatchRunner(*args, **kwargs)[source]¶

Bases: Protocol

Optional capability protocol for running a declared agent over a frame.

Implemented by loom.ai.etl and injected into ETLExecutor and ETLCompiler as a separate dependency, so the ETL pillar depends on this contract and never on the AI pillar. Used exclusively to serve WithAgent declarations.

validate(name, output_type, *, max_usd)[source]¶

Check, without spending anything, that name can serve a step.

Parameters:

name (str) – Agent name declared by the step.
output_type (type[Any]) – Type the step expects every answer to decode into.
max_usd (Decimal | None) – Budget of one execution of the step, when declared.

Returns:

Every issue found; empty when the agent can serve the step.

Return type:

tuple[AgentIssue, …]

version(name)[source]¶

Return the version of agent name, stable while its behaviour is.

Parameters:: name (str) – Agent name declared by the step.
Returns:: The value written in the agent_version column.
Return type:: str

map(name, frame, *, keys, prompt, output_type, max_usd)[source]¶

Run agent name once per row of frame.

Parameters:

name (str) – Agent name declared by the step.
frame (Any) – Backend frame holding the rows to answer.
keys (Sequence[str]) – Columns identifying a row, copied to the output.
prompt (object) – Backend expression, or column name, giving each row’s prompt.
output_type (type[Any]) – Type every answer is decoded into.
max_usd (Decimal | None) – Budget of this call; None spends without a ceiling.

Returns:

One output row per input row, failures included.

Return type:

Any

class loom.etl.runtime.AgentIssue(kind, message)[source]¶

Bases: object

One reason a declared agent cannot serve a step.

Parameters:

kind (AgentIssueKind)
message (str)

kind¶

What is wrong.

Type:: loom.etl.runtime.contracts.AgentIssueKind

message¶

Human-readable detail, naming the agent and never a secret.

Type:: str

class loom.etl.runtime.AgentIssueKind(value)[source]¶

Bases: StrEnum

Why a declared agent cannot serve a step.

exception loom.etl.runtime.ConfigValueError(key, value_type, failure)[source]¶

Bases: ConfigError

A declared config value is missing, unresolvable or of the wrong type.

The message names the key and the expected type only; neither the value nor the underlying exception is attached (__cause__ and __context__ are None).

Parameters:

key (str)
value_type (object)
failure (ConfigValueFailure)

Return type:

None

key¶: Dot-separated config path.

failure¶: Why the value could not be supplied.

class loom.etl.runtime.ConfigValueFailure(value)[source]¶

Bases: StrEnum

Why a config value could not be supplied.

class loom.etl.runtime.TableDiscovery(*args, **kwargs)[source]¶

Bases: Protocol

Protocol for catalog-backed table existence and schema discovery.

Used by ETLCompiler when a catalog is injected at construction time to validate that source and target tables are valid before execution begins.

exists(ref)[source]¶

Return True when the table exists in the catalog.

Parameters:: ref (TableRef) – Logical table reference to check.
Returns:: True when the table is registered.
Return type:: bool

columns(ref)[source]¶

Return table column names in schema order.

Parameters:: ref (TableRef) – Logical table reference.
Returns:: Tuple of column names, or () when unknown.
Return type:: tuple[str, …]

schema(ref)[source]¶

Return full table schema.

Parameters:: ref (TableRef) – Logical table reference.
Returns:: Ordered column schema tuple, or None when table is missing.
Return type:: tuple[ColumnSchema, …] | None

update_schema(ref, schema)[source]¶

Persist table schema after a successful write.

Parameters:

ref (TableRef) – Logical table reference.
schema (tuple[ColumnSchema, ...]) – Authoritative table schema.

Return type:

None

class loom.etl.runtime.SourceReader(*args, **kwargs)[source]¶

Bases: Protocol

Protocol for reading one ETL source specification into a frame.

read(spec, params_instance, /)[source]¶

Read the source and return a frame.

Parameters:

spec (TableSourceSpec | FileSourceSpec | TempSourceSpec | MongoSourceSpec | ClickHouseSourceSpec | DynamoDbSourceSpec) – Compiled source specification.
params_instance (Any) – Concrete params for current run.

Returns:

Backend frame type accepted by the step execute() method.

Return type:

Any

class loom.etl.runtime.SQLExecutor(*args, **kwargs)[source]¶

Bases: Protocol

Optional capability protocol for SQL execution over source frames.

execute_sql(frames, query, /)[source]¶

Execute a SQL query against backend frames.

Used by StepSQL execution path in ETLExecutor.

Parameters:

frames (dict[str, Any]) – Alias -> frame mapping from resolved step sources.
query (str) – SQL query text.

Returns:

Backend frame type with query results.

Return type:

Any

class loom.etl.runtime.TargetWriter(*args, **kwargs)[source]¶

Bases: Protocol

Protocol for writing one frame into one ETL target specification.

write(frame, spec, params_instance, /, *, streaming=False, write_ctx=None)[source]¶

Write frame to target.

Parameters:

frame (Any) – Frame returned by step execute().
spec (TargetSpec) – Compiled target specification.
params_instance (Any) – Concrete params for current run.
streaming (bool) – Hint for lazy backends to use streaming materialization.
write_ctx (WriteContext | None) – Execution context for audit-column injection. None disables audit columns regardless of config.

Return type:

None