Environment variables¶
Periplo is configured with environment variables. Every setting is named
PERIPLO_<SETTING>, is read once when the application starts, and is validated before
anything else happens: a missing or malformed value stops the start-up with an error
that names the variable, for example PERIPLO_MAX_RESULT_ROWS: Expected `int` >= 1.
Numbers are written as plain digits (67108864, not 64MiB). Booleans accept true,
false, 1 and 0. Lists are comma-separated; spaces around each item are ignored and
empty items are dropped.
Where the container image sets a value different from the application default, both are given.
Catalog and sources¶
Variable |
Default |
Meaning |
|---|---|---|
|
unset; image: |
Path of the sources file. Required: the application refuses to start without one. |
|
|
Most folders one discovery may list per source. Reaching it stops the walk and reports the result as partial, instead of listing a huge bucket forever. |
Queries¶
Variable |
Default |
Meaning |
|---|---|---|
|
|
Queries that may run at the same time. |
|
|
Most rows a query returns; a longer result is cut and marked as truncated. A request may ask for fewer rows, never for more. |
|
|
Size of the Arrow stream after which a query sends no further batches; the result is marked as truncated. |
|
|
Time limit of each query, in seconds; must be greater than 0. A query that runs longer ends with a |
Table metadata¶
Schemas, statistics and history come from each table’s Delta log. Periplo keeps the snapshots it has opened, and what it derived from them, in memory.
Variable |
Default |
Meaning |
|---|---|---|
|
|
How long a table’s snapshot is served without checking storage for a newer version. |
|
|
Memory budget of the derived metadata cache. |
|
|
Open Delta snapshots kept in memory. |
/health/ready reports the counters of both caches (hits, misses, evictions, bytes)
next to the readiness state.
Web console and logging¶
Variable |
Default |
Meaning |
|---|---|---|
|
unset; image: |
Directory of the built web console, served by the API process on the same port. Unset serves the API only. |
|
|
|
|
|
Level of the service log, as a standard level name such as |
ETL orchestrator¶
The ETL section talks to Prefect. It is off until
PERIPLO_PREFECT_API_URL is set; everything else keeps working without it. See
ETL console for what it shows.
Variable |
Default |
Meaning |
|---|---|---|
|
unset (ETL off) |
Base URL of Prefect’s REST API, for example |
|
unset |
Prefect Cloud API key, sent as |
|
unset |
|
|
unset |
Base URL of Prefect’s own UI, used for the “Open in orchestrator” links. It must be reachable from the user’s browser. Unset shows no links; it is never derived from the API URL. |
|
empty |
Comma-separated tags. Only deployments that carry all of them are shown, and the runs Periplo asks Prefect for are filtered by them. Empty shows every deployment of the workspace. |
|
|
Time limit of each call to Prefect, in seconds. |
|
|
Whether the console may start runs and pause or resume schedules. Read the warning below before turning it on. |
|
empty |
Comma-separated log line prefixes that the log viewer folds as noise by default. Empty folds nothing. |
Warning
The open-source build has no authentication. With PERIPLO_ETL_ALLOW_OPERATE=true,
anyone who can reach the console can run ETLs and pause or resume their schedules. Keep
it off, or put an authenticating layer in front of Periplo (see Security).
Outside Periplo’s own settings¶
These are read by the libraries Periplo runs on, not by Periplo itself.
Variable |
Read by |
Meaning |
|---|---|---|
|
the S3 clients |
Credentials and region from the standard AWS chain. Without them, the chain falls back to the role of the machine or task. |
|
uvicorn |
Address the server listens on. The image sets |
|
uvicorn |
Seconds requests in flight get to finish after |
The Compose file of the repository also reads PERIPLO_PORT (host port, 8080 by default)
and PERIPLO_IMAGE (image to run). They configure Compose, not the application.