Skip to content

Configuration files

Forest commands write all of these files. You add by hand only a stage's sync_by, when it needs other units, and manifest, if you want --id. The examples below come from a workspace built with:

forest init
forest remote add origin /path/to/remote
forest remote use origin
forest checkout demo
forest add raw ./data/raw
forest add results ./data/results --direction push
# then set sync_by: file under results in .forest/checkouts/demo/forest.yaml
forest push raw
File Git
.forest/config.yaml committed
.forest/checkouts/<name>/forest.yaml committed
.forest/HEAD ignored
.forest/local.yaml ignored
.forest/checkouts/<name>/local.yaml ignored
.forest/checkouts/<name>/sync_state.json ignored

forest init writes these ignore lines to .gitignore, plus lines for the *.lock files next to config.yaml and sync_state.json.

.forest/config.yaml

Lists the workspace's checkouts and remotes.

version: 1
checkouts:
  demo:
    remote: origin
remotes:
  origin:
    url: /path/to/remote
Key Meaning
version Format version, 1.
checkouts.<name>.remote The remote this checkout saved when it was created. Omitted if no remote was selected.
remotes.<name>.url Absolute local path, s3://..., sftp://... or user@host:/path. Must not contain credentials.
remotes.<name>.region Optional. S3 region.
remotes.<name>.endpoint Optional. S3-compatible endpoint URL.
remotes.<name>.profile Optional. S3 auth profile.
remotes.<name>.key_file Required for SFTP. Private key file.
remotes.<name>.known_hosts Required for SFTP. known_hosts file.

.forest/local.yaml

active_remote: origin
stage_paths: {}
Key Meaning
active_remote Set by forest remote use. New checkouts save this remote. If it is unset and the workspace has exactly one remote, new checkouts use that one.
stage_paths Written empty here. Bindings live in each checkout's local.yaml.

.forest/HEAD

One line: the active checkout name.

.forest/checkouts/<name>/forest.yaml

Stage definitions for one checkout, shared through Git.

project: demo
stages:
  raw:
    remote_path: demo/raw
  results:
    remote_path: demo/results
    sync_by: file
    direction: push
Key Meaning
project Checkout name.
stages.<stage>.remote_path Prefix under the remote URL. Default <checkout>/<stage>. forest add STAGE PATH REMOTE:PATH sets it to PATH.
stages.<stage>.sync_by What one unit is: subdirectory (default, omitted; each subdirectory), directory (the whole path) or file (each file). Edit this key to change it; no flag sets it.
stages.<stage>.direction both (default, omitted), push or pull. Edit this key to change it; no flag overrides it.
remotes Older checkouts may define remotes here, with the same keys as in config.yaml. New checkouts keep remotes in config.yaml.
manifest.file CSV file, relative to the workspace root.
manifest.id_column Column that --id matches.
manifest.stage_columns <stage>: <column>. The column holds that stage's unit names.

Example manifest section, with samples.csv containing sample,raw_unit and S01,run1:

manifest:
  file: samples.csv
  id_column: sample
  stage_columns:
    raw: raw_unit

$ forest status --id S01
synced:origin  raw/run1/a.txt  matches remote

.forest/checkouts/<name>/local.yaml

Settings for one checkout on this machine only.

active_remote: origin
stage_paths:
  raw: data/raw
  results: data/results
Key Meaning
active_remote Copy of the checkout's saved remote, or null. If both are set, it must match checkouts.<name>.remote in config.yaml. An older checkout with no saved remote uses this value instead.
stage_paths.<stage> Local path bound by forest add or forest bind. Relative paths resolve from the workspace root. Paths inside .forest/ are rejected.

A fresh clone has no local.yaml. forest checkout NAME recreates it and lists the stages to bind:

$ forest checkout demo
Switched to checkout 'demo'.
Run: forest bind raw <PATH>
Run: forest bind results <PATH>
$ forest bind raw ./data/raw
Bound stage 'raw' -> data/raw
$ forest pull raw
Transferred: 2 B, 1 files
PULL raw/run1  1 files, 2 B <- origin

.forest/checkouts/<name>/sync_state.json

Forest keeps the latest push and the latest pull of each unit, per remote. push, pull, status and diff read it. Don't edit it. If a push or pull is refused because of this history, see Sync recovery.