5. Push, pull & status¶
With stages bound and a remote configured, the git-shaped verbs do what you expect — against data instead of commits.
First push¶
NOTICE: 43 B / 43 B, 100%, 0 B/s, ETA -
NOTICE: 24 B / 24 B, 100%, 0 B/s, ETA -
PUSH raw/plot-01 2 files, 67 B -> origin
NOTICE: 34 B / 34 B, 100%, 0 B/s, ETA -
NOTICE: 25 B / 25 B, 100%, 0 B/s, ETA -
PUSH raw/plot-02 2 files, 59 B -> origin
NOTICE: 44 B / 44 B, 100%, 0 B/s, ETA -
PUSH processed/processed 1 files, 44 B -> origin
The NOTICE lines are rclone's per-transfer stats (timestamps trimmed here,
and from all later snippets); the PUSH lines are forest reporting one line
per unit. All three units — two from raw, one from processed — are now
on the remote, under each stage's remote_path:
~/forest-remote/
survey/
raw/
plot-01/ notes.txt temps.csv
plot-02/ notes.txt temps.csv
processed/ summary.csv
Real filenames, real directories — not a content-addressed blob store. You could browse that with a file manager.
One thing you don't see: after each transfer, forest checked the moved
files against the remote before recording anything — the order is always
transfer → verify → record. Verification is silent when it passes; a
transfer that can't be confirmed prints an ERROR, exits 1, and records
nothing, so forest never claims a sync it can't prove (--no-verify skips
the check).
Status, change, diff, push again¶
synced:origin raw/plot-01 67 B pushed 2026-07-02 22:32
synced:origin raw/plot-02 59 B pushed 2026-07-02 22:32
synced:origin processed/processed 44 B pushed 2026-07-02 22:32
Now record another day of temperatures in plot 1 and watch the state change:
stale:origin raw/plot-01 73 B pushed 2026-07-02 22:32
synced:origin raw/plot-02 59 B pushed 2026-07-02 22:32
synced:origin processed/processed 44 B pushed 2026-07-02 22:32
status classifies each unit as synced, stale (local changes since the
last push), local-only, or remote-only — and it does so from the local
sync state, without touching the network. diff names the changed files:
Pushing again moves only what changed; unchanged units are skipped:
PUSH raw/plot-01 2 files, 73 B -> origin
SKIP raw/plot-02 already synced to origin
SKIP processed/processed already synced to origin
Same-size edits and --checksum¶
That appended reading grew the file, which is how status noticed it. Not
every edit changes a file's size — correct the new reading in place
(9.9 instead of 8.9, same bytes) and default status sees nothing:
synced:origin raw/plot-01 73 B pushed 2026-07-02 22:32
synced:origin raw/plot-02 59 B pushed 2026-07-02 22:32
synced:origin processed/processed 44 B pushed 2026-07-02 22:32
That's a deliberate trade: default status compares paths and sizes, which
is what keeps it instant and offline. When you want the truth about
contents, --checksum re-hashes local files against the SHA-256 hashes
recorded at push time:
stale:origin raw/plot-01 73 B pushed 2026-07-02 22:32
synced:origin raw/plot-02 59 B pushed 2026-07-02 22:32
synced:origin processed/processed 44 B pushed 2026-07-02 22:32
push itself always hashes — it is the command that claims "already
synced", so it never trusts sizes alone. The same-size edit re-pushes with
no flags at all:
PUSH raw/plot-01 2 files, 73 B -> origin
SKIP raw/plot-02 already synced to origin
SKIP processed/processed already synced to origin
Listing the remote¶
forest ls asks the remote what it has (this one does use the network):
raw/
├── plot-01/
│ ├── notes.txt
│ └── temps.csv
└── plot-02/
├── notes.txt
└── temps.csv
processed/
└── summary.csv
Losing data and getting it back¶
Time for the payoff. Delete a plot — then recover it from the remote:
synced:origin raw/plot-01 73 B pushed 2026-07-02 22:32
synced:origin processed/processed 44 B pushed 2026-07-02 22:32
remote-only:origin raw/plot-02 59 B pushed 2026-07-02 22:32
SKIP raw/plot-01 already synced to origin
PULL raw/plot-02 2 files, 59 B <- origin
SKIP processed/processed already synced to origin
plot-02 is back on disk, byte-for-byte, and its pull is recorded in sync
state. The skipped units keep their existing records unchanged.
That recovery worked because the files were gone. Pull is more careful when a file is still there with changes you never pushed. Edit one in place — even keeping its size — and pull refuses to clobber it:
SKIP raw/plot-01 already synced to origin
ERROR raw/plot-02 local file has uncommitted changes: notes.txt (use --force to overwrite)
SKIP processed/processed already synced to origin
The command exits 1 and the dirty unit is untouched (clean units skip). Discarding your local edit in favor of the remote version is an explicit decision:
PULL raw/plot-01 2 files, 73 B <- origin
PULL raw/plot-02 2 files, 59 B <- origin
PULL processed/processed 1 files, 44 B <- origin
Pulls are also atomic: every file downloads to a temporary name and is renamed into place only once complete. A pull killed mid-transfer leaves the old bytes or the new bytes — never a truncated file.
Protecting the remote: force-with-lease¶
Push has the mirror-image guard. If the remote changed behind forest's back — a teammate uploading by hand, another machine pushing — push refuses to overwrite state it has never seen. Our remote is a plain directory, so an out-of-band change is one append away; make one, plus a legitimate local edit to give push a reason to transfer:
echo '3,9.1' >> ~/forest-remote/survey/raw/plot-02/temps.csv # someone else's change
echo '3,11.0' >> data/raw/plot-02/temps.csv # your new reading
forest push
SKIP raw/plot-01 already synced to origin
ERROR raw/plot-02 remote changed since last sync: survey/raw/plot-02/temps.csv (use --force to overwrite)
SKIP processed/processed already synced to origin
Like git's force-with-lease, the refusal means "the remote is not where you
left it" (exit 1). Look at what changed — forest ls, or just read the
file on a directory remote — then either pull --force to adopt the
remote version, or overwrite it deliberately:
SKIP raw/plot-01 already synced to origin
PUSH raw/plot-02 2 files, 66 B -> origin
SKIP processed/processed already synced to origin
Note the SKIPs: --force only overrides the lease. It never re-pushes
content that is already synced.
Bound stages only: bare verbs vs --all¶
Bare push/pull/status/diff cover every bound stage; unbound
stages warn and are skipped. Try it:
Warning: stage 'processed' is not bound; skipping. Run: forest bind processed <PATH>
SKIP raw/plot-01 already synced to origin
SKIP raw/plot-02 already synced to origin
--all keeps strict semantics — any unbound stage aborts the whole
transfer, so scripts that must move everything fail loudly (exit code 1):
Rebind before moving on:
What gets synced¶
Forest syncs all files in a unit, skipping OS junk (.DS_Store,
AppleDouble ._* files, *.tmp) and applying no content-based rules —
it moves files and tracks their sync state; it does not interpret them.
Transient transfer failures are retried (tunable via
environment variables), and a run aborts
early after 5 consecutive unit failures instead of hammering a dead remote.
Recap: bare verbs cover bound stages; --all is strict; status/diff
read local sync state (--checksum re-hashes contents); ls asks the
remote; data survives rm -rf. Every transfer is verified before it's
recorded, pull refuses to clobber unpushed local edits, push refuses to
clobber a remote that changed behind forest's back — and --force is
always an explicit decision.
Next: move less than everything — targeting slices.