Verify every transfer against the remote by default

Date: 2026-07-03

Forest had an rclone check wrapper that nothing called (docs/research/dud-rclone-lessons.md §6 flagged it as "one call away"). Transfers trusted rclone's exit code alone: a truncated upload, a backend that lied about success, or corruption between transfer and record went unnoticed until much later — dud accepts this gap; forest should not.

forest push and forest pull now verify the transferred file set against the remote immediately after every successful transfer, and --no-verify opts out per invocation. The ordering is transfer → verify → record: a unit that fails verification prints ERROR <unit> verification failed: <detail>, exits nonzero, and records no sync entry — for pull that leaves the unit visibly out of sync, for push it means a re-push is required. Verification is on by default because a data-sync tool's first duty is to not lie about having synced; the cost is bounded (see below) and the escape hatch is explicit.

Mechanics (rclone.check_files): layout-preserving batches run rclone check --one-way --files-from-raw <transferred set> — sizes plus content hashes wherever the backend pair supports them, restricted to exactly the files just moved (remote extras are never an error; pushes are additive). Renamed or remapped keys (sync_by=file, remote_path remaps) cannot go through rclone check, so each is stat'ed with lsjson --stat with --hash-type flags and compared by size, plus the strongest available SHA-256, SHA-1, or MD5 digest. Backends that expose no usable hash retain a size-only best-effort check. Checks run through the standard retry/metrics path; a mismatch is reported as a verification failure, not retried as a network error, and does not trip the circuit breaker (the transfer just succeeded, so the remote is alive — a mismatch is an integrity signal, not a health signal).

Skipped (SKIP … already synced) and --dry-run units receive no post-transfer verification because nothing moved; pull skips instead require the live source-hash proof in ADR 0023. Auditing data at rest is status --checksum (local, ADR 0017) — not a hidden remote scan inside push.

Consequences

• Every default push/pull pays one rclone check over the transferred set (hash computation on both sides for local remotes; size+hash-where-listed for cloud backends). --no-verify restores the old single-pass cost.
• A verify failure leaves sync state unchanged — status/re-push/re-pull converge naturally; nothing records success that was not observed.
• Rename-path verification against backends that expose no usable hash is size-only; proving content equality there requires a later hash-capable audit or retransfer.
• Fake transports in tests must provide check_transfer (returning []) alongside push/pull/ls.
• Contract locked in tests/test_verify_lock.py (VAL-VERIFY-001..005).