Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Unreleased

### Fixed

- Sync: keep finalization fetches blob-filtered when completing shallow Git history, and explain live-child owner release refusals with safe lease recovery guidance. [PR 2854](https://github.com/openclaw/crabbox/pull/2854).

## 0.78.1 - 2026-10-15

### Highlights
Expand Down
14 changes: 14 additions & 0 deletions docs/features/sync.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,6 +318,13 @@ the original error. These labels add diagnostic context, not permission to
continue or retry; arbitrary protocol output is not added to those transport
error messages. An ambiguous inspection still fails closed.

A release refusal with `protocol state CHILD` and exit 75 means the witnessed
remote phase is still active; the owner is retained. Cancelling local SSH does
not prove that its remote Git fetch or other phase stopped. Wait for that phase
to finish before retrying. If it remains stuck, use the run's printed
`stop_command` to release the lease and retry on a fresh lease. `--full-resync`
still needs the same owner and cannot override a live child.

POSIX and WSL2 children register themselves before executing the requested
workload. Registration waits at most five seconds for the owner lock; it does
not leave a background child waiting indefinitely for a start file. A failed
Expand Down Expand Up @@ -354,6 +361,13 @@ Once ownership is established, sync runs these steps:
10. Finalize: git-hydrate the worktree against the configured base ref, run the
mass-deletion sanity check, and record the new fingerprint.

Finalization requests blob-filtered Git fetches, including when completing a
shallow repository left by workflow setup or a previous command. On origins
that support filtering, commit ancestry becomes available without downloading
Comment on lines +364 to +366

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply blob filtering to native Windows finalization

This statement is false for native Windows targets: syncWindowsNative still finalizes through windowsGitCoherence, whose fetch at internal/cli/sync_windows_target.go:264 has neither --filter=blob:none nor shallow-history completion. Consequently a native Windows workspace can still download all fetched blobs, and a shallow checkout does not gain the promised ancestry. Git documents that --filter=blob:none omits file contents until needed; mirror the filtering and shallow handling in the Windows finalizer, or scope the documented fix explicitly to POSIX/WSL targets.

Useful? React with 👍 / 👎.

every historical version of every file. Git can fetch historical file contents
when a later command explicitly requests them. Origins that ignore filtering
can still transfer full historical contents.

For a new or empty POSIX workspace with Git seeding disabled (`sync.gitSeed:
false`) and no requested Git overlay, the manifest-writing command also checks
whether the target has GNU tar or bsdtar and gzip. Eligible first transfers use
Expand Down
7 changes: 4 additions & 3 deletions internal/cli/ssh.go
Original file line number Diff line number Diff line change
Expand Up @@ -2955,10 +2955,11 @@ fi
if !opts.GitOverlay && !opts.PlainManifest && opts.HydrateGit && opts.BaseRef != "" {
refspec := "+refs/heads/" + opts.BaseRef + ":refs/remotes/origin/" + opts.BaseRef
script += `if exact_git_root && git remote get-url origin >/dev/null 2>&1; then
hydrate_fetch() { git fetch --quiet --filter=blob:none origin "$@"; }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fall back when hydration Git lacks filtering

On a POSIX target running Git 2.18, a hydration-only sync (sync.gitSeed: false, so coherence is disabled while HydrateGit remains enabled) now sends every fetch attempt through this helper, and each fails on the unsupported --filter option. The trailing || true then reports a successful sync and records hydration metadata even though the repository was never unshallowed or updated; previously these fetches worked without filtering. The Git 2.18 fetch manual documents --unshallow but has no --filter option, so this path should detect filter support or retry hydration without it.

Useful? React with 👍 / 👎.

if [ "$(git rev-parse --is-shallow-repository 2>/dev/null)" = true ]; then
git fetch --quiet --unshallow origin ` + shellQuote(refspec) + ` || git fetch --quiet --depth=1000 origin ` + shellQuote(refspec) + ` || git fetch --quiet origin ` + shellQuote(refspec) + ` || git fetch --quiet origin ` + shellQuote(opts.BaseRef) + ` || true
hydrate_fetch --unshallow ` + shellQuote(refspec) + ` || hydrate_fetch --depth=1000 ` + shellQuote(refspec) + ` || hydrate_fetch ` + shellQuote(refspec) + ` || hydrate_fetch ` + shellQuote(opts.BaseRef) + ` || true
else
git fetch --quiet origin ` + shellQuote(refspec) + ` || git fetch --quiet origin ` + shellQuote(opts.BaseRef) + ` || true
hydrate_fetch ` + shellQuote(refspec) + ` || hydrate_fetch ` + shellQuote(opts.BaseRef) + ` || true
fi
fi
`
Expand Down Expand Up @@ -3058,7 +3059,7 @@ coherence_committed=; coherence_mutated=; head_changed=; index_changed=
tmp_ref="refs/crabbox/sync-$expected_token"; advertised_branch=` + shellQuote(plan.Branch) + `; expected_origin=` + shellQuote(plan.RemoteURL) + `
` + remoteGitOriginTransportFunctions() + `
coherence_fetch() {
set -- --quiet --no-tags
set -- --quiet --filter=blob:none --no-tags

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep sync working when Git fetch lacks filters

On a POSIX target running Git 2.18, this unconditional option makes every coherence fetch fail because that version's documented fetch options include --unshallow but not --filter. The shell converts this into exit 69, and gitOriginRuntimeFallbackResult does not classify the resulting unknown-option diagnostic, so an otherwise supported origin-based sync now aborts instead of falling back to plain-manifest sync. Please detect filter support or handle this diagnostic as a safe fallback without performing the problematic unfiltered history fetch.

Useful? React with 👍 / 👎.

# An existing shallow boundary can hide ancestors of the advertised branch.
if [ "$(git rev-parse --is-shallow-repository 2>/dev/null)" = true ]; then
set -- "$@" --unshallow
Expand Down
61 changes: 61 additions & 0 deletions internal/cli/sync_finalize_history_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
//go:build !windows

package cli

import (
"os"
"os/exec"
"path/filepath"
"testing"
)

func TestRemoteFinalizeSyncDoesNotDownloadHistoricalBlobs(t *testing.T) {
f := newGitCoherenceFixture(t)
runGit(t, f.origin, "config", "uploadpack.allowFilter", "true")
oldBlob := gitOutput(f.source, "rev-parse", f.a+":tracked.txt")
for _, scenario := range []struct {
name string
coherence, hydrate bool
}{
{name: "coherence", coherence: true},
{name: "hydrate", hydrate: true},
{name: "coherence-and-hydrate", coherence: true, hydrate: true},
} {
t.Run(scenario.name, func(t *testing.T) {
workdir := filepath.Join(t.TempDir(), "work")
origin := "file://" + filepath.ToSlash(f.origin)
if out, err := exec.Command("git", "clone", "--quiet", "--depth=1", origin, workdir).CombinedOutput(); err != nil {
t.Fatalf("shallow clone: %v\n%s", err, out)
}
assertOldBlobMissing := func() {
t.Helper()
cmd := exec.Command("git", "-C", workdir, "-c", "protocol.allow=never", "cat-file", "-e", oldBlob)
cmd.Env = append(os.Environ(), "GIT_NO_LAZY_FETCH=1")
if err := cmd.Run(); err == nil {
t.Fatal("downloaded a blob that exists only in history")
}
}
assertOldBlobMissing()
plan := gitCoherencePlan{}
if scenario.coherence {
plan = f.plan(t, f.c)
plan.RemoteURL = origin
}
const token = "1234567890abcdef1234567890abcdef"
stageCoherenceFinalize(t, workdir, token)
command := remoteFinalizeSync(workdir, remoteSyncFinalizeOptions{
Token: token, Coherence: plan, HydrateGit: scenario.hydrate, BaseRef: "main",
})
if out, err := exec.CommandContext(t.Context(), "/bin/sh", "-c", command).CombinedOutput(); err != nil {
t.Fatalf("finalize: %v\n%s", err, out)
}
assertOldBlobMissing()
requireGitOutput(t, workdir, "false", "rev-parse", "--is-shallow-repository")
requireGitOutput(t, workdir, f.c, "rev-parse", "HEAD")
// Commit ancestry remains available even though historical file bytes
// are deferred until a caller explicitly asks Git for them.
runGit(t, workdir, "merge-base", "--is-ancestor", f.a, "HEAD")
requireGitOutput(t, workdir, "A", "show", f.a+":tracked.txt")
})
}
}
4 changes: 3 additions & 1 deletion internal/cli/workspace_owner.go
Original file line number Diff line number Diff line change
Expand Up @@ -664,7 +664,9 @@ func (o *workspaceOwner) Close(ctx context.Context) (err error) {
}
renewErr := o.Err()
response, releaseErr := callWorkspaceOwnerTransport(ctx, o.callTimeout(), o.transport, workspaceOwnerRemoteRequest{Action: workspaceOwnerRelease, Key: o.key, Token: o.token, TTL: o.ttl})
if releaseErr != nil {
if response == "CHILD" && ctx.Err() == nil && (releaseErr == nil || exitCode(releaseErr) == 75) {
releaseErr = errors.Join(Exit(7, "release remote workspace owner failed closed: remote phase is still active (protocol state CHILD); owner retained; wait for the phase to finish, or use the printed stop_command and retry on a fresh lease"), releaseErr)
} else if releaseErr != nil {
releaseErr = workspaceOwnerProtocolError(response, releaseErr)
releaseErr = workspaceOwnerCallError("release remote workspace owner", releaseErr)
} else if response != "RELEASED" {
Expand Down
47 changes: 47 additions & 0 deletions internal/cli/workspace_owner_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,53 @@ func TestWorkspaceOwnerTransportContextDiagnostics(t *testing.T) {
}
}

func TestWorkspaceOwnerReleaseChildDiagnostic(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("requires a POSIX exit-status fixture")
}
for _, code := range []int{0, 75, 255} {
t.Run(strconv.Itoa(code), func(t *testing.T) {
var cause error
if code != 0 {
cause = exec.Command("/bin/sh", "-c", "exit "+strconv.Itoa(code)).Run()
if cause == nil {
t.Fatal("exit fixture succeeded")
}
}
calls := 0
owner := &workspaceOwner{
stop: make(chan struct{}), done: make(chan struct{}),
transport: workspaceOwnerTransportFunc(func(_ context.Context, req workspaceOwnerRemoteRequest) (string, error) {
calls++
if req.Action != workspaceOwnerRelease {
t.Fatalf("unexpected action %s", req.Action)
}
return "CHILD", cause
}),
}
close(owner.done)
err := owner.Close(t.Context())
if err == nil || calls != 1 {
t.Fatalf("release did not fail closed once: calls=%d err=%v", calls, err)
}
if code == 255 {
if !strings.Contains(err.Error(), "ambiguous remote state") || strings.Contains(err.Error(), "owner retained") {
t.Fatalf("transport failure became a confirmed refusal: %v", err)
}
return
}
for _, want := range []string{"protocol state CHILD", "remote phase is still active", "owner retained", "stop_command", "fresh lease"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("missing %q in %v", want, err)
}
}
if cause != nil && !errors.Is(err, cause) {
t.Fatal("release diagnostic lost the original refusal")
}
})
}
}

func newFakeWorkspaceOwnerRemote() *fakeWorkspaceOwnerRemote {
return &fakeWorkspaceOwnerRemote{changed: make(chan struct{})}
}
Expand Down
Loading