Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: f3: Keep retrying the task forever #337

Merged
merged 3 commits into from
Dec 6, 2024
Merged

fix: f3: Keep retrying the task forever #337

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
003cee3
fix: f3: Keep retrying the task forever
magik6k Dec 3, 2024
ee46c1c
Update tasks/f3/f3_task.go
magik6k Dec 4, 2024
2dba4e1
make gen
magik6k Dec 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion tasks/f3/f3_task.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@

import (
"context"
"database/sql"

Check failure on line 5 in tasks/f3/f3_task.go

View workflow job for this annotation

GitHub Actions / build-debug 

"database/sql" imported and not used

Check failure on line 5 in tasks/f3/f3_task.go

View workflow job for this annotation

GitHub Actions / build-2k 

"database/sql" imported and not used

Check failure on line 5 in tasks/f3/f3_task.go

View workflow job for this annotation

GitHub Actions / build-forest 

"database/sql" imported and not used

Check failure on line 5 in tasks/f3/f3_task.go

View workflow job for this annotation

GitHub Actions / build-calibnet 

"database/sql" imported and not used

Check failure on line 5 in tasks/f3/f3_task.go

View workflow job for this annotation

GitHub Actions / build-mainnet 

"database/sql" imported and not used

Check failure on line 5 in tasks/f3/f3_task.go

View workflow job for this annotation

GitHub Actions / gen-check 

"database/sql" imported and not used

Check failure on line 5 in tasks/f3/f3_task.go

View workflow job for this annotation

GitHub Actions / lint 

"database/sql" imported and not used (typecheck)

Check failure on line 5 in tasks/f3/f3_task.go

View workflow job for this annotation

GitHub Actions / lint 

"database/sql" imported and not used) (typecheck)

Check failure on line 5 in tasks/f3/f3_task.go

View workflow job for this annotation

GitHub Actions / test (test-itest-curio, ./itests/curio_test.go) 

"database/sql" imported and not used

Check failure on line 5 in tasks/f3/f3_task.go

View workflow job for this annotation

GitHub Actions / test (test-all, `go list ./... | grep -v curio/itests`) 

"database/sql" imported and not used
magik6k marked this conversation as resolved.
Show resolved Hide resolved
"errors"
"github.com/yugabyte/pgx/v5"
magik6k marked this conversation as resolved.
Show resolved Hide resolved
"time"

logging "github.com/ipfs/go-log/v2"
magik6k marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -71,6 +73,12 @@
var spID int64
err = f.db.QueryRow(ctx, "SELECT sp_id FROM f3_tasks WHERE task_id = $1", taskID).Scan(&spID)
if err != nil {
if err == pgx.ErrNoRows {
// can only happen when the SP ID was removed from the cluster configuration
log.Warnw("marking f3 task with no corresponding spid as done", "task", taskID)
return true, nil
}

return false, xerrors.Errorf("failed to get sp_id: %w", err)
}

Expand Down Expand Up @@ -195,7 +203,10 @@
Gpu: 0,
Ram: 10 << 20,
},
MaxFailures: 1,
// No MaxRetries, never kill this task
RetryWait: func(retries int) time.Duration {
return time.Minute
},
}
}

Expand Down
Loading