Skip to content

fix(go): fall back to row-based reader when Storage Read isn't accelerated #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
will-sargent-dbtlabs wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix(go): fall back to row-based reader when Storage Read isn't accelerated #6

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions go/record_reader.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,16 +160,16 @@ func runQuery(ctx context.Context, logger *slog.Logger, query *bigquery.Query, e
if useLegacyAPI {
arrowIterator = newRowBasedArrowIterator(iter, alloc)
} else if !iter.IsAccelerated() {
// !IsAccelerated() -> failed to get Arrow stream -> we are
// probably lacking permissions. readSessionUser may sound
// unrelated but creating a "read session" is the first step
// of using the Storage API. Note that Google swallows the
// real error, so this is the best we can do.
// https://cloud.google.com/bigquery/docs/reference/storage#create_a_session
return nil, js, -1, wrap(adbc.Error{
Code: adbc.StatusUnauthorized,
Msg: "[bq] Arrow reader requires roles/bigquery.readSessionUser, see https://github.com/apache/arrow-adbc/issues/3282",
})
// !IsAccelerated() means no Storage Read session was created —
// either a permissions gap (readSessionUser, see
// https://github.com/apache/arrow-adbc/issues/3282) or the
// Storage API being transiently unavailable (Google swallows
// the real reason either way). The row-based reader doesn't
// need Storage Read at all, so fall back to it instead of
// failing a query/test over a read-path capability gap. See
// https://github.com/dbt-labs/dbt-core/issues/15463.
logger.WarnContext(ctx, "storage read API not accelerated for query, falling back to row-based reader", "job_id", job.ID())
arrowIterator = newRowBasedArrowIterator(iter, alloc)
} else {
if arrowIterator, err = iter.ArrowIterator(); err != nil {
return nil, js, -1, wrap(errToAdbcErr(adbc.StatusInternal, err, "read Arrow query results"))
Expand Down
Loading