Skip to content

sql: reimplement pg_*_is_visible as Go builtins#171344

Closed
rafiss wants to merge 1 commit into
cockroachdb:masterfrom
rafiss:pg-is-visible-go-builtins
Closed

sql: reimplement pg_*_is_visible as Go builtins#171344
rafiss wants to merge 1 commit into
cockroachdb:masterfrom
rafiss:pg-is-visible-go-builtins

Conversation

@rafiss

@rafiss rafiss commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

pg_function_is_visible, pg_table_is_visible, and pg_type_is_visible are called
once per row by ORM and psql \d introspection queries. They were implemented as
SQL-body builtins whose Postgres-correct shadow check ("is this the first object
of its name on the search path?") scanned pg_proc / pg_class / pg_type. Because
those virtual tables are indexed only by OID, the shadow check forced a full
materialization of the catalog on every call: O(catalog) per call, and O(N^2)
under the per-row introspection pattern. This made catalog introspection very
slow and was the cause of the TestDescribe timeout in #171054.

This commit reimplements the three builtins in Go (pkg/sql/pg_is_visible.go),
exposed through new eval.Planner methods. Instead of scanning the virtual
catalogs, each builtin resolves the object's name and schema from its OID and
walks the current search path doing leased-cache-backed lookups, mirroring
Postgres's RelationIsVisible / TypeIsVisible / FunctionIsVisible:

  • pg_function_is_visible resolves the OID's name and signature, then uses the
    normal function resolver to find the candidate overloads and checks that the
    first one (by search-path precedence) with matching argument types is the
    target. This preserves Postgres's signature-aware behavior: same-named
    functions with disjoint argument lists do not shadow each other.
  • pg_table_is_visible and pg_type_is_visible walk the search path and check, per
    schema, whether an object of the same name exists, stopping at the first match.

The per-call cost drops from a full catalog populate to a handful of
descriptor/namespace lookups, eliminating the quadratic blow-up.

Behavior is unchanged from the SQL implementation: visibility, cross-schema
shadowing, signature-aware function visibility, NULL for a non-existent OID, and
NULL for objects in another database all match. A few cases are handled
explicitly: relations and types in other databases (which have no row in the
per-database virtual catalogs) return NULL; virtual tables (e.g.
pg_catalog.pg_class) are resolved through pg_class so they remain visible; and
index entries, whose hashed OIDs cannot be reversed to a descriptor, fall back
to a pg_class lookup.

This restores the Go-builtin approach used before acf5006, which had moved
these to SQL bodies for a performance win that later regressed once the bodies
were made shadow-aware.

Performance

BenchmarkORMQueries/django_table_introspection calls pg_table_is_visible
once per row. Base master vs this PR (./dev bench --count=6, compared with
benchstat):

                                      │    master     │              this PR               │
                                      │    sec/op      │    sec/op     vs base              │
django_table_introspection_1_table      2737.5m ± 27%   287.0m ± 22%  -89.52% (p=0.002 n=6)
django_table_introspection_8_tables     2980.5m ± 17%   274.8m ±  5%  -90.78% (p=0.002 n=6)
geomean                                   2.856         280.8m        -90.17%

                                      │    master     │              this PR               │
                                      │     B/op       │     B/op      vs base              │
django_table_introspection_1_table      1425.9Mi ± 0%   106.2Mi ± 6%  -92.55% (p=0.002 n=6)
django_table_introspection_8_tables     1501.9Mi ± 0%   104.1Mi ± 0%  -93.07% (p=0.002 n=6)

                                      │    master     │              this PR               │
                                      │  allocs/op     │  allocs/op    vs base             │
django_table_introspection_1_table      8714.5k ± 0%   639.3k ± 7%   -92.66% (p=0.002 n=6)
django_table_introspection_8_tables     9569.0k ± 0%   624.3k ± 1%   -93.48% (p=0.002 n=6)

TestDescribe (psql \d/\df/\dT over the full catalog) drops from ~75s on
master to ~10.6s with this PR.

Resolves: #171054
Informs: #108334
Epic: none

Release note: None

pg_function_is_visible, pg_table_is_visible, and pg_type_is_visible are called
once per row by ORM and psql \d introspection queries. They were implemented as
SQL-body builtins whose Postgres-correct shadow check ("is this the first object
of its name on the search path?") scanned pg_proc / pg_class / pg_type. Because
those virtual tables are indexed only by OID, the shadow check forced a full
materialization of the catalog on every call: O(catalog) per call, and O(N^2)
under the per-row introspection pattern. This made catalog introspection very
slow and was the cause of the TestDescribe timeout in cockroachdb#171054.

This commit reimplements the three builtins in Go (pkg/sql/pg_is_visible.go),
exposed through new eval.Planner methods. Instead of scanning the virtual
catalogs, each builtin resolves the object's name and schema from its OID and
walks the current search path doing leased-cache-backed lookups, mirroring
Postgres's RelationIsVisible / TypeIsVisible / FunctionIsVisible:

* pg_function_is_visible resolves the OID's name and signature, then uses the
  normal function resolver to find the candidate overloads and checks that the
  first one (by search-path precedence) with matching argument types is the
  target. This preserves Postgres's signature-aware behavior: same-named
  functions with disjoint argument lists do not shadow each other.
* pg_table_is_visible and pg_type_is_visible walk the search path and check, per
  schema, whether an object of the same name exists, stopping at the first match.

The per-call cost drops from a full catalog populate to a handful of
descriptor/namespace lookups, eliminating the quadratic blow-up.

Behavior is unchanged from the SQL implementation: visibility, cross-schema
shadowing, signature-aware function visibility, NULL for a non-existent OID, and
NULL for objects in another database all match. A few cases are handled
explicitly: relations and types in other databases (which have no row in the
per-database virtual catalogs) return NULL; virtual tables (e.g.
pg_catalog.pg_class) are resolved through pg_class so they remain visible; and
index entries, whose hashed OIDs cannot be reversed to a descriptor, fall back
to a pg_class lookup.

This restores the Go-builtin approach used before acf5006, which had moved
these to SQL bodies for a performance win that later regressed once the bodies
were made shadow-aware.

Resolves: cockroachdb#171054
Epic: none

Release note: None

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
@trunk-io

trunk-io Bot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@cockroach-teamcity

Copy link
Copy Markdown
Member

This change is Reviewable

@rafiss rafiss closed this Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cli/clisqlshell: TestDescribe failed

2 participants