Skip to content

include synonyms in odbcListObjects() output #773

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 13 commits into
base: main
Choose a base branch
from
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

S3method(odbcListColumns,OdbcConnection)
S3method(odbcListObjectTypes,default)
S3method(odbcListObjects,"Microsoft SQL Server")
S3method(odbcListObjects,OdbcConnection)
S3method(odbcPreviewObject,OdbcConnection)
export(databricks)
Expand Down
4 changes: 4 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@
* The `"OdbcConnection"` method for `dbQuoteIdentifier()` will no longer
pass `x` to `encodeString()` before returning, for consistency with the
default implementation in DBI (@simonpcouch, #765).

* `dbListTables()`, `dbExistsTable()`, and `odbcListObjects()` now support
synonyms with Microsoft SQL Server. This also means that users can now
interact with synonyms using the Connections pane (@simonpcouch, #773).

# odbc 1.4.2

Expand Down
77 changes: 72 additions & 5 deletions R/driver-sql-server.R
Original file line number Diff line number Diff line change
Expand Up @@ -73,11 +73,15 @@ setMethod("dbExistsTable", c("Microsoft SQL Server", "character"),
stopifnot(length(name) == 1)
if (isTempTable(conn, name, ...)) {
query <- paste0("SELECT OBJECT_ID('tempdb..", name, "')")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it's worth checking to see if this works for synonyms too? (without the tempdb..). If so, that would substantially simplify this function, and maybe make it simpler?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@simonpcouch Did you see this comment?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did! Just haven't made a moment to come back to this PR to address this comment and run some quick benchmarks to see how it affects performance. Will re-request review then, likely tomorrow!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh sorry, I though you had re-requested a review, but I must've been misreading my notifications.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was almost able to make this work with:

    dots <- list(...)
    name_loc <- paste0(
      c(dots[["catalog_name"]], dots[["schema_name"]], name),
      collapse = "."
    )
    return(!is.na(dbGetQuery(conn, paste0("SELECT OBJECT_ID('", name_loc, "')"))[[1]]))

The only issue is that OBJECT_ID() seems to only be able to work with fully qualified names. i.e.

library(DBI)
library(odbc)
con <- dbConnect(odbc(), dsn = "MicrosoftSQLServer", uid = "SA",
                 pwd = Sys.getenv("sqlServerPass"))

dbExecute(con, "create schema odbc")
#> [1] 0
dbExecute(con, "create table odbc.test (x int)")
#> [1] 0
dbExecute(con, "create synonym odbc.test2 for odbc.test")
#> [1] 0

# when fully qualified, works fine:
dbExistsTable(con, SQL("master.odbc.test2"))
#> [1] TRUE

# but not without schema/catalog specified
"test2" %in% dbListTables(con)
#> [1] TRUE
dbExistsTable(con, "test2")
#> [1] FALSE

Created on 2024-04-10 with reprex v2.1.0

Specifically, this triggered this DBItest test.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmm, is dbListTables() correct here? i.e. SELECT * FROM test2 won't work (IIUC) because it's not in the current schema. So should it actually be listed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:thinkies: The DBI docs for the generic read:

dbListTables() returns a character vector that enumerates all tables and views in the database.

so... I think it's correct that dbListTables() would list test2.


Some statements:

  1. DBI docs: dbListTables() returns a character vector that enumerates all tables and views in the database.
  2. DBItest, linked above: For all tables listed by dbListTables(), dbExistsTable() should return TRUE.
  3. Us, on slack: maybe the principle is that if SELECT * FROM foo works then dbExistsTable(con, "foo") should return TRUE?

So dbListTables() and dbExistsTable() should agree, and dbListTables() seems to be doing the right thing. The connection to SELECT * FROM foo doesn't seem to hold up, at least for SQL Server.

FWIW, with this PR:

library(DBI)
library(odbc)
con <- dbConnect(odbc(), dsn = "MicrosoftSQLServer", uid = "SA",
                 pwd = Sys.getenv("sqlServerPass"))


"test" %in% dbListTables(con)
#> [1] TRUE
dbExistsTable(con, "test")
#> [1] TRUE
dbGetQuery(con, "SELECT * FROM test")
#> Error in eval(expr, envir, enclos): nanodbc/nanodbc.cpp:1711: 00000
#> [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Invalid object name 'test'.

"test2" %in% dbListTables(con)
#> [1] TRUE
dbExistsTable(con, "test2")
#> [1] TRUE
dbGetQuery(con, "SELECT * FROM test2")
#> Error in eval(expr, envir, enclos): nanodbc/nanodbc.cpp:1711: 00000
#> [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Invalid object name 'test2'.

Created on 2024-04-10 with reprex v2.1.0

With CRAN, both "test2" calls are FALSE.

Copy link
Member

@hadley hadley Apr 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure that the DBI docs are underspecified here — I think they really should read a "character vector that enumerates all tables and views in the database for the active schema/catalog.".

So maybe we're blowing out the scope of this PR, but I think we should narrow down this behaviour because I think that we should be returning FALSE for the test calls too.

But might be easiest to discuss this live on a call with some examples?

!is.na(dbGetQuery(conn, query)[[1]])
} else {
df <- odbcConnectionTables(conn, name = name, ...)
NROW(df) > 0
return(!is.na(dbGetQuery(conn, query)[[1]]))
}
df <- odbcConnectionTables(conn, name = name, ...)
if (NROW(df) > 0) {
return(TRUE)
}

synonyms <- dbGetQuery(conn, synonyms_query(...))
name %in% synonyms$name
}
)

Expand Down Expand Up @@ -107,7 +111,9 @@ setMethod("dbListTables", "Microsoft SQL Server",
res <- c(res, res_temp)
}

res
synonyms <- dbGetQuery(conn, synonyms_query(catalog_name, schema_name))

c(res, synonyms$name)
}
)

Expand Down Expand Up @@ -204,3 +210,64 @@ setMethod("odbcDataType", "Microsoft SQL Server",
)
}
)

#' @rdname SQLServer
#' @description
#' ## `odbcListObjects()`
#'
#' This method makes tables that are synonyms visible in the Connections pane.
# See (#221).
#' @usage NULL
#' @export
`odbcListObjects.Microsoft SQL Server` <- function(connection,
catalog = NULL,
schema = NULL,
name = NULL,
type = NULL,
...) {
objects <- NextMethod(
object = connection,
catalog = catalog,
schema = schema,
name = name,
type = type,
...
)

if (!any(objects$type == "table") && nrow(objects) > 0) {
return(objects)
}

synonyms <- dbGetQuery(connection, synonyms_query(catalog, schema))

if (!is.null(name)) {
synonyms <- synonyms[synonyms$name == name, , drop = FALSE]
}

rbind(
objects,
data.frame(name = synonyms$name, type = rep("table", length(synonyms$name)))
)
}

synonyms_query <- function(catalog_name = NULL, schema_name = NULL) {
sys_synonyms <- paste0(c(catalog_name, "sys.synonyms"), collapse = ".")

res <- paste0("IF DB_ID(", sQuote(catalog_name, "'"), ") IS NOT NULL
SELECT
[schema] = SCHEMA_NAME(Syn.schema_id),
name = Syn.name
FROM ", sys_synonyms, " AS Syn")

if (is.null(schema_name)) {
return(paste0(res, " WHERE ", filter_is_table))
}

paste0(
res,
" WHERE SCHEMA_NAME(Syn.schema_id) = '", schema_name, "' AND ",
filter_is_table
)
}

filter_is_table <- "OBJECTPROPERTY(Object_ID(Syn.base_object_name), 'IsTable') = 1;"
6 changes: 6 additions & 0 deletions man/SQLServer.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 48 additions & 0 deletions tests/testthat/test-driver-sql-server.R
Original file line number Diff line number Diff line change
Expand Up @@ -337,3 +337,51 @@ test_that("captures multiline errors message", {
}
)
})

test_that("odbcListObjects shows synonyms (#221)", {
con <- test_con("SQLSERVER")
db <- dbGetQuery(con, "SELECT db_name()")[1,1]
dbExecute(con, "CREATE SCHEMA testSchema")
dbExecute(con, "CREATE SCHEMA testSchema2")

on.exit({
dbExecute(con, "DROP TABLE IF EXISTS testSchema.tbl")
dbExecute(con, "DROP SYNONYM IF EXISTS testSchema.tbl2")
dbExecute(con, "DROP SYNONYM IF EXISTS testSchema2.tbl2")
dbExecute(con, "DROP SCHEMA IF EXISTS testSchema")
dbExecute(con, "DROP SCHEMA IF EXISTS testSchema2")
})

dbExecute(con, "CREATE TABLE testSchema.tbl (x int)")
expect_equal(nrow(odbcListObjects(con, catalog = db, schema = "testSchema")), 1)

dbExecute(con, "CREATE SYNONYM testSchema.tbl2 for testSchema.tbl")
expect_equal(nrow(odbcListObjects(con, catalog = db, schema = "testSchema")), 2)
expect_length(dbListTables(con, catalog_name = db, schema_name = "testSchema"), 2)
expect_equal(
nrow(odbcListObjects(con, catalog = db, schema = "testSchema", name = "tbl")),
1
)
expect_false("tbl2" %in% odbcListObjects(con)$name)
expect_false("tbl2" %in% odbcListObjects(con, catalog = db)$name)
expect_true(
dbExistsTable(con, name = "tbl2", catalog_name = db, schema_name = "testSchema")
)
expect_true(dbExistsTable(con, name = Id(db, "testSchema", "tbl2")))
expect_true(dbExistsTable(con, name = Id("testSchema", "tbl2")))
expect_true(dbExistsTable(con, name = Id("tbl2")))
expect_true(dbExistsTable(con, name = "tbl2"))
dbExecute(con, "DROP SYNONYM testSchema.tbl2")
expect_false(dbExistsTable(con, name = Id(db, "testSchema", "tbl2")))

# ensure query filters out synonyms in schemas other than the one
# where the base object lives effectively
dbExecute(con, "CREATE SYNONYM testSchema2.tbl2 for testSchema.tbl")
expect_equal(nrow(odbcListObjects(con, catalog = db, schema = "testSchema")), 1)
expect_equal(nrow(odbcListObjects(con, catalog = db, schema = "testSchema2")), 1)
expect_length(dbListTables(con, catalog_name = db, schema_name = "testSchema2"), 1)
expect_true(
dbExistsTable(con, name = "tbl2", catalog_name = db, schema_name = "testSchema2")
)
expect_true(dbExistsTable(con, name = Id(db, "testSchema2", "tbl2")))
})
Loading