SNOW-1869750: [Feature]: don't crash the Provider if the authentication is set to key-pair, but `private_key` is not specified #3322

ahsanshafiq · 2024-12-30T06:28:21Z

Terraform CLI Version

1.10.2

Terraform Provider Version

0.94.1

Company Name

No response

Terraform Configuration

terraform {
  required_providers {
    snowflake = {
      source  = "Snowflake-Labs/snowflake"
      version = "~> 0.94.1"
    }
  }
}

Object type(s)

No response

Expected Behavior

terraform apply tfplan executes without terraform provider crashing.

Actual Behavior

	github.com/jmoiron/[email protected]/sqlx.go:642 +0x48
github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk.NewClient(0x0?)
	github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk/client.go:153 +0x12f
github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/provider.ConfigureProvider(0x20e014f?)
	github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/provider/provider.go:764 +0x19d2
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Provider).Configure(0xc0009c5a40, {0x24b1278, 0xc000c7da10}, 0xc0009de910)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/provider.go:296 +0x1b8
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ConfigureProvider(0xc0007d3998, {0x24b1278?, 0xc000c7cc90?}, 0xc000c39800)
	github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:611 +0x3a9
github.com/hashicorp/terraform-plugin-mux/tf5to6server.v5tov6Server.ConfigureProvider({{0x24c7fc0?, 0xc0007d3998?}}, {0x24b1278?, 0xc000c7cc90?}, 0xc000c397a0?)
	github.com/hashicorp/[email protected]/tf5to6server/tf5to6server.go:86 +0x182
github.com/hashicorp/terraform-plugin-mux/tf6muxserver.(*muxServer).ConfigureProvider(0xc000ac4460, {0x24b1278?, 0xc000c7c990?}, 0xc000c7c960?)
	github.com/hashicorp/[email protected]/tf6muxserver/mux_server_ConfigureProvider.go:28 +0x14f
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).ConfigureProvider(0xc000842320, {0x24b1278?, 0xc000c7c150?}, 0xc000b4f100)
	github.com/hashicorp/[email protected]/tfprotov6/tf6server/server.go:558 +0x2db
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ConfigureProvider_Handler({0x20639e0?, 0xc000842320}, {0x24b1278, 0xc000c7c150}, 0xc0009c6380, 0x0)
	github.com/hashicorp/[email protected]/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:464 +0x169
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0009fc000, {0x24b1278, 0xc000c7c0c0}, {0x24c31e0, 0xc000002000}, 0xc0006785a0, 0xc000b59d70, 0x348acf0, 0x0)
	google.golang.org/[email protected]/server.go:1369 +0xe23
google.golang.org/grpc.(*Server).handleStream(0xc0009fc000, {0x24c31e0, 0xc000002000}, 0xc0006785a0)
	google.golang.org/[email protected]/server.go:1780 +0x1016
google.golang.org/grpc.(*Server).serveStreams.func2.1()
	google.golang.org/[email protected]/server.go:1019 +0x8b
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 16
	google.golang.org/[email protected]/server.go:1030 +0x135

Error: The terraform-provider-snowflake_v0.94.1 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Steps to Reproduce

In Azure pipeline, apply changes using the tfplan file generated in a earlier job, within the same stage.

How much impact is this issue causing?

High

Logs

No response

Additional Information

I am executing terraform apply tfplan in a downstream Azure pipeline job where tfplan was created in an earlier job.

Would you like to implement a fix?

Yeah, I'll take it 😎

The text was updated successfully, but these errors were encountered:

sfc-gh-dszmolka · 2024-12-30T07:55:20Z

hi - thanks for reporting this issue with us. Looks like the provider is crashing when trying to create a connection. To aid troubleshooting, could you please provide a bit more information about the issue ?

is this the first time you're trying out the provider or it was always working with the same version using the same resources, just now started crashing ?
would you be able to send us the .tf files you're using, or even better, create a reproduction and send it , which reproduction is representative of what you're doing and when executed, exhibits the same behaviour you're seeing?
would you be able to try and upgrade to v0.100.0 of the provider or even the stable v1.0.0, and use the same .tf. files and resources, and see if it still crashes?

Adding TF_LOG=DEBUG to the runtime envvars also could help by generating verbose logs which could help pinpointing the problem.

Any detail helps and thank you in advance for providing them to us !

ahsanshafiq · 2024-12-31T05:32:22Z

Thanks a lot for your prompt response. When you said "Looks like the provider is crashing when trying to create a connection.", I looked at my configurations and found out that the environment variable for private key i.e. SNOWFLAKE_PRIVATE_KEY was not set. After setting this, the issue is resolved.

sfc-gh-dszmolka · 2024-12-31T08:12:09Z

i'm glad to hear you're unblocked now and could find the source of the issue - thank you for testing it quickly and letting is know !

i'm going to transform this Issue into an enhancement, because i would like to reproduce it for myself and also see if we can do this differently instead of mysteriously panicking when SNOWFLAKE_PRIVATE_KEY is not specified.
Again, thank you for this feedback!

sfc-gh-dszmolka · 2024-12-31T10:06:42Z

full panic stack, which clearly points to where the issue is happening (when gosnowflake tries to create a JWT from the key provided). Putting it here for folks to find it easier and have a pointer to where the issue is.

It is also an issue in the latest v1.0.1 provider, sadly.

Stack trace from the terraform-provider-snowflake plugin:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x180610c]

goroutine 13 [running]:
crypto/rsa.(*PrivateKey).Public(...)
	/usr/local/go/src/crypto/rsa/rsa.go:126
github.com/snowflakedb/gosnowflake.prepareJWTToken(0xc0006f2800)
	/root/go/pkg/mod/github.com/snowflakedb/[email protected]/auth.go:476 +0x2c
github.com/snowflakedb/gosnowflake.createRequestBody(0xc0000a8d80, 0xc00014be80?, {{0xc00096ab3e, 0x1c}, {0x23336c9, 0x5}, {0xc00068dd28, 0x8}, {0x23464d3, 0x9}, ...}, ...)
	/root/go/pkg/mod/github.com/snowflakedb/[email protected]/auth.go:439 +0x489
github.com/snowflakedb/gosnowflake.authenticate.func1()
	/root/go/pkg/mod/github.com/snowflakedb/[email protected]/auth.go:339 +0x65
github.com/snowflakedb/gosnowflake.(*retryHTTP).execute(0xc00120bfc0)
	/root/go/pkg/mod/github.com/snowflakedb/[email protected]/retry.go:310 +0x20a
github.com/snowflakedb/gosnowflake.postAuthRestful({0x2780900?, 0x3929700?}, 0x2351f3a?, 0xc000811418?, 0xc00120c188?, 0x30?, 0x2c?, 0x30?)
	/root/go/pkg/mod/github.com/snowflakedb/[email protected]/restful.go:199 +0xef
github.com/snowflakedb/gosnowflake.postAuth({0x2780900, 0x3929700}, 0xc0005a7860, 0xc000dc8f60, 0xc000811418, 0xc000dc9020, 0xc0005cd0e0, 0x45d964b800)
	/root/go/pkg/mod/github.com/snowflakedb/[email protected]/auth.go:231 +0x363
github.com/snowflakedb/gosnowflake.authenticate({0x2780900, 0x3929700}, 0xc0000a8d80, {0x0, 0x0, 0x0}, {0x0, 0x0, 0x0})
	/root/go/pkg/mod/github.com/snowflakedb/[email protected]/auth.go:359 +0x11e8
github.com/snowflakedb/gosnowflake.authenticateWithConfig(0xc0000a8d80)
	/root/go/pkg/mod/github.com/snowflakedb/[email protected]/auth.go:553 +0x33e
github.com/snowflakedb/gosnowflake.SnowflakeDriver.OpenWithConfig({}, {_, _}, {{0xc00096ab07, 0xf}, {0xc00096ab00, 0x5}, {0xc00096ab00, 0x0}, {0x0, ...}, ...})
	/root/go/pkg/mod/github.com/snowflakedb/[email protected]/driver.go:44 +0x165
github.com/snowflakedb/gosnowflake.SnowflakeDriver.Open({}, {0xc00096ab00, 0xa7})
	/root/go/pkg/mod/github.com/snowflakedb/[email protected]/driver.go:27 +0x106
github.com/luna-duclos/instrumentedsql.dsnConnector.Connect(...)
	/root/go/pkg/mod/github.com/luna-duclos/[email protected]/connector.go:53
github.com/luna-duclos/instrumentedsql.wrappedConnector.Connect({{{0x2764b60, 0x2469ac8}, {0x27695a0, 0x3929700}, 0x0, 0x0}, {0x2770018, 0xc000fa0a40}, 0xc00086cf40}, {0x2780900, ...})
	/root/go/pkg/mod/github.com/luna-duclos/[email protected]/connector.go:33 +0x1f3
database/sql.(*DB).conn(0xc0005a75f0, {0x2780900, 0x3929700}, 0x1)
	/usr/local/go/src/database/sql/sql.go:1415 +0x71e
database/sql.(*DB).PingContext.func1(0x46?)
	/usr/local/go/src/database/sql/sql.go:883 +0x3a
database/sql.(*DB).retry(0xc000dc8db0?, 0xc00120d0a0)
	/usr/local/go/src/database/sql/sql.go:1566 +0x42
database/sql.(*DB).PingContext(0xc0005a75f0, {0x2780900, 0x3929700})
	/usr/local/go/src/database/sql/sql.go:882 +0x89
database/sql.(*DB).Ping(...)
	/usr/local/go/src/database/sql/sql.go:900
github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/sdk.NewClient(0xc0006f2200?)
	/test/localprovider/terraform-provider-snowflake/pkg/sdk/client.go:158 +0x1ce
github.com/Snowflake-Labs/terraform-provider-snowflake/pkg/provider.ConfigureProvider({0x2336d64?, 0x6?}, 0xc000886e80)
	/test/localprovider/terraform-provider-snowflake/pkg/provider/provider.go:555 +0x208
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Provider).Configure(0xc0000f3da0, {0x2780938, 0xc000dc80f0}, 0xc000884640)
	/root/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/provider.go:306 +0x20d
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ConfigureProvider(0xc00060c2d0, {0x2780938?, 0xc000f853e0?}, 0xc000fa4438)
	/root/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:611 +0x39b
github.com/hashicorp/terraform-plugin-mux/tf5to6server.v5tov6Server.ConfigureProvider({{0x279b680?, 0xc00060c2d0?}}, {0x2780938?, 0xc000f853e0?}, 0xc000fa4408?)
	/root/go/pkg/mod/github.com/hashicorp/[email protected]/tf5to6server/tf5to6server.go:86 +0x182
github.com/hashicorp/terraform-plugin-mux/tf6muxserver.(*muxServer).ConfigureProvider(0xc000b54e70, {0x2780938?, 0xc000f85110?}, 0xc000fa4408)
	/root/go/pkg/mod/github.com/hashicorp/[email protected]/tf6muxserver/mux_server_ConfigureProvider.go:28 +0x134
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).ConfigureProvider(0xc000a6b720, {0x2780938?, 0xc000f848d0?}, 0xc00086c580)
	/root/go/pkg/mod/github.com/hashicorp/[email protected]/tfprotov6/tf6server/server.go:558 +0x2ca
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ConfigureProvider_Handler({0x22be740, 0xc000a6b720}, {0x2780938, 0xc000f848d0}, 0xc000886300, 0x0)
	/root/go/pkg/mod/github.com/hashicorp/[email protected]/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:464 +0x1a6
google.golang.org/grpc.(*Server).processUnaryRPC(0xc000b28800, {0x2780938, 0xc000f84840}, {0x2795a60, 0xc0000da300}, 0xc000db0000, 0xc000e4d170, 0x38af190, 0x0)
	/root/go/pkg/mod/google.golang.org/[email protected]/server.go:1369 +0xdf8
google.golang.org/grpc.(*Server).handleStream(0xc000b28800, {0x2795a60, 0xc0000da300}, 0xc000db0000)
	/root/go/pkg/mod/google.golang.org/[email protected]/server.go:1780 +0xe8b
google.golang.org/grpc.(*Server).serveStreams.func2.1()
	/root/go/pkg/mod/google.golang.org/[email protected]/server.go:1019 +0x8b
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 12
	/root/go/pkg/mod/google.golang.org/[email protected]/server.go:1030 +0x125

Error: The terraform-provider-snowflake plugin crashed!

as mentioned issue comes from here https://github.com/snowflakedb/gosnowflake/blob/v1.10.0/auth.go#L476 when gosnowflake tries to read the private key - but in this case the private key is nil so the Public() function panics

	pubBytes, err := x509.MarshalPKIXPublicKey(config.PrivateKey.Public())

Working on a fix.

sfc-gh-dszmolka · 2024-12-31T10:42:59Z

proposed a PR for the underlying gosnowflake driver (snowflakedb/gosnowflake#1285) so it could error out gracefully (with a descriptive error message) instead of panicking.

sfc-gh-dszmolka · 2025-01-16T13:28:32Z

the gosnowflake side change is merged, and will be part of the next gosnowflake release.
after its release, and when the Provider is rebased onto that specific release, will be available here too.

Therefore, keeping this one open for a bit until a Provider version is out which has the gosnowflake with the enhancement.

sfc-gh-dszmolka · 2025-02-05T14:06:07Z

fix released with gosnowflake 1.13.0, after the Provider is rebased to such version or higher, will be available in the subsequent Provider release

- Upgrade libraries, tools, and Go version to 1.22. - Add a note about the new field in the driver. - Ignore linter warnings about using the new field (this will be continued in SNOW-1917271). - Replace `ConcatSlices` with `slices.Concat`. - Exclude a listing rule for the generated structs. - Remove `avast/retry-go` library. Use our own solution instead (it was already used in other places before). - Remove `golangci-lint` from tools. It is not needed, because of a custom installation in the Makefile. - Increase a warehouse test timeout. - Fix tests of granting all streamlits and add a note to the docs about the grants. - Remove the deprecated linter.  ## Test Plan  * [ ] acceptance tests  * [ ] … ## References  #3322 #3316 ## TODO - Upgrade Go to 1.23 and 1.24 when it's available - Close relevant dependabot PRs.

sfc-gh-jmichalak · 2025-02-27T10:17:04Z

Hi @ahsanshafiq,

We've released v1.0.4 (release, migration guide) with the updated driver to v1.13.0.

sfc-gh-jmichalak · 2025-03-21T11:43:16Z

Closing due to inactivity.

ahsanshafiq added the bug Reproducible issue with the provider or Snowflake behavior. label Dec 30, 2024

ahsanshafiq changed the title ~~[Bug]:~~ [Bug]: The terraform-provider-snowflake_v0.94.1 plugin crashed! Dec 30, 2024

sfc-gh-dszmolka self-assigned this Dec 30, 2024

sfc-gh-dszmolka added the status-triage Issue is under initial triage label Dec 30, 2024

sfc-gh-dszmolka added the status-information_needed Additional information is required from the reporter label Dec 30, 2024

sfc-gh-dszmolka changed the title ~~[Bug]: The terraform-provider-snowflake_v0.94.1 plugin crashed!~~ SNOW-1869750: [Bug]: The terraform-provider-snowflake_v0.94.1 plugin crashed! Dec 30, 2024

sfc-gh-dszmolka added the status-in_progress Issue is worked on by the dev team label Dec 31, 2024

sfc-gh-dszmolka mentioned this issue Dec 31, 2024

SNOW-1869750 add check for empty private key before trying to generate JWT from it snowflakedb/gosnowflake#1285

Merged

sfc-gh-dszmolka changed the title ~~SNOW-1869750: [Bug]: The terraform-provider-snowflake_v0.94.1 plugin crashed!~~ SNOW-1869750: [Feature]: don't panic the Provider if the authentication is set to key-pair, but private_key is not specified Jan 2, 2025

sfc-gh-dszmolka added status-fixed_awaiting_release The issue has been fixed, its PR merged, and now awaiting the next release cycle of the connector. and removed status-in_progress Issue is worked on by the dev team labels Jan 16, 2025

sfc-gh-jmichalak mentioned this issue Feb 7, 2025

chore: Upgrade deps #3389

Merged

2 tasks

sfc-gh-jmichalak closed this as completed Mar 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SNOW-1869750: [Feature]: don't crash the Provider if the authentication is set to key-pair, but `private_key` is not specified #3322

SNOW-1869750: [Feature]: don't crash the Provider if the authentication is set to key-pair, but `private_key` is not specified #3322

ahsanshafiq commented Dec 30, 2024

sfc-gh-dszmolka commented Dec 30, 2024 •

edited

Loading

ahsanshafiq commented Dec 31, 2024

sfc-gh-dszmolka commented Dec 31, 2024

sfc-gh-dszmolka commented Dec 31, 2024 •

edited

Loading

sfc-gh-dszmolka commented Dec 31, 2024

sfc-gh-dszmolka commented Jan 16, 2025

sfc-gh-dszmolka commented Feb 5, 2025 •

edited

Loading

sfc-gh-jmichalak commented Feb 27, 2025

sfc-gh-jmichalak commented Mar 21, 2025

SNOW-1869750: [Feature]: don't crash the Provider if the authentication is set to key-pair, but private_key is not specified #3322

SNOW-1869750: [Feature]: don't crash the Provider if the authentication is set to key-pair, but private_key is not specified #3322

Comments

ahsanshafiq commented Dec 30, 2024

Terraform CLI Version

Terraform Provider Version

Company Name

Terraform Configuration

Category

Object type(s)

Expected Behavior

Actual Behavior

Steps to Reproduce

How much impact is this issue causing?

Logs

Additional Information

Would you like to implement a fix?

sfc-gh-dszmolka commented Dec 30, 2024 • edited Loading

ahsanshafiq commented Dec 31, 2024

sfc-gh-dszmolka commented Dec 31, 2024

sfc-gh-dszmolka commented Dec 31, 2024 • edited Loading

sfc-gh-dszmolka commented Dec 31, 2024

sfc-gh-dszmolka commented Jan 16, 2025

sfc-gh-dszmolka commented Feb 5, 2025 • edited Loading

sfc-gh-jmichalak commented Feb 27, 2025

sfc-gh-jmichalak commented Mar 21, 2025

SNOW-1869750: [Feature]: don't crash the Provider if the authentication is set to key-pair, but `private_key` is not specified #3322

SNOW-1869750: [Feature]: don't crash the Provider if the authentication is set to key-pair, but `private_key` is not specified #3322

sfc-gh-dszmolka commented Dec 30, 2024 •

edited

Loading

sfc-gh-dszmolka commented Dec 31, 2024 •

edited

Loading

sfc-gh-dszmolka commented Feb 5, 2025 •

edited

Loading