Merge pull request #2976 from redpanda-data/ss/spicedb-plugin

spicedb: add spicedb watch input

Author: simon0191

CHANGELOG.md

@@ -13,6 +13,7 @@ All notable changes to this project will be documented in this file.
 - (Benthos) Field `follow_redirects` added to the `http` processor. (@ooesili)
 - New CLI flag `--secrets` added. (@Jeffail)
 - New CLI flag `--disable-telemetry` added. (@Jeffail)
+- New experimental `spicedb` watch input.
 
 ### Fixed
 

docs/modules/components/pages/inputs/spicedb_watch.adoc

@@ -0,0 +1,315 @@
+= spicedb_watch
+:type: input
+:status: stable
+:categories: ["Services","SpiceDB"]
+
+
+
+////
+     THIS FILE IS AUTOGENERATED!
+
+     To make changes, edit the corresponding source file under:
+
+     https://github.com/redpanda-data/connect/tree/main/internal/impl/<provider>.
+
+     And:
+
+     https://github.com/redpanda-data/connect/tree/main/cmd/tools/docs_gen/templates/plugin.adoc.tmpl
+////
+
+// © 2024 Redpanda Data Inc.
+
+
+component_type_dropdown::[]
+
+
+Consume messages from the Watch API from SpiceDB.
+
+
+[tabs]
+======
+Common::
++
+--
+
+```yml
+# Common config fields, showing default values
+input:
+  label: ""
+  spicedb_watch:
+    endpoint: grpc.authzed.com:443 # No default (required)
+    bearer_token: ""
+    cache: "" # No default (required)
+```
+
+--
+Advanced::
++
+--
+
+```yml
+# All config fields, showing default values
+input:
+  label: ""
+  spicedb_watch:
+    endpoint: grpc.authzed.com:443 # No default (required)
+    bearer_token: ""
+    max_receive_message_bytes: 4MB
+    cache: "" # No default (required)
+    cache_key: authzed.com/spicedb/watch/last_zed_token
+    tls:
+      enabled: false
+      skip_cert_verify: false
+      enable_renegotiation: false
+      root_cas: ""
+      root_cas_file: ""
+      client_certs: []
+```
+
+--
+======
+
+The SpiceDB input allows you to consume messages from the Watch API of a SpiceDB instance.
+This input is useful for applications that need to react to changes in the data managed by SpiceDB in real-time.
+
+== Credentials
+
+You need to provide the endpoint of your SpiceDB instance and a Bearer token for authentication.
+
+== Cache
+
+The zed token of the newest update consumed and acked is stored in a cache in order to start reading from it each time the input is initialised.
+Ideally this cache should be persisted across restarts.
+
+
+== Fields
+
+=== `endpoint`
+
+The SpiceDB endpoint.
+
+
+*Type*: `string`
+
+
+```yml
+# Examples
+
+endpoint: grpc.authzed.com:443
+```
+
+=== `bearer_token`
+
+The SpiceDB Bearer token used to authenticate against the SpiceDB instance.
+[CAUTION]
+====
+This field contains sensitive information that usually shouldn't be added to a config directly, read our xref:configuration:secrets.adoc[secrets page for more info].
+====
+
+
+
+*Type*: `string`
+
+*Default*: `""`
+
+```yml
+# Examples
+
+bearer_token: t_your_token_here_1234567deadbeef
+```
+
+=== `max_receive_message_bytes`
+
+Maximum message size in bytes the SpiceDB client can receive.
+
+
+*Type*: `string`
+
+*Default*: `"4MB"`
+
+```yml
+# Examples
+
+max_receive_message_bytes: 100MB
+
+max_receive_message_bytes: 50mib
+```
+
+=== `cache`
+
+A cache resource to use for performing unread message backfills, the ID of the last message received will be stored in this cache and used for subsequent requests.
+
+
+*Type*: `string`
+
+
+=== `cache_key`
+
+The key identifier used when storing the ID of the last message received.
+
+
+*Type*: `string`
+
+*Default*: `"authzed.com/spicedb/watch/last_zed_token"`
+
+=== `tls`
+
+Custom TLS settings can be used to override system defaults.
+
+
+*Type*: `object`
+
+
+=== `tls.enabled`
+
+Whether custom TLS settings are enabled.
+
+
+*Type*: `bool`
+
+*Default*: `false`
+
+=== `tls.skip_cert_verify`
+
+Whether to skip server side certificate verification.
+
+
+*Type*: `bool`
+
+*Default*: `false`
+
+=== `tls.enable_renegotiation`
+
+Whether to allow the remote server to repeatedly request renegotiation. Enable this option if you're seeing the error message `local error: tls: no renegotiation`.
+
+
+*Type*: `bool`
+
+*Default*: `false`
+Requires version 3.45.0 or newer
+
+=== `tls.root_cas`
+
+An optional root certificate authority to use. This is a string, representing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate.
+[CAUTION]
+====
+This field contains sensitive information that usually shouldn't be added to a config directly, read our xref:configuration:secrets.adoc[secrets page for more info].
+====
+
+
+
+*Type*: `string`
+
+*Default*: `""`
+
+```yml
+# Examples
+
+root_cas: |-
+  -----BEGIN CERTIFICATE-----
+  ...
+  -----END CERTIFICATE-----
+```
+
+=== `tls.root_cas_file`
+
+An optional path of a root certificate authority file to use. This is a file, often with a .pem extension, containing a certificate chain from the parent trusted root certificate, to possible intermediate signing certificates, to the host certificate.
+
+
+*Type*: `string`
+
+*Default*: `""`
+
+```yml
+# Examples
+
+root_cas_file: ./root_cas.pem
+```
+
+=== `tls.client_certs`
+
+A list of client certificates to use. For each certificate either the fields `cert` and `key`, or `cert_file` and `key_file` should be specified, but not both.
+
+
+*Type*: `array`
+
+*Default*: `[]`
+
+```yml
+# Examples
+
+client_certs:
+  - cert: foo
+    key: bar
+
+client_certs:
+  - cert_file: ./example.pem
+    key_file: ./example.key
+```
+
+=== `tls.client_certs[].cert`
+
+A plain text certificate to use.
+
+
+*Type*: `string`
+
+*Default*: `""`
+
+=== `tls.client_certs[].key`
+
+A plain text certificate key to use.
+[CAUTION]
+====
+This field contains sensitive information that usually shouldn't be added to a config directly, read our xref:configuration:secrets.adoc[secrets page for more info].
+====
+
+
+
+*Type*: `string`
+
+*Default*: `""`
+
+=== `tls.client_certs[].cert_file`
+
+The path of a certificate to use.
+
+
+*Type*: `string`
+
+*Default*: `""`
+
+=== `tls.client_certs[].key_file`
+
+The path of a certificate key to use.
+
+
+*Type*: `string`
+
+*Default*: `""`
+
+=== `tls.client_certs[].password`
+
+A plain text password for when the private key is password encrypted in PKCS#1 or PKCS#8 format. The obsolete `pbeWithMD5AndDES-CBC` algorithm is not supported for the PKCS#8 format.
+
+Because the obsolete pbeWithMD5AndDES-CBC algorithm does not authenticate the ciphertext, it is vulnerable to padding oracle attacks that can let an attacker recover the plaintext.
+[CAUTION]
+====
+This field contains sensitive information that usually shouldn't be added to a config directly, read our xref:configuration:secrets.adoc[secrets page for more info].
+====
+
+
+
+*Type*: `string`
+
+*Default*: `""`
+
+```yml
+# Examples
+
+password: foo
+
+password: ${KEY_PASSWORD}
+```
+
+