Skip to content
This repository was archived by the owner on May 5, 2025. It is now read-only.

Commit 21d5f8c

Browse files
antoinecoJeffNeff
andcommitted
Document Google Cloud Storage Source
Co-authored-by: Jeffrey Naef <[email protected]>
1 parent 483c996 commit 21d5f8c

File tree

9 files changed

+119
-0
lines changed

9 files changed

+119
-0
lines changed
10.6 KB
Loading
37.5 KB
Loading
48.6 KB
Loading
20.7 KB
Loading
46.8 KB
Loading
37.9 KB
Loading
35.7 KB
Loading

docs/sources/googlecloudstorage.md

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
# Event Source for Google Cloud Storage
2+
3+
This event source receives change notifications emerging from a [Google Cloud Storage][gc-storage] bucket by subscribing
4+
to a [Google Cloud Pub/Sub][gc-storage-events] topic.
5+
6+
## Prerequisites
7+
8+
### Storage Bucket
9+
10+
The storage bucket is expected to exist prior to running this event source. If you haven't already done so, create a
11+
bucket by following the instructions from the [Cloud Storage How-To Guides][gc-storage-howto].
12+
13+
### Pub/Sub Topic and Subscription
14+
15+
A Pub/Sub topic is required to be able to consume change notifications from Cloud Storage.
16+
17+
You can either create this topic manually by following the instructions at [Managing topics and
18+
subscriptions][gc-pubsub-adm], or instead use the [`gsutil`][gsutil] command-line tool to create the Pub/Sub topic and
19+
add a notification configuration to the bucket in a single step, as described at [Using Pub/Sub notifications for Cloud
20+
Storage][gc-storage-setup].
21+
22+
![Topic](../images/googlecloudstorage-source/topic-1.png)
23+
24+
Once a topic exists, it is necessary to create a _subscription_ in order to allow the TriggerMesh event source for
25+
Google Cloud Storage to pull events from this topic. The documentation page [Managing topics and
26+
subscriptions][gc-pubsub-adm] explains this procedure in details.
27+
28+
Take note of the name of this subscription, it is a required input to be able to run an instance of the Google Cloud
29+
Storage event source.
30+
31+
![Subscription](../images/googlecloudstorage-source/subscription-1.png)
32+
33+
### Notification Configuration
34+
35+
Change notifications need to be enabled in the selected bucket by applying a _notification configuration_.
36+
37+
If you have already followed the instructions from the previous section and applied a notification configuration at the
38+
same time as you created the Pub/Sub topic using `gsutil`, your bucket is already configured for sending events.
39+
Otherwise, follow the instructions at [Using Pub/Sub notifications for Cloud Storage][gc-storage-setup] and add a new
40+
notification configuration.
41+
42+
Below is an example of command which applies a notification configuration to a bucket called `eventsource-dev`, with a
43+
Pub/Sub topic called `triggermesh-storage-source` set as event destination.
44+
45+
```console
46+
$ gsutil notification create -t triggermesh-storage-source -f json gs://eventsource-dev
47+
Created Cloud Pub/Sub topic projects/my-project/topics/triggermesh-storage-source
48+
Created notification config projects/_/buckets/eventsource-dev/notificationConfigs/1
49+
```
50+
51+
```console
52+
$ gsutil notification list gs://eventsource-dev
53+
projects/_/buckets/eventsource-dev/notificationConfigs/1
54+
Cloud Pub/Sub topic: projects/my-project/topics/triggermesh-storage-source
55+
```
56+
57+
!!! note
58+
It is currently not possible to manage Pub/Sub notifications via the Cloud Console.
59+
60+
### Service Account
61+
62+
A [Service Account][gc-storage-svcacc] is required to authenticate the event source and allow it to interact with Google
63+
Cloud Pub/Sub. You can create a service account by following the instructions at [Creating and managing service
64+
accounts][gc-iam-svcacc].
65+
66+
The service account must be granted an [IAM Role][gc-iam-roles] with at least the following permission:
67+
68+
- `pubsub.subscriptions.consume`
69+
70+
The predefined `roles/pubsub.subscriber` role is one example of role that is suitable for use with the TriggerMesh event
71+
source for Google Cloud Storage.
72+
73+
![Service account](../images/googlecloudstorage-source/iam-1.png)
74+
75+
Create a [key in JSON format][gc-iam-key] for this service account and save it. It is required to be able to run an
76+
instance of the Google Cloud Storage event source.
77+
78+
## Deploying an Instance of the Source
79+
80+
Open the Bridge creation screen and add a source of type `Google Cloud Storage`.
81+
82+
![Adding a Google Cloud Storage source](../images/googlecloudstorage-source/create-bridge-1.png)
83+
84+
In the Source creation form, give a name to the event source and add the following information:
85+
86+
* [**Secret**][gc-iam-key]: Service account key in JSON format, as described in the previous sections.
87+
* **Google Cloud project name**: Name of the Google Cloud project containing the storage bucket.
88+
* **Pub/Sub Subscription name**: Name of a Pub/Sub subscription which pulls messages from the topic where the Cloud
89+
Storage bucket sends its change notifications.
90+
91+
![Google Cloud Storage source form](../images/googlecloudstorage-source/create-bridge-2.png)
92+
93+
After clicking the `Save` button, you will be taken back to the Bridge editor. Proceed to adding the remaining
94+
components to the Bridge, then submit it.
95+
96+
![Bridge overview](../images/googlecloudstorage-source/create-bridge-3.png)
97+
98+
A ready status on the main _Bridges_ page indicates that the event source is ready to consume Cloud Storage
99+
notifications from the Pub/Sub topic.
100+
101+
![Bridge status](../images/googlecloudstorage-source/create-bridge-4.png)
102+
103+
## Event Types
104+
105+
The TriggerMesh event source for Google Cloud Storage emits events of the following type:
106+
107+
* `com.triggermesh.googlecloudstorage.event`
108+
109+
[gc-storage]: https://cloud.google.com/storage/docs/introduction
110+
[gc-storage-events]: https://cloud.google.com/storage/docs/pubsub-notifications
111+
[gc-storage-howto]: https://cloud.google.com/storage/docs/how-to
112+
[gc-storage-svcacc]: https://cloud.google.com/storage/docs/projects#service-accounts
113+
[gc-storage-setup]: https://cloud.google.com/storage/docs/reporting-changes
114+
[gc-iam-svcacc]: https://cloud.google.com/iam/docs/creating-managing-service-accounts
115+
[gc-iam-key]: https://cloud.google.com/iam/docs/creating-managing-service-account-keys
116+
[gc-iam-roles]: https://cloud.google.com/iam/docs/understanding-roles
117+
[gc-pubsub-adm]: https://cloud.google.com/pubsub/docs/admin
118+
[gsutil]: https://cloud.google.com/storage/docs/gsutil

docs/sources/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ solutions on our Cloud.
1717
* [AWS SNS](./awssns.md): Subscribe to messages from a AWS SNS topic.
1818
* [AWS SQS](./awssqs.md): Capture messages from an AWS SQS queue.
1919
* [Webhook](./webhook.md): Generic Webhook integration.
20+
* [Google Cloud Storage](./googlecloudstorage.md): Capture change notifications from a Google Cloud Storage bucket.
2021
* [HTTP Poller](./httppoller.md): Generic HTTP Poller.
2122
* [OCIMetrics](./ocimetrics.md): Oracle Cloud Infrastructure Metrics
2223
* [Salesforce](./salesforce.md): Consume messages from Salesforce Streaming API.

0 commit comments

Comments
 (0)