-
Notifications
You must be signed in to change notification settings - Fork 79
update doc #2828
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
update doc #2828
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,51 +1,91 @@ | ||
| ## Collect Office365 events through Azure EventHub | ||
|
|
||
| This is the alternative way to collect Office365 events. | ||
| This is the alternative way to collect Office365 events. It is recommended when: | ||
|
|
||
| - The Management API approach is not suitable for your environment | ||
| - You are using **China tenants** (operated by 21Vianet) | ||
|
|
||
| !!! note "China tenants" | ||
| If you are using a China tenant (21Vianet), replace portal URLs as follows: | ||
|
|
||
| - Azure Portal: `https://portal.azure.cn` | ||
| - Microsoft Purview compliance portal: `https://compliance.microsoft.cn` | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| This setup guide will show you how to generate, store and forward events produced by Office 365 service to Sekoia.io. | ||
| Theses changes have to be made from the Azure web portal ([https://portal.azure.com](https://portal.azure.com)). | ||
| These changes have to be made from the Azure web portal ([https://portal.azure.com](https://portal.azure.com)). | ||
|
|
||
| ### A. Event Hubs | ||
|
|
||
| As a prerequisite you need an `Event Hub` (e.g. company-eventhub) and to choose an existing `resourceGroup` or create a new one (e.g. company-resource-group). | ||
| You also need your `Subscription ID` if you don't have a default one. | ||
|
|
||
| Navigate to: `Home > Cost Management + Billing > Subscriptions`. From there, copy the relevant `Subscription ID` that will be used in the command line (e.g. uuid) | ||
| Then you use Azure PowerShell (within Cloud Shell interface for example): you will create a global `Event Hubs`, then specific `Event Hub` (e.g. o365-event). | ||
| Navigate to: `Home > Cost Management + Billing > Subscriptions`. From there, copy the relevant `Subscription ID` that will be used in the command line (e.g. uuid). | ||
| Then use Azure PowerShell (within the Cloud Shell interface for example): you will create a global `Event Hubs` namespace, then a specific `Event Hub` (e.g. o365-event). | ||
| ```powershell | ||
| PS Azure:\> az eventhubs namespace create --name company-eventhub --resource-group company-resource-group --enable-kafka true --subscription uuid | ||
|
|
||
| PS Azure:\> az eventhubs eventhub create --resource-group company-resource-group --namespace-name company-eventhub --name o365-event --message-retention 3 --partition-count 4 --subscription uuid | ||
| ``` | ||
|
|
||
| Navigate to: `Home > Event Hubs > company-eventhub - Shared access policies`. From there, you can create a policy (e.g. RootManageSharedAccessKey) with the claims `Manage`, `Send` and `Listen`, and note the `Primary Key` that will be used as the `SharedAccessKey`. | ||
| Navigate to: `Home > Event Hubs > company-eventhub > o365-event - Shared access policies`. From there, you can create a policy (e.g. sekoiaio-nifi) with the claims `Listen`. | ||
| > Once created, click on the policy and save the `Connection string-primary key`, to be sent to Sekoia.io. | ||
| Navigate to: `Home > Event Hubs > company-eventhub > o365-event - Consumer groups`. From there, you can create a consumer group (e.g. sekoiaio-nifi). | ||
|
|
||
| ### B. Office 365 | ||
| Navigate to: `Home > Event Hubs > company-eventhub > o365-event - Shared access policies`. From there, you can create a policy (e.g. sekoiaio) with the claim `Listen`. | ||
| > Once created, click on the policy and save the `Connection string-primary key` — it will be used when configuring the Sekoia.io playbook. | ||
|
|
||
|
Comment on lines
32
to
+36
|
||
| Navigate to: `Home > Event Hubs > company-eventhub > o365-event - Consumer groups`. From there, create a consumer group (e.g. sekoiaio). | ||
|
|
||
| Office 365 has to be added through Azure portal following the [Microsoft documentation](https://docs.microsoft.com/en-us/azure/azure-monitor/insights/solution-office-365) | ||
| ### B. Stream M365 Unified Audit logs (Exchange, SharePoint, General) | ||
|
|
||
| Then you need to activate and configure the Office 365 diagnostic settings. | ||
| Navigate to: `Home > Office 365 > Monitoring > Diagnostic settings`: | ||
| - Add a new diagnostic setting, and select `Stream to an event hub`and click on configure. | ||
| - Select the previously created `Event hubs`, `Event Hub` and `SharedAccessKey`. | ||
| - Choose a name for this configuration and click on `Save`. | ||
| Exchange, SharePoint and General audit events are part of the **M365 Unified Audit Log**. They are **not** configured in the Azure Portal but in the **Microsoft Purview compliance portal**. | ||
|
|
||
| !!! note | ||
| A single streaming configuration in Microsoft Purview covers all audit log categories (Exchange, SharePoint, Teams, OneDrive, General, etc.) at once. No separate configuration per category is required. | ||
|
|
||
| 1. Go to the [Microsoft Purview compliance portal](https://compliance.microsoft.com) (`https://compliance.microsoft.cn` for China tenants) | ||
| 2. Navigate to `Audit > Audit log streaming` | ||
| 3. Click `Turn on streaming` (if not already enabled) | ||
| 4. Select `Stream to Azure Event Hubs` as the destination | ||
| 5. Enter the Event Hub namespace connection string and the Event Hub name created in section A | ||
| 6. Save the configuration | ||
|
|
||
| > If audit logging is not yet enabled for your organization, refer to [Turn auditing on or off](https://docs.microsoft.com/en-us/microsoft-365/compliance/turn-audit-log-search-on-or-off). | ||
|
|
||
| ### Create the intake | ||
|
|
||
| Go to the [intake page](https://app.sekoia.io/operations/intakes), create a new intake from the format `Office365` and choose the manual way. | ||
| 1. Go to the Sekoia.io [Intakes page](https://app.sekoia.io/operations/intakes) | ||
| 2. Click `+ Intake` | ||
| 3. Search for **`Office 365`** and select the **Office 365** format | ||
| 4. Give it a name and assign it to the relevant Entity | ||
| 5. When prompted for the configuration mode, select **`Manual`** — this skips the OAuth/Management API flow and provides a raw Intake Key | ||
| 6. Copy the **Intake Key** — it will be used in the playbook below | ||
|
|
||
| ### Pull events | ||
|
|
||
| Go to the [playbook page](https://app.sekoia.io/operations/playbooks) and create a new playbook with the `Consume Eventhub messages`. | ||
| 1. Go to the [Playbooks page](https://app.sekoia.io/operations/playbooks) | ||
| 2. Create a new playbook using the **`Consume Eventhub messages`** trigger | ||
| 3. Configure the trigger with the following fields: | ||
|
|
||
| | Field | Value | | ||
| |---|---| | ||
| | **Configuration Name** | A name for this configuration | | ||
| | **categories** | Leave empty to collect all event categories, or specify a subset | | ||
| | **Chunk Size** | `1000` (default) | | ||
| | **Hub Connection String** | The `Connection string-primary key` from the `Listen` policy on your Event Hub | | ||
| | **Hub Consumer Group** | `sekoiaio` (or the consumer group created in section A) | | ||
| | **Hub Name** | The name of your Event Hub (e.g. `o365-event`) | | ||
| | **Intake Key** | The Intake Key copied from the intake created above | | ||
| | **Intake Server** | `https://intake.sekoia.io` (default) | | ||
| | **Storage Connection String** | The `Connection string-primary key` of your Azure Storage account | | ||
| | **Storage Container Name** | The name of the Azure Storage blob container | | ||
|
|
||
| Set up the trigger configuration with the EventHub's `Connection string-primary key`, the hub name, the consumer group, the storage's `Connection string-primary key` and the storage's container. | ||
| 4. Start the playbook and verify events flow into your [Events page](https://app.sekoia.io/operations/events) | ||
|
|
||
| Start the playbook and enjoy your [events](https://app.sekoia.io/operations/events). | ||
| !!! note "Multiple clients" | ||
| For each client, create a dedicated Event Hub (or separate Event Hubs within the same namespace), a dedicated Sekoia intake, and a dedicated playbook. This ensures log isolation between tenants. | ||
|
|
||
| ## Further Readings | ||
| - [Microsoft Stream Azure monitoring data to an event hub](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/stream-monitoring-data-event-hubs) | ||
| - [Microsoft: Stream Azure monitoring data to an event hub](https://docs.microsoft.com/en-us/azure/azure-monitor/platform/stream-monitoring-data-event-hubs) | ||
| - [Microsoft: Audit log streaming in Microsoft Purview](https://learn.microsoft.com/en-us/purview/audit-streaming) | ||
| - [Microsoft: Turn auditing on or off](https://docs.microsoft.com/en-us/microsoft-365/compliance/turn-audit-log-search-on-or-off) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In section A, the text says you need an
Event Hub(e.g. company-eventhub), but the commands immediately below create an Event Hubs namespace namedcompany-eventhuband then an Event Hub namedo365-event. Please adjust the wording socompany-eventhubis clearly described as the Event Hubs namespace to avoid confusion during setup.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@copilot open a new pull request to apply changes based on this feedback