From b8052a5bcc4ab685118075bdce5791e6888d9975 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Tue, 9 Sep 2025 13:51:59 +0800 Subject: [PATCH 01/49] fix(alarm): Correct the alarm list --- en_US/observability/alarms.md | 24 +++++++++++++----------- zh_CN/observability/alarms.md | 18 ++---------------- 2 files changed, 15 insertions(+), 27 deletions(-) diff --git a/en_US/observability/alarms.md b/en_US/observability/alarms.md index ff50f2f76..e51fd059c 100644 --- a/en_US/observability/alarms.md +++ b/en_US/observability/alarms.md @@ -22,17 +22,19 @@ The levels are defined from development perspectives and are only for recommenda ::: -| **Alarm** | Level | Description | **Details** | **Threshold** | -| :------------------------ | -------- | :----------------------------------------------------------- | :------------------------------------------- | :----------------------------------------------------------- | -| high_system_memory_usage | Warning | System memory usage is too high | "System memory usage is higher than ~p%" | `os_mon.sysmem_high_watermark = 70%` | -| high_process_memory_usage | Warning | Single Erlang process memory usage is too high (percentage of system memory usage) | Process memory usage is higher than ~p% | `os_mon.procmem_high_watermark = 5%` | -| high_cpu_usage | Warning | CPU usage is too high | ~p% cpu usage | `os_mon.cpu_high_watermark = 80%` `os_mon.cpu_low_watermark = 60%` | -| too_many_processes | Warning | Too many processes | ~p% process usage | `vm_mon.process_high_watermark = 80%` `vm_mon.process_low_watermark = 60%` | -| license_quota | Warning | License exceeds quota | License: the number of connections exceeds % | `license.connection_high_watermark_alarm = 80%` `license.connection_low_watermark_alarm = 75%` | -| license_expiry | Critical | License expired | License will be expired at % | - | -| partition | Critical | Partition occurs at node | Partition occurs at node ~s | - | -| resource | Critical | Resource is disconnected | Resource ~s(~s) is down | - | -| conn_congestion | Critical | Connection process congestion | Connection congested | - | +| **Alarm** | Level | Description | **Details** | **Threshold** | +| :---------------------------------- | -------- | :----------------------------------------------------------- | :------------------------------------------- | :----------------------------------------------------------- | +| high_system_memory_usage | Warning | System memory usage is too high | "System memory usage is higher than ~p%" | `os_mon.sysmem_high_watermark = 70%` | +| high_process_memory_usage | Warning | Single Erlang process memory usage is too high (percentage of system memory usage) | Process memory usage is higher than ~p% | `os_mon.procmem_high_watermark = 5%` | +| high_cpu_usage | Warning | CPU usage is too high | ~p% cpu usage | `os_mon.cpu_high_watermark = 80%` `os_mon.cpu_low_watermark = 60%` | +| too_many_processes | Warning | Too many processes | ~p% process usage | `vm_mon.process_high_watermark = 80%` `vm_mon.process_low_watermark = 60%` | +| license_quota | Warning | License exceeds quota | License: the number of connections exceeds % | `license.connection_high_watermark_alarm = 80%` `license.connection_low_watermark_alarm = 75%` | +| license_expiry | Critical | License expired | License will be expired at % | - | +| mnesia_transaction_manager_overload | Warning | mnesia overloaded; mailbox size: N | mailbox size = N | `sysmon.mnesia_tm_mailbox_threshold = 500` | +| broker_pool_overload | Warning | broker pool overloaded; mailbox size: N | mailbox size = N | `sysmon.broker_pool_mailbox_threshold = 500` | +| partition | Critical | Partition occurs at node | Partition occurs at node ~s | - | +| resource | Critical | Resource is disconnected | Resource ~s(~s) is down | - | +| conn_congestion | Critical | Connection process congestion | Connection congested | - | ## Get Alarms diff --git a/zh_CN/observability/alarms.md b/zh_CN/observability/alarms.md index b31723d8f..b5c945763 100644 --- a/zh_CN/observability/alarms.md +++ b/zh_CN/observability/alarms.md @@ -20,34 +20,20 @@ EMQX 提供内置的监控和告警功能，用于监视内部状态变化，如 ::: -**EMQX 开源版告警列表：** - | **告警** | 级别 | 描述 | **详情** | **阈值** | | ----------------------------------- | ---- | -------------------------------------------------- | ----------------------- | ------------------------------------------------------------ | | high_system_memory_usage | 警告 | 系统内存使用过高 | "系统内存使用高于 ~p%" | `os_mon.sysmem_high_watermark = 70%` | | high_process_memory_usage | 警告 | 单个 Erlang 进程内存使用过高（占系统内存的百分比） | 进程内存使用高于 ~p% | `os_mon.procmem_high_watermark = 5%` | | high_cpu_usage | 警告 | CPU 使用率过高 | ~p% CPU 使用率 | `os_mon.cpu_high_watermark = 80%` `os_mon.cpu_low_watermark = 60%` | | too_many_processes | 警告 | 进程过多 | ~p% 进程使用率 | `vm_mon.process_high_watermark = 80%` `vm_mon.process_low_watermark = 60%` | +| license_quota | 警告 | License 超过配额 | License：连接数超过 % | `license.connection_high_watermark_alarm = 80%` `license.connection_low_watermark_alarm = 75%` | +| license_expiry | 严重 | License 过期 | License 将于 % 过期 | - | | mnesia_transaction_manager_overload | 警告 | mnesia 事务管理器过载；邮箱消息数量：N | mailbox size = N | `sysmon.mnesia_tm_mailbox_threshold = 500` | | broker_pool_overload | 警告 | broker 消息处理池过载；邮箱消息数量：N | mailbox size = N | `sysmon.broker_pool_mailbox_threshold = 500` | | partition | 严重 | 节点发生分区 | 节点发生分区 ~s | - | | resource | 严重 | 资源断开连接 | 资源 ~s（~s）已断开连接 | - | | conn_congestion | 严重 | 连接过程拥塞 | 连接拥塞 | - | -**EMQX 企业版告警列表：** - -| **告警** | 级别 | 描述 | **详情** | **阈值** | -| ------------------------- | ---- | -------------------------------------------------- | ----------------------- | ------------------------------------------------------------ | -| high_system_memory_usage | 警告 | 系统内存使用过高 | "系统内存使用高于 ~p%" | `os_mon.sysmem_high_watermark = 70%` | -| high_process_memory_usage | 警告 | 单个 Erlang 进程内存使用过高（占系统内存的百分比） | 进程内存使用高于 ~p% | `os_mon.procmem_high_watermark = 5%` | -| high_cpu_usage | 警告 | CPU 使用率过高 | ~p% CPU 使用率 | `os_mon.cpu_high_watermark = 80%` `os_mon.cpu_low_watermark = 60%` | -| too_many_processes | 警告 | 进程过多 | ~p% 进程使用率 | `vm_mon.process_high_watermark = 80%` `vm_mon.process_low_watermark = 60%` | -| license_quota | 警告 | License 超过配额 | License：连接数超过 % | `license.connection_high_watermark_alarm = 80%` `license.connection_low_watermark_alarm = 75%` | -| license_expiry | 严重 | License 过期 | License 将于 % 过期 | - | -| partition | 严重 | 节点发生分区 | 节点发生分区 ~s | - | -| resource | 严重 | 资源断开连接 | 资源 ~s（~s）已断开连接 | - | -| conn_congestion | 严重 | 连接过程拥塞 | 连接拥塞 | - | - ## 获取告警信息 EMQX 提供多种方式获取告警并查看详细信息。其中一种方式是通过 EMQX Dashboard 查看告警，您可以在此查看已触发的活动或历史告警列表，然而 Dashboard 仅作为一个便于查看告警概览信息的中心。 From 3028206319806f1a6c67835f19b11b2aec12d364 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 9 Sep 2025 06:07:37 +0000 Subject: [PATCH 02/49] Initial plan From 3eb230e0cea3acaf0b6175fbca1498083887b0e6 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 9 Sep 2025 06:12:23 +0000 Subject: [PATCH 03/49] Add clear node name configuration example for single Docker deployment Co-authored-by: id <138221+id@users.noreply.github.com> --- en_US/deploy/install-docker.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/en_US/deploy/install-docker.md b/en_US/deploy/install-docker.md index ce67c8e21..bfc5c0933 100644 --- a/en_US/deploy/install-docker.md +++ b/en_US/deploy/install-docker.md @@ -48,6 +48,21 @@ This section will introduce how to use the Docker image to install the latest ve 3. EMQX employs the `data/mnesia/` directory for data storage. It's crucial to choose a stable identifier, such as a hostname or Fully Qualified Domain Name (FQDN), to serve as the node name. This practice avoids data loss caused by node name changes. + To configure the node name for a single node deployment, use the `EMQX_NODE_NAME` environment variable with the format `emqx@hostname`. You should also set the container hostname to match: + + ```bash + docker run -d \ + --name emqx \ + --hostname node1.emqx.com \ + -e "EMQX_NODE_NAME=emqx@node1.emqx.com" \ + -p 1883:1883 -p 8083:8083 \ + -p 8084:8084 -p 8883:8883 \ + -p 18083:18083 \ + emqx/emqx-enterprise:@EE_VERSION@ + ``` + + **Note:** The node name must follow the format `emqx@` where `` should match the container's hostname or a stable FQDN. + ## Use Docker Compose to Build an EMQX Cluster Docker Compose is a tool for defining and running multi-container Docker applications. This section introduces how to use Docker Compose to create a static EMQX cluster. From 4085f132af1cb55c206f3f6c59ea8181bea0f6dc Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 9 Sep 2025 06:22:43 +0000 Subject: [PATCH 04/49] Update existing Docker command to include node name configuration instead of adding duplicate command Co-authored-by: id <138221+id@users.noreply.github.com> --- en_US/deploy/install-docker.md | 15 +++------------ zh_CN/deploy/install-docker.md | 6 ++++++ 2 files changed, 9 insertions(+), 12 deletions(-) diff --git a/en_US/deploy/install-docker.md b/en_US/deploy/install-docker.md index bfc5c0933..5dd520176 100644 --- a/en_US/deploy/install-docker.md +++ b/en_US/deploy/install-docker.md @@ -36,6 +36,8 @@ This section will introduce how to use the Docker image to install the latest ve ```bash docker run -d --name emqx-enterprise \ + --hostname node1.emqx.com \ + -e "EMQX_NODE_NAME=emqx@node1.emqx.com" \ -p 1883:1883 -p 8083:8083 \ -p 8084:8084 -p 8883:8883 \ -p 18083:18083 \ @@ -48,18 +50,7 @@ This section will introduce how to use the Docker image to install the latest ve 3. EMQX employs the `data/mnesia/` directory for data storage. It's crucial to choose a stable identifier, such as a hostname or Fully Qualified Domain Name (FQDN), to serve as the node name. This practice avoids data loss caused by node name changes. - To configure the node name for a single node deployment, use the `EMQX_NODE_NAME` environment variable with the format `emqx@hostname`. You should also set the container hostname to match: - - ```bash - docker run -d \ - --name emqx \ - --hostname node1.emqx.com \ - -e "EMQX_NODE_NAME=emqx@node1.emqx.com" \ - -p 1883:1883 -p 8083:8083 \ - -p 8084:8084 -p 8883:8883 \ - -p 18083:18083 \ - emqx/emqx-enterprise:@EE_VERSION@ - ``` + To configure the node name for a single node deployment, use the `EMQX_NODE_NAME` environment variable with the format `emqx@hostname`. You should also set the container hostname to match, as shown in the example above. **Note:** The node name must follow the format `emqx@` where `` should match the container's hostname or a stable FQDN. diff --git a/zh_CN/deploy/install-docker.md b/zh_CN/deploy/install-docker.md index e0574de7b..2aa5c99ca 100644 --- a/zh_CN/deploy/install-docker.md +++ b/zh_CN/deploy/install-docker.md @@ -33,6 +33,8 @@ ```bash docker run -d --name emqx-enterprise \ + --hostname node1.emqx.com \ + -e "EMQX_NODE_NAME=emqx@node1.emqx.com" \ -p 1883:1883 -p 8083:8083 \ -p 8084:8084 -p 8883:8883 \ -p 18083:18083 \ @@ -45,6 +47,10 @@ 3. 由于 EMQX 使用 `data/mnesia/<节点名>` 作为数据存储目录，请使用 hostname 或者 FQDN 等固定的信息作为节点名，避免因为节点名称变动导致数据丢失。 + 对于单节点部署，需要使用 `EMQX_NODE_NAME` 环境变量配置节点名，格式为 `emqx@hostname`。您还应该设置容器主机名以保持一致，如上面示例所示。 + + **注意：** 节点名必须遵循 `emqx@` 格式，其中 `` 应该与容器的主机名或稳定的 FQDN 匹配。 + ## 通过 Docker Compose 构建 EMQX 集群 Docker Compose 是一个用于编排和运行多容器的工具，下面将指导您通过 Docker Compose 创建简单的 EMQX 静态集群用于测试。 From d4de9260782cf28d88c24e86b18728ffc26d3ca1 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Wed, 10 Sep 2025 10:57:13 +0800 Subject: [PATCH 05/49] Update dir.yaml --- dir.yaml | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/dir.yaml b/dir.yaml index 597037357..ae23df3b0 100644 --- a/dir.yaml +++ b/dir.yaml @@ -440,25 +440,25 @@ - data-integration/rule-sql-builtin-functions - data-integration/rule-sql-jq # - data-integration/rule-sql-user-defined-function - - title_en: Smart Data Hub - title_cn: 数据智能中心 - title_ja: スマートデータハブ - path: data-integration/smart-data-hub + +- title_en: Smart Data Hub + title_cn: 数据智能中心 + title_ja: スマートデータハブ + path: data-integration/smart-data-hub + collapsed: true + children: + - title_en: Schema Registry + title_cn: Schema Registry + title_ja: スキーマレジストリ + path: data-integration/schema-registry collapsed: true children: - - title_en: Schema Registry - title_cn: Schema Registry - title_ja: スキーマレジストリ - path: data-integration/schema-registry - collapsed: true - children: - - data-integration/schema-registry-example-avro - - data-integration/schema-registry-example-protobuf - - data-integration/schema-registry-example-external-http - - data-integration/sparkplug - - data-integration/schema-validation - - data-integration/message-transformation - + - data-integration/schema-registry-example-avro + - data-integration/schema-registry-example-protobuf + - data-integration/schema-registry-example-external-http + - data-integration/sparkplug + - data-integration/schema-validation + - data-integration/message-transformation - title_en: Flow Designer title_cn: Flow 设计器 title_ja: Flow デザイナー From 4576cd23723f56e327f5997bea4989028f95f5ef Mon Sep 17 00:00:00 2001 From: Shawn <506895667@qq.com> Date: Wed, 10 Sep 2025 20:59:53 +0800 Subject: [PATCH 06/49] feat: add docs for mcp and multimedia server --- .python-version | 1 + dir.yaml | 45 ++ en_US/emqx-ai/mcp-over-mqtt/architecture.md | 90 ++++ en_US/emqx-ai/mcp-over-mqtt/overview.md | 21 + en_US/emqx-ai/mcp-over-mqtt/specification.md | 496 ++++++++++++++++++ en_US/emqx-ai/multimedia-ai/architecture.md | 48 ++ .../emqx-ai/multimedia-ai/message-protocol.md | 372 +++++++++++++ en_US/emqx-ai/multimedia-ai/overview.md | 23 + en_US/emqx-ai/overview.md | 9 + en_US/emqx-ai/sdks/mcp-sdk-erlang.md | 189 +++++++ en_US/emqx-ai/sdks/mcp-sdk-esp32-c.md | 0 en_US/emqx-ai/sdks/mcp-sdk-paho-c.md | 0 en_US/emqx-ai/sdks/mcp-sdk-python.md | 130 +++++ en_US/emqx-ai/sdks/mcp-sdk-typescript.md | 0 en_US/emqx-ai/sdks/mcp-sdks-overview.md | 0 en_US/emqx-ai/sdks/multimedia-ai/overview.md | 11 + .../sdks/multimedia-ai/webrtc-typescript.md | 1 + en_US/emqx-ai/sdks/overview.md | 3 + gen.py | 2 +- ja_JP/emqx-ai/mcp-over-mqtt/architecture.md | 0 ja_JP/emqx-ai/mcp-over-mqtt/overview.md | 0 ja_JP/emqx-ai/mcp-over-mqtt/specification.md | 0 ja_JP/emqx-ai/multimedia-ai/architecture.md | 0 .../emqx-ai/multimedia-ai/message-protocol.md | 0 ja_JP/emqx-ai/multimedia-ai/overview.md | 0 ja_JP/emqx-ai/overview.md | 0 ja_JP/emqx-ai/sdks/mcp-sdk-erlang.md | 0 ja_JP/emqx-ai/sdks/mcp-sdk-esp32-c.md | 0 ja_JP/emqx-ai/sdks/mcp-sdk-paho-c.md | 0 ja_JP/emqx-ai/sdks/mcp-sdk-python.md | 0 ja_JP/emqx-ai/sdks/mcp-sdk-typescript.md | 0 ja_JP/emqx-ai/sdks/mcp-sdks-overview.md | 0 ja_JP/emqx-ai/sdks/multimedia-ai/overview.md | 0 .../sdks/multimedia-ai/webrtc-typescript.md | 0 ja_JP/emqx-ai/sdks/overview.md | 0 pyproject.toml | 9 + zh_CN/emqx-ai/mcp-over-mqtt/architecture.md | 90 ++++ zh_CN/emqx-ai/mcp-over-mqtt/overview.md | 21 + zh_CN/emqx-ai/mcp-over-mqtt/specification.md | 489 +++++++++++++++++ zh_CN/emqx-ai/multimedia-ai/architecture.md | 55 ++ .../emqx-ai/multimedia-ai/message-protocol.md | 369 +++++++++++++ zh_CN/emqx-ai/multimedia-ai/overview.md | 22 + zh_CN/emqx-ai/overview.md | 9 + zh_CN/emqx-ai/sdks/mcp-sdk-erlang.md | 189 +++++++ zh_CN/emqx-ai/sdks/mcp-sdk-esp32-c.md | 0 zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md | 0 zh_CN/emqx-ai/sdks/mcp-sdk-python.md | 130 +++++ zh_CN/emqx-ai/sdks/mcp-sdk-typescript.md | 0 zh_CN/emqx-ai/sdks/mcp-sdks-overview.md | 0 zh_CN/emqx-ai/sdks/multimedia-ai/overview.md | 11 + .../sdks/multimedia-ai/webrtc-typescript.md | 1 + zh_CN/emqx-ai/sdks/overview.md | 3 + 52 files changed, 2838 insertions(+), 1 deletion(-) create mode 100644 .python-version create mode 100644 en_US/emqx-ai/mcp-over-mqtt/architecture.md create mode 100644 en_US/emqx-ai/mcp-over-mqtt/overview.md create mode 100644 en_US/emqx-ai/mcp-over-mqtt/specification.md create mode 100644 en_US/emqx-ai/multimedia-ai/architecture.md create mode 100644 en_US/emqx-ai/multimedia-ai/message-protocol.md create mode 100644 en_US/emqx-ai/multimedia-ai/overview.md create mode 100644 en_US/emqx-ai/overview.md create mode 100644 en_US/emqx-ai/sdks/mcp-sdk-erlang.md create mode 100644 en_US/emqx-ai/sdks/mcp-sdk-esp32-c.md create mode 100644 en_US/emqx-ai/sdks/mcp-sdk-paho-c.md create mode 100644 en_US/emqx-ai/sdks/mcp-sdk-python.md create mode 100644 en_US/emqx-ai/sdks/mcp-sdk-typescript.md create mode 100644 en_US/emqx-ai/sdks/mcp-sdks-overview.md create mode 100644 en_US/emqx-ai/sdks/multimedia-ai/overview.md create mode 100644 en_US/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md create mode 100644 en_US/emqx-ai/sdks/overview.md create mode 100644 ja_JP/emqx-ai/mcp-over-mqtt/architecture.md create mode 100644 ja_JP/emqx-ai/mcp-over-mqtt/overview.md create mode 100644 ja_JP/emqx-ai/mcp-over-mqtt/specification.md create mode 100644 ja_JP/emqx-ai/multimedia-ai/architecture.md create mode 100644 ja_JP/emqx-ai/multimedia-ai/message-protocol.md create mode 100644 ja_JP/emqx-ai/multimedia-ai/overview.md create mode 100644 ja_JP/emqx-ai/overview.md create mode 100644 ja_JP/emqx-ai/sdks/mcp-sdk-erlang.md create mode 100644 ja_JP/emqx-ai/sdks/mcp-sdk-esp32-c.md create mode 100644 ja_JP/emqx-ai/sdks/mcp-sdk-paho-c.md create mode 100644 ja_JP/emqx-ai/sdks/mcp-sdk-python.md create mode 100644 ja_JP/emqx-ai/sdks/mcp-sdk-typescript.md create mode 100644 ja_JP/emqx-ai/sdks/mcp-sdks-overview.md create mode 100644 ja_JP/emqx-ai/sdks/multimedia-ai/overview.md create mode 100644 ja_JP/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md create mode 100644 ja_JP/emqx-ai/sdks/overview.md create mode 100644 pyproject.toml create mode 100644 zh_CN/emqx-ai/mcp-over-mqtt/architecture.md create mode 100644 zh_CN/emqx-ai/mcp-over-mqtt/overview.md create mode 100644 zh_CN/emqx-ai/mcp-over-mqtt/specification.md create mode 100644 zh_CN/emqx-ai/multimedia-ai/architecture.md create mode 100644 zh_CN/emqx-ai/multimedia-ai/message-protocol.md create mode 100644 zh_CN/emqx-ai/multimedia-ai/overview.md create mode 100644 zh_CN/emqx-ai/overview.md create mode 100644 zh_CN/emqx-ai/sdks/mcp-sdk-erlang.md create mode 100644 zh_CN/emqx-ai/sdks/mcp-sdk-esp32-c.md create mode 100644 zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md create mode 100644 zh_CN/emqx-ai/sdks/mcp-sdk-python.md create mode 100644 zh_CN/emqx-ai/sdks/mcp-sdk-typescript.md create mode 100644 zh_CN/emqx-ai/sdks/mcp-sdks-overview.md create mode 100644 zh_CN/emqx-ai/sdks/multimedia-ai/overview.md create mode 100644 zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md create mode 100644 zh_CN/emqx-ai/sdks/overview.md diff --git a/.python-version b/.python-version new file mode 100644 index 000000000..2c0733315 --- /dev/null +++ b/.python-version @@ -0,0 +1 @@ +3.11 diff --git a/dir.yaml b/dir.yaml index 0af7ab4e2..364e611c4 100644 --- a/dir.yaml +++ b/dir.yaml @@ -643,6 +643,51 @@ # children: # - data-integration/rule-configs +- title_en: EMQX AI + title_cn: EMQX AI + title_ja: EMQX AI + path: emqx-ai/overview + collapsed: true + children: + - title_en: MCP over MQTT + title_cn: MCP over MQTT + title_ja: MCP over MQTT + path: emqx-ai/mcp-over-mqtt/overview + children: + - emqx-ai/mcp-over-mqtt/architecture + - emqx-ai/mcp-over-mqtt/specification + - title_en: Multimedia Server + title_cn: 多媒体服务器 + title_ja: マルチメディアサーバー + path: emqx-ai/multimedia-ai/overview + children: + - emqx-ai/multimedia-ai/architecture + - emqx-ai/multimedia-ai/message-protocol + - title_en: SDKs and Examples + title_cn: SDK 和示例 + title_ja: SDK と例 + path: emqx-ai/sdks/overview + collapsed: true + children: + - title_en: MCP SDK + title_cn: MCP SDK + title_ja: MCP SDK + path: emqx-ai/sdks/mcp-sdks-overview + collapsed: true + children: + - emqx-ai/sdks/mcp-sdk-esp32-c + - emqx-ai/sdks/mcp-sdk-paho-c + - emqx-ai/sdks/mcp-sdk-python + - emqx-ai/sdks/mcp-sdk-typescript + - emqx-ai/sdks/mcp-sdk-erlang + - title_en: Multimedia SDK + title_cn: 多媒体 SDK + title_ja: マルチメディア SDK + path: emqx-ai/sdks/multimedia-ai/overview + collapsed: true + children: + - emqx-ai/sdks/multimedia-ai/webrtc-typescript + - title_en: Administration Guide title_cn: 管理员指南 title_ja: 管理者ガイド diff --git a/en_US/emqx-ai/mcp-over-mqtt/architecture.md b/en_US/emqx-ai/mcp-over-mqtt/architecture.md new file mode 100644 index 000000000..89eef53e3 --- /dev/null +++ b/en_US/emqx-ai/mcp-over-mqtt/architecture.md @@ -0,0 +1,90 @@ +# Architecture + +## Core Components of the MQTT Transport + +MCP over MQTT introduces a centralized MQTT broker, while other components (Hosts, Clients, Servers) remain unchanged. + +```mermaid +graph LR + subgraph "Application Host Process" + H[Host] + C1[Client 1] + C2[Client 2] + C3[Client 3] + H --> C1 + H --> C2 + H --> C3 + end + + subgraph "MQTT Broker" + B[Broker] + C1 --> B + C2 --> B + C3 --> B + end + + subgraph "Servers" + S1[Server A
External APIs] + R1[("Remote
Resource A")] + B --> S1 + S1 <--> R1 + end + + subgraph "Servers" + S2[Server B
External APIs] + R2[("Remote
Resource B")] + B --> S2 + S2 <--> R2 + end +``` + +### Host, Client, and Server + +The Host, Client, and Server components remain unchanged: + +- The host process acts as the container and coordinator of the clients. +- Each client is created by the host and maintains an isolated server connection. +- Servers provide specialized context and capabilities. + +With the exception that the clients and servers communicate with the MQTT broker instead of directly with each other. + +See [Core Components](https://modelcontextprotocol.io/docs/learn/architecture#concepts-of-mcp) for more details. + +### MQTT Broker + +The MQTT broker acts as a centralized message router: +- Facilitates communication between clients and servers. +- Support service discovery and service registration (via retained messages). +- Authenticates and authorizes clients and servers. + +## Server Side Load Balancing and Scalability + +To achieve MCP server-side load balancing and scalability, an MCP server can start multiple instances (processes), each using a unique `server-id` as the MQTT Client ID to establish an independent MQTT connection. All instances of an MCP server share the same `server-name`. + +The client must first subscribe to the service discovery topic to obtain the list of `server-id`s for a specific `server-name`. Then, based on a client-defined server selection strategy (e.g., random selection or round-robin), it initiates an `initialize` request to one of the `server-id`s. After initialization is complete, the MCP client communicates with the selected MCP server instance on a specific RPC topic. + +```mermaid +graph LR + + C1["MCP Client1"] + C2["MCP Client2"] + C3["MCP Client3"] + C4["MCP Client4"] + + subgraph "MCP Server Instances (server-name-a)" + S1[Server Instance 1] + S2[Server Instance 2] + end + + C1 <-- "RPC topic of client-1 and server instance 1" --> S1 + C2 <-- "RPC topic of client-2 and server instance 1" --> S1 + C3 <-- "RPC topic of client-3 and server instance 2" --> S2 + C4 <-- "RPC topic of client-4 and server instance 2" --> S2 + +``` + +This allows us to achieve high availability and scalability on the MCP server side: + +- When scaling up, existing MCP clients remain connected to the old server instances, while new MCP clients have the opportunity to initiate initialization requests to the new server instances. + +- When scaling down, MCP clients could re-initiate initialization requests to the MCP server, thereby connecting to another server instance. \ No newline at end of file diff --git a/en_US/emqx-ai/mcp-over-mqtt/overview.md b/en_US/emqx-ai/mcp-over-mqtt/overview.md new file mode 100644 index 000000000..d69fb5f06 --- /dev/null +++ b/en_US/emqx-ai/mcp-over-mqtt/overview.md @@ -0,0 +1,21 @@ +# MCP over MQTT + +MCP over MQTT is an implementation of [MCP](https://modelcontextprotocol.io/docs/getting-started/intro) based on the MQTT protocol, designed to provide efficient, low-latency tool invocation and messaging capabilities for AI applications. With MCP over MQTT, developers can easily integrate AI models and services into IoT systems, enabling seamless interaction between devices and AI. + +## Why Use MQTT for MCP + +MQTT is a lightweight and widely used protocol for IoT and edge computing. It is designed to handle unreliable networks and low bandwidth, making it ideal for communication between edge devices and cloud services. + +By using MQTT as the transport layer for MCP, we extend the application scope of MCP to a wider range of scenarios, including edge computing, IoT, and cloud services—anywhere MQTT is needed. + +## Features + +MCP over MQTT supports all MCP features and adds the following: + +- **Built-in service registration and discovery**: MCP clients can discover available MCP servers from the MQTT broker. + +- **Built-in load balancing and scalability**: MCP servers can be horizontally scaled by adding more MCP server instances, while maintaining server-side state. + +- **Support for centralized authentication and authorization**: MCP over MQTT can leverage the authentication and authorization mechanisms of the MQTT broker, ensuring that only authorized clients can access MCP services. + +- **Support for service name management and distribution**: On top of MCP, EMQX introduces the concept of MCP service names for identification and classification. Users can centrally design and distribute MCP service names on EMQX, simplifying the management and maintenance of multiple MCP services. diff --git a/en_US/emqx-ai/mcp-over-mqtt/specification.md b/en_US/emqx-ai/mcp-over-mqtt/specification.md new file mode 100644 index 000000000..4691ba1a1 --- /dev/null +++ b/en_US/emqx-ai/mcp-over-mqtt/specification.md @@ -0,0 +1,496 @@ +# Specification + +This specification defines the MQTT-specific requirements like MQTT topics and client ID formats. It also outlines the lifecycle of the MQTT transport, including service discovery, initialization, capability list changes, resource updates, and shutdown procedures. + +It should be read in conjunction with the [MCP Specification](https://modelcontextprotocol.io/specification/2025-06-18). + +## Terminology + +- **server-name**: The server name is the identifier of a MCP server, which will be included in the topic. + + Multiple connections with the same `server-name` are considered multiple instances of the same MCP server and provide exactly the same service. When the MCP client sends an initialize message, it should select one of them according to a client-side determined strategy. + + Multiple MCP servers with different `server-name`s may still provide similar functions. In this case, when the client sends an initialize message, it should select one of them to establish a connection as needed. The selection criteria can be based on the client's permissions, recommendations from a LLM, or the user's choice. + + After connected to the MQTT broker, the broker may suggest a `server-name` to the MCP server by including a `MCP-SERVER-NAME` user property in the MQTT CONNECT message. If so, the MCP server **MUST** use this `server-name` as its server name. If the broker does not suggest a `server-name`, the MCP server **SHOULD** use a default `server-name` based on the functionality it provides. + + The `server-name` must be a hierarchical topic style separated by `/` so that the client can subscribe to a certain type of MCP server using MQTT topic wildcards, for example: `server-type/sub-type/name`. + + The `server-name` should not `+` and `#` characters. + + The `server-name` should be unique among all MCP servers. + +- **server-name-filter**: The MQTT topic filter to match the `server-name`, it may include `/`, `+` and `#` characters. See descriptions about **server-name** for more details. + + After connected to the MQTT broker, the broker may suggest a `server-name-filter` to the MCP client by including a `MCP-SERVER-NAME-FILTERS` user property in the MQTT CONNACK message. If so, the MCP client **MUST** use this `server-name-filter` to subscribe to the server's presence topic. The value of the `MCP-SERVER-NAME-FILTERS` is a JSON array of strings, each string is a MQTT topic filter. + If the broker does not suggest a `server-name-filter`, the MCP client **SHOULD** use a default `server-name-filter` based on the functionality it provides. + +- **server-id**: The MQTT Client ID of a MCP server instance. Any string except `/`, `+` and `#`. It must be globally unique and will also be included in the topic. + +- **mcp-client-id**: The MQTT Client ID of the client. Any string except `/`, `+` and `#`. It must be globally unique and will be included in the topic. Each time an initialization request is made, a different client-id must be used. + +## Message Topics + +MCP over MQTT transmits messages through MQTT topics. This protocol includes the following message topics: + +| Topic Name | Topic Name | Description | +|----------------------------------|---------------------------------------------------------------------|------------------------------------------------------------------------------------| +| Server's Control Topic | `$mcp-server/{server-id}/{server-name}` | Used for sending and receiving initialization messages and other control messages. | +| Server's Capability Change Topic | `$mcp-server/capability/{server-id}/{server-name}` | Used for sending and receiving server capability list changed or the resource updated notification. | +| Server's Presence Topic | `$mcp-server/presence/{server-id}/{server-name}` | Used for sending and receiving server's online/offline status messages. | +| Client's Presence Topic | `$mcp-client/presence/{mcp-client-id}` | Used for sending and receiving client's online/offline status messages. | +| Client's Capability Change Topic | `$mcp-client/capability/{mcp-client-id}` | Used for sending and receiving client capability list changed notification. | +| RPC Topic | `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` | Used for sending and receiving RPC requests/responses, and notification messages. | + +## MQTT Protocol Version + +The MCP server and client **MUST** use MQTT Protocol version 5.0. + +## User Property + +For `CONNECT` messages, the following user properties **MUST** be set: +- `MCP-COMPONENT-TYPE`: `mcp-client` or `mcp-server`. +- `MCP-META`: A JSON object containing metadata about the MCP component, such as its version, implementation details, and location. This metadata can be used by the broker to suggest a server name to the MCP server or a server name filter to the MCP client. + +For `CONNACK` messages sent by the broker, the following user properties **MAY** be set: +- `MCP-SERVER-NAME`: The broker suggested server name for the MCP server. Only present if it's a MCP server. +- `MCP-RBAC`: A JSON array of server names and its corresponding role names, which can be used by the MCP client to determine the roles it has for the MCP server. Each element in the array is a JSON object with two fields: `server_name` and `role_name`. Only present if it's a MCP client. +- `MCP-SERVER-NAME-FILTERS`: The broker suggested server name filters. It's a JSON array of strings, each string is a MQTT topic filter that the MCP client can use to subscribe to the server's presence topic. This allows the client to filter the servers it is interested in based on its permissions or other criteria. Only present if it's a MCP client. + +For `PUBLISH` messages, the following user properties **MUST** be set: +- `MCP-COMPONENT-TYPE`: `mcp-client` or `mcp-server`. +- `MCP-MQTT-CLIENT-ID`: MQTT client ID of the sender. + +## Session Expiry Interval + +The session expiry interval **MUST** be set to 0, meaning the session will be cleaned up when the client disconnects. + +## MQTT Client ID + +### MCP Server + +The Client ID of the MCP server can be any string except `/`, `+` and `#`, referred to as `server-id`. + +### MCP Client + +The Client ID of the MCP client, referred to as `mcp-client-id`, can be any string except `/`, `+` and `#`, each time an initialization request is made, a different client-id must be used. + +## MQTT Topics and Topic Filters + +### MCP Server Subscriptions + +| Topic Filter | Explanation | +|-------------------------------------------------------|----------------------------------------------------------------------------------------------------------| +| `$mcp-server/{server-id}/{server-name}` | The control topic of the MCP server to receive control messages. | +| `$mcp-client/capability/{mcp-client-id}` | The MCP client’s capability change topic to receive capability list changed notification of the clients. | +| `$mcp-client/presence/{mcp-client-id}` | The MCP client’s presence topic to receive the disconnected notification of the clients. | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` | The RPC topic to receive RPC requests, RPC responses, and notifications from a MCP client. | + +::: info +- The server **MUST** set the **No Local** option for the RPC topic (`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`) subscription to avoid receiving its own messages. +::: + +### MCP Server Publications + +| Topic Name | Messages | +|------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------| +| `$mcp-server/capability/{server-id}/{server-name}` | capability list changed or resource updated notification. | +| `$mcp-server/presence/{server-id}/{server-name}` | Presence messages for the MCP server.
See [ServiceDiscovery](#service-discovery) for more details | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` | RPC requests, responses and notifications. | + +::: info +- The server **MUST** set the **RETAIN** flag to `True` for the topic `$mcp-server/presence/{server-id}/{server-name}` when publishing the server presence message. +- When connecting to the MQTT broker, the server **MUST** set `$mcp-server/presence/{server-id}/{server-name}` as the will topic with an empty payload to clear the retain message in case of an unexpected disconnection. +::: + + +### MCP Client Subscriptions + +| Topic Filter | Explanation | +|--------------------------------------------------------------------|------------------------------------------------------------------------------------------------| +| `$mcp-server/capability/{server-id}/{server-name-filter}` | The capability change topic to receive capability list changed or resource updated notification of the MCP server. | +| `$mcp-server/presence/+/{server-name-filter}` | The presence topic to receive the presence message of the MCP server. | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}` | The RPC topic to receive PRC requests, responses and notifications sent by the MCP server. | + +::: info +- The client **MUST** set the **No Local** option for the RPC topic (`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}`) subscription to avoid receiving its own messages. +::: + +### MCP Client Publications + +| Topic Name | Messages | +|------------------------------------------------------|--------------------------------------------------------------------| +| `$mcp-server/{server-id}/{server-name}` | Send control messages like the initialize request. | +| `$mcp-client/capability/{mcp-client-id}` | Send client capability list changed notification | +| `$mcp-client/presence/{mcp-client-id}` | Send disconnected notification for the MCP client. | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` | The RPC topic to send RPC requests/responses to a specific server. | + +::: info +- When connecting to the MQTT broker, the client **MUST** set `$mcp-client/presence/{mcp-client-id}` as the will topic with a "disconnected" notification as the payload to notify the server in case of an unexpected disconnection. +::: + +## Service Discovery + +### Service Registration + +After the MCP server starts, it registers its service with the MQTT broker. The presence topic for service discovery and registration is: `$mcp-server/presence/{server-id}/{server-name}`. + +The MCP server **MUST** publish a "server/online" notification to the service presence topic when they start, with the **RETAIN** flag set to `True`. + +The "server/online" notification **SHOULD** provide only limited information about the server to avoid the message size being too large. The client can request more detailed information after initialization. + +- A brief description of the MCP server's functionality to help clients determine which MCP servers they need to initialize. +- Some metadata, such as roles and permissions, to help clients understand the access control policies of the MCP server. The `rbac` field in the metadata can include roles, each with a name, description, allowed methods, allowed tools, and allowed resources, which maybe used by the MQTT broker to implement role-based access control (RBAC) for the MCP server. + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/server/online", + "params": { + "server_name": "example/server", + "description": "This is a brief description about the functionalities provided by this MCP server to allow clients to choose as needed. If tools are provided, it explains what tools are available but does not include tool parameters to reduce message size.", + "meta": { + "rbac": { + "roles": [ + { + "name": "admin", + "description": "Administrator role with full access", + "allowed_methods": [ + "notifications/initialized", + "ping", "tools/list", "tools/call", "resources/list", "resources/read", + "resources/subscribe", "resources/unsubscribe" + ], + "allowed_tools": "all", + "allowed_resources": "all" + }, + { + "name": "user", + "description": "User role with limited access", + "allowed_methods": [ + "notifications/initialized", + "ping", "tools/list", "tools/call", "resources/list", "resources/read" + ], + "allowed_tools": [ + "get_vehicle_status", "get_vehicle_location" + ], + "allowed_resources": [ + "file:///vehicle/telemetry.data" + ] + } + ] + } + } + } +} +``` + +More detailed information, such as parameter details of the tools, **SHOULD** only be fetched by the client when needed, by sending `**/list` requests to the server. + +The client can subscribe to the `$mcp-server/presence/+/{server-name-filter}` topic at any time, where `{server-name-filter}` is a filter for the server name. + +For example, if the server name is `{server-type}/{sub-type}/{name}`, and the client determines through its permissions that it can only access MCP servers of type `{server-type}/{sub-type}`, it can subscribe to `$mcp-server/presence/+/{server-type}/{sub-type}/#`, thereby subscribing to the service presence topic for all MCP servers of the `{sub-type}` type at once. + +Although the client can subscribe to `$mcp-server/presence/+/#` to get all types of MCP servers, the administrator might restrict it through ACL (Access Control List) on the MQTT broker to only send and receive messages on RPC topics like `$mcp-rpc/{mcp-client-id}/{server-id}/{server-type}/{sub-type}/#`. Therefore, subscribing to overly broad topics is not useful. By designing the `{server-name-filter}` appropriately, the client can reduce interference from irrelevant information. + +### Service Unregistration + +When connecting to the MQTT broker, the server must set `$mcp-server/presence/{server-id}/{server-name}` as the will topic, with an empty payload will message, to clear the registration information in case of an unexpected disconnection. + +Before actively disconnecting from the MQTT broker, the server **MUST** send an empty payload message to the `$mcp-server/presence/{server-id}/{server-name}` topic to clear the registration information. + +On the `$mcp-server/presence/{server-id}/{server-name}` topic: + +- When the client receives a `server/online` notification, it should record the `{server-id}` as one of the instances of that `{server-name}`. +- When the client receives an empty payload message, it should clear the cached `{server-id}`. As long as any instance of that `{server-name}` is online, the client should consider the MCP server to be online. + +The message flow for service registration and unregistration is as follows: + +```mermaid +sequenceDiagram + participant MCP_Server as MCP Server + participant MQTT_Broker as MQTT Broker + participant MCP_Client as MCP Client + + MCP_Server ->> MQTT_Broker: Register Service
Topic: $mcp-server/presence/{server-id}/{server-name}
Retain: True + Note right of MQTT_Broker: Store Retained Messages + + MCP_Client ->> MQTT_Broker: Subscribe Services
Topic Filter: $mcp-server/presence/+/ {server-name-filter} + + MQTT_Broker ->> MCP_Client: Description of Service
Topic: $mcp-server/presence/{server-id}/{server-name}
Payload: "notifications/server/online" + Note left of MCP_Client: Record the server-id for a server-name. + + MCP_Server ->> MQTT_Broker: Unregister Service
Topic: $mcp-server/presence/{server-id}/{server-name}
Retain: True
Payload: Empty + Note right of MQTT_Broker: Clean Retained Messages + + MQTT_Broker ->> MCP_Client: Description of Service
Topic: $mcp-server/presence/{server-id}/{server-name}
Payload: Empty + Note left of MCP_Client: Remove the server-id +``` + +## Initialization + +This section only describes the MQTT transport specific parts of the initialization phase, please see [Lifecycle](https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle#initialization) for more details. + +The initialization phase **MUST** be the first interaction between client and server. + +The client **MUST** subscribe to the RPC topic (`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`) before sending the initialization request, with the **No Local** subscription option. + +The server **MUST** subscribe to the RPC topic (`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`) before responding to the initialization response, with the **No Local** subscription option. + +```mermaid +sequenceDiagram + participant MCP_Client as MCP Client + participant MCP_Server as MCP Server + + Note right of MCP_Client: Subscribe the
server's RPC topic + MCP_Client ->> MCP_Server: Initialize Request
Topic: $mcp-server/{server-id}/{server-name} + Note left of MCP_Server: Subscribe the
client's RPC topic + MCP_Server ->> MCP_Client: Initialize Response
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + MCP_Client ->> MCP_Server: Initialized
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + MCP_Client ->> MCP_Server: RPC Req/Resp/Notification
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + MCP_Server ->> MCP_Client: RPC Req/Resp/Notification
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} +``` + +The client **MUST** initiate this phase by sending an `initialize` request to the topic `$mcp-server/{server-id}/{server-name}` containing: + +- Protocol version supported +- Client capabilities +- Client implementation information + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "method": "initialize", + "params": { + "protocolVersion": "2024-11-05", + "capabilities": { + "roots": { + "listChanged": true + }, + "sampling": {} + }, + "clientInfo": { + "name": "ExampleClient", + "version": "1.0.0" + } + } +} +``` + +The server **MUST** respond with its own capabilities to the topic `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` and information: + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "result": { + "protocolVersion": "2024-11-05", + "capabilities": { + "logging": {}, + "prompts": { + "listChanged": true + }, + "resources": { + "subscribe": true, + "listChanged": true + }, + "tools": { + "listChanged": true + } + }, + "serverInfo": { + "name": "ExampleServer", + "version": "1.0.0" + } + } +} +``` + +After successful initialization, the client **MUST** send an initialized notification to indicate it is ready to begin normal operations, to the topic `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/initialized" +} +``` + +## Capability List Changed + +Before initiating the Initialize request, the MCP client **MUST** subscribe to the MCP server's capability list changed topic: `$mcp-server/capability/{server-id}/{server-name-filter}`, where `{server-name-filter}` is a filter for the server name. + +Before the MCP server responds to the initialization request, it **MUST** first subscribe to the MCP client's capability list changed topic: `$mcp-client/capability/{mcp-client-id}`. + +If there are subsequent capability list updates: + +- The server will send a notification to: `$mcp-server/capability/{server-id}/{server-name}` +- The client will send a notification to: `$mcp-client/capability/{mcp-client-id}` + +The payload of the capability list changed notification depends on the specific capability that has changed. For example "notifications/tools/list_changed" for tools. After receiving a capability list change notification, the client or server needs to retrieve the updated capability list. See the specific capability documentation for details. + +```mermaid + +sequenceDiagram + participant MCP_Client as MCP Client + participant MCP_Server as MCP Server + + Note right of MCP_Client: Client subscribes the server's
capability change topic + MCP_Client ->> MCP_Server: Initialize + + Note left of MCP_Server: Server subscribes the client's
capability change topic + MCP_Server ->> MCP_Client: Initialize Response + MCP_Client ->> MCP_Server: Initialized + + MCP_Server -->> MCP_Client: Capability List Changed
Topic: $mcp-server/capability/{server-id}/{server-name} + + MCP_Client ->> MCP_Server: List Capability
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + + MCP_Server -->> MCP_Client: List Capability Response
$mcp-rpc/{mcp-client-id}/{server-id}/{server-name} +``` + +## Resource Update + +The MCP protocol specifies that the client can subscribe to changes of a specific resource. + +If the server provides the capability to subscribe to resources, the client can subscribe to the resource changes before sending the initialized notification. + +The topic for the client to subscribe to resource changes is: `$mcp-server/capability/{server-id}/{server-name}`. + +When a resource changes, the server **SHOULD** send a notification to `$mcp-server/capability/{server-id}/{server-name}`. + +```mermaid + +sequenceDiagram + participant MCP_Client as MCP Client + participant MCP_Server as MCP Server + + MCP_Client ->> MCP_Server: Initialize + MCP_Server ->> MCP_Client: Initialize Response + Note right of MCP_Client: Client subscribes the server's
resource update topic + MCP_Client ->> MCP_Server: Initialized + + MCP_Client ->> MCP_Server: List Resources
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + + MCP_Server -->> MCP_Client: List Resources Response
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name}
URIs: [{resource-uri}, {resource-uri}, ...] + + MCP_Server -->> MCP_Client: Resource Updated
Topic: $mcp-server/capability/{server-id}/{server-name}
URI: {resource-uri} + + MCP_Client ->> MCP_Server: Read Resource
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name}
URI: {resource-uri} + + MCP_Server -->> MCP_Client: Read Resource Response
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name}
URI: {resource-uri} +``` + +## Shutdown + +### Server Disconnect + +The server **MUST** connect with a will message to notify the client when it disconnects unexpectedly, the will topic is `$mcp-server/presence/{server-id}/{server-name}` and the payload is empty. + +Before a MCP server disconnects from the MQTT broker, the server **MUST** send an empty message to the presence topic `$mcp-server/presence/{server-id}/{server-name}`. + +The MCP server may want to 'de-initialize' with a MCP client but still keep the connection with the MQTT broker, in this case it **MUST** send a "disconnected" notification to the rpc topic `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` and then unsubscribe from the following topics: +- `$mcp-client/capability/{mcp-client-id}` +- `$mcp-client/presence/{mcp-client-id}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` + +The message format for the MCP server's "disconnected" notification is: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/disconnected" +} +``` + +When a MCP client receives an empty payload message on the server's presence topic or a "disconnected" notification on the rpc topic, it **MUST** consider the server to be offline and clear the cached `{server-id}` for that `{server-name}` and unsubscribe from the following topics: +- `$mcp-server/capability/{server-id}/{server-name-filter}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}` + +### Client Disconnect + +The server **MUST** subscribe to the client's presence topic (`$mcp-client/presence/{mcp-client-id}`) before sending the initialization response. + +The client **MUST** connect with a will message to notify the server when it disconnects unexpectedly, the will topic is `$mcp-client/presence/{mcp-client-id}` and the payload is a "disconnected" notification. + +Before the client disconnects from the MQTT broker, it **MUST** send a "disconnected" notification to the topic `$mcp-client/presence/{mcp-client-id}`. + +The client may want to 'de-initialize' with a MCP server but still keep the connection with the MQTT broker, in this case it **MUST** send a "disconnected" notification to the rpc topic `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` and then unsubscribe from the following topics: +- `$mcp-server/capability/{server-id}/{server-name-filter}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}` + +After the MCP server receives the "disconnected" notification, it **MUST** unsubscribe from the following topics: +- `$mcp-client/capability/{mcp-client-id}` +- `$mcp-client/presence/{mcp-client-id}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` + +The message format for the MCP client's "disconnected" notification is: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/disconnected" +} +``` + +## Health Checks + +The client or the server **MAY** send `ping` requests to the server at any time to check the health of their counterpart. + +- If the client does not receive a `ping` response from the server within a reasonable time, it **MUST** send a "disconnected" notification to the topic `$mcp-client/presence/{mcp-client-id}` and disconnect itself. +- If the server does not receive a `ping` response from the client within a reasonable time, it **MUST** send any other PRC requests to the client. + +For more information, see [Ping](https://modelcontextprotocol.io/specification/2025-06-18/basic/utilities/ping). + +## Timeouts + +All RPC requests are sent asynchronously via MQTT messages, so timeout issues need to be considered. The timeout duration may vary for different RPC requests, but it should be configurable. + +Below are the recommended default timeout values for each type of RPC request in this protocol: + +- "initialize": 30 seconds +- "ping": 10 seconds +- "roots/list": 30 seconds +- "resources/list": 30 seconds +- "tools/list": 30 seconds +- "prompts/list": 30 seconds +- "prompts/get": 30 seconds +- "sampling/createMessage": 60 seconds +- "resources/read": 30 seconds +- "resources/templates/list": 30 seconds +- "resources/subscribe": 30 seconds +- "tools/call": 60 seconds +- "completion/complete": 60 seconds +- "logging/setLevel": 30 seconds + + + +## Error Handling + +Implementations **SHOULD** be prepared to handle these error cases: + +- Protocol version mismatch +- Failure to negotiate required capabilities +- Initialize request timeout +- Shutdown timeout + +Implementations **SHOULD** implement appropriate timeouts for all requests, to prevent +hung connections and resource exhaustion. + +Example initialization error: + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "error": { + "code": -32602, + "message": "Unsupported protocol version", + "data": { + "supported": ["2025-03-26"], + "requested": "1.0.0" + } + } +} +``` \ No newline at end of file diff --git a/en_US/emqx-ai/multimedia-ai/architecture.md b/en_US/emqx-ai/multimedia-ai/architecture.md new file mode 100644 index 000000000..2d75f91f3 --- /dev/null +++ b/en_US/emqx-ai/multimedia-ai/architecture.md @@ -0,0 +1,48 @@ +# Architecture + +The following diagram illustrates the architecture of a typical multimedia AI system built using EMQX AI. + +```mermaid +flowchart LR + Devices <-- WebRTC --> MultimediaServer + MultimediaServer <-- STDIO --> AIAgents + MultimediaServer <--> ASRTTS + AIAgents <--> LLM + + Devices[Devices] + MultimediaServer[Multimedia
Server] + AIAgents[AI Agents] + ASRTTS[ASR/TTS] + LLM[LLM] +``` + +The following diagram illustrates the interaction flow between the components: + +```mermaid +sequenceDiagram + actor Customer as Device + participant LoginPage as Multimedia Server + participant P1 as AI Agents + participant P2 as LLM + + Customer ->>+ LoginPage: WebRTC Audio + LoginPage ->> LoginPage: ASR + LoginPage ->> P1: ASR Results + P1 ->> P2: Process ASR with MCP Tools + P2 ->> P1: LLM Results + P1 ->> P1: Process LLM Results + P1 ->> LoginPage: TTS and send to Device + LoginPage ->> LoginPage: TTS + LoginPage ->> Customer: WebRTC Audio + Customer ->> LoginPage: WebRTC Video + P1 ->> LoginPage: Realtime Image Analysis + LoginPage ->> LoginPage: Image Analysis + LoginPage ->> P1: Image Analysis Result + P1 ->> P2: Summary the Analysis Reuslt + P2 ->> P1: Summary + P1 ->> LoginPage: TTS and send to Device + LoginPage ->> Customer: WebRTC Audio + P1 ->> P1: Some other processing + P1 ->> LoginPage: Send message to Device + LoginPage ->> Customer: MQTT message +``` diff --git a/en_US/emqx-ai/multimedia-ai/message-protocol.md b/en_US/emqx-ai/multimedia-ai/message-protocol.md new file mode 100644 index 000000000..c6e02bf66 --- /dev/null +++ b/en_US/emqx-ai/multimedia-ai/message-protocol.md @@ -0,0 +1,372 @@ +## Message Protocol + +This document describes the message protocol used for interaction between the multimedia server, clients (devices), and AI agents. + +## WebRTC Signaling via MQTT + +After establishing the MQTT connection, the client needs to use the following MQTT topic to set up the WebRTC connection: + +- `$webrtc//multimedia_proxy`: The MQTT topic for signaling messages between the client and the multimedia proxy for WebRTC connection setup. The client should subscribe to this topic to receive signaling messages from the multimedia proxy. + +- `$webrtc/`: The MQTT topic for the device to receive signaling messages. + +The client should send `offer` and `candidate` messages to the `$webrtc//multimedia_proxy` topic and wait for `answer` and `candidate` messages from the multimedia proxy on the `$webrtc/` topic to establish the WebRTC connection. + +The format of the signaling messages for setting up the WebRTC connections: + +```json +{ + "type": "sdp_offer", + "data": { + "sdp": , + "type": "offer" + } +} +``` + +```json +{ + "type": "sdp_answer", + "data": { + "sdp": , + "type": "answer" + } +} +``` + +```json +{ + "type": "ice_candidate", + "data": { + "candidate": , + "sdpMid": , + "sdpMLineIndex": , + "usernameFragment": + } +} +``` + +The `data` field above can be generated using the [RTCPeerConnection API](https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection) in the browser, for example: + +```javascript +// Create an offer +const offer = await pc.createOffer(); +await pc.setLocalDescription(offer); +const message = { + type: "sdp_offer", + data: offer +}; +// Send the message to the multimedia proxy via MQTT +mqttClient.publish(`$webrtc/${deviceId}/multimedia_proxy`, JSON.stringify(message)); +``` + +```javascript +// Handle the answer from the multimedia proxy +mqttClient.on('message', (topic, message) => { + const msg = JSON.parse(message.toString()); + if (msg.type === 'sdp_answer') { + const answer = msg.data; + pc.setRemoteDescription(new RTCSessionDescription(answer)); + } else if (msg.type === 'ice_candidate') { + const candidate = new RTCIceCandidate(msg.data); + pc.addIceCandidate(candidate); + } +}); +``` + +The multimedia proxy will send the `webrtc_terminated` message to the client when the WebRTC connection is terminated: + +```json +{ + "type": "webrtc_terminated", + "reason": "reason for termination" +} +``` + +See [signaling_mqtt.js](https://github.com/emqx/emqx-multimedia-proxy/blob/main/apps/emqx_media_proxy_web/assets/js/signaling_mqtt.js) for a complete example of the client-side signaling implementation using MQTT. You can go to http://localhost:4000/webrtc_mqtt to try the demo. + + +## Normal Messages via MQTT + +The multimedia proxy can send normal messages to the device via the following MQTT topic: + +- `$message/`: The MQTT topic for the device to receive normal messages from the multimedia proxy. +- `$message//multimedia_proxy`: The MQTT topic for the multimedia proxy to receive arbitrary messages from the device, will be sent to the AI agents using the `message_from_device` method. + +### The format of the normal messages sent to the device: + +The multimedia proxy can send the following types of messages to the device via the `$message/` topic. + +A `asr_response` message is sent when ASR results are available: + +```json +{ + "type": "asr_response", + "format": "merged" | "raw", + "results": +} +``` + +A `tts_begin` message is sent when a TTS task is started: + +```json +{ + "type": "tts_begin", + "task_id": "task_id" +} +``` + +A `tts_text` message is sent when a text is converted to speech and the text should also be sent to the device: + +```json +{ + "type": "tts_text", + "task_id": "task_id", + "text": "text" +} +``` + +A `tts_complete` message is sent when the TTS task is completed: + +```json +{ + "type": "tts_complete", + "task_id": "task_id" +} +``` + +A `tts_terminate` message is sent when the TTS task is finished or terminated: + +```json +{ + "type": "tts_terminate", + "task_id": "task_id" +} +``` + +A `message` message is sent to device when the agent sends an arbitrary message to the device (by the `message_to_device` method): + +```json +{ + "type": "message", + "payload": +} +``` + +### The format of the arbitrary messages sent from the device to the multimedia proxy: + +The device can send arbitrary messages to the multimedia proxy via the `$message//multimedia_proxy` topic. The format of the messages is: + +```json +{ + "type": "message", + "payload": +} +``` + + +## The Interaction Protocol between Multimedia Proxy and AI Agents + +Using AI agents can enhance the capabilities of the multimedia proxy, such as processing ASR results according to specific business logic or sending messages in arbitrary formats to devices. + +The multimedia proxy interacts with AI agents using a simple JSON RPC 2.0 based protocol. The messages are sent over STDIO (standard input/output). Messages are delimited by newlines (`\n`), and MUST NOT contain embedded newlines. + +- **Initialization**: + After the STDIO connection is established, the agent must send an initialization message to the multimedia proxy, to negotiate the protocol version and configs: + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "method": "init", + "params": { + "protocol_version": "1.0", + "configs": { + "asr": { + // If enabled, multimedia proxy will send merged ASR text (based on the timestamps of the sentences) every time a new ASR result is available, otherwise it is the agent's responsibility to merge the ASR results. + "auto_merge": false + } + } + } + } + ``` + + The multimedia proxy will respond with an acknowledgment: + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "result": "ok" + } + ``` + +- **ASR Result**: + The multimedia proxy sends the ASR results as notifications to the AI agents in the following format: + ```json + { + "jsonrpc": "2.0", + "method": "asr_result", + "params": { + // The current device ID + "device_id": "device_id", + "text": "Recognized text" + } + } + ``` + +- **TTS and Send**: + + The AI agents can request the multimedia proxy to perform TTS and send the audio back to the specific device. + + First the agent should send a `tts_and_send_start` message to start a TTS task, and then send one or more `tts_and_send` messages to send the texts to be converted to speech. The texts of the same task can be sent in one batch or in separate messages, but they must have the same `task_id`. Finally, the agent should send a `tts_and_send_finish` message to indicate the end of the TTS task. + + The start message: + ```json + { + "jsonrpc": "2.0", + "id": "3", + "method": "tts_and_send_start", + "params": { + // The deivce ID to send the audio to + "device_id": "device_id", + // + "task_id": "aaa", + "text": "Text to be converted to speech" + } + } + ``` + + The texts to be converted to speech can be sent in one batch: + ```json + [ + { + "jsonrpc": "2.0", + "id": "4", + "method": "tts_and_send", + "params": { + // The deivce ID to send the audio to + "device_id": "device_id", + // + "task_id": "aaa", + "text": "Text to be converted to speech" + } + }, + { + "jsonrpc": "2.0", + "id": "5", + "method": "tts_and_send", + "params": { + // The deivce ID to send the audio to + "device_id": "device_id", + // + "task_id": "aaa", + "text": ", and more text can be send in one batch" + } + }, + { + "jsonrpc": "2.0", + "id": "6", + "method": "tts_and_send_finish", + "params": { + // The device ID to send the audio to + "device_id": "device_id", + // The task ID of the TTS task + "task_id": "aaa" + } + } + ] + ``` + + The `tts_and_send_start` and `tts_and_send_finish` messages canbe sent in the same batch with the `tts_and_send` messages, or in separate messages. + + The multimedia proxy will acknowledge the request with "ok" or errors: + ```json + [ + { + "jsonrpc": "2.0", + "id": "4", + "result": "ok" + }, + { + "jsonrpc": "2.0", + "id": "5", + "result": "ok" + }, + { + "jsonrpc": "2.0", + "id": "6", + "result": "ok" + } + ] + ``` + +- **Image Analysis**: + The AI agents can request the multimedia proxy to perform image analysis: + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "method": "image_analysis", + "params": { + // The ID of the device to capture images from + "device_id": "device_id", + // The count of images to capture and analyze + "image_count": 2, + "capture_interval": 1000, // Interval in milliseconds between captures + "image_format": "jpeg", // Format of the captured images + "user_prompt": "Analyze the images and provide insights" + } + } + ``` + + The multimedia proxy will respond with the analysis results: + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "result": { + "analysis_result": "Analysis result" + } + } + ``` + +- **Forward Messages Received from Device**: + The multimedia proxy will send messages received from `$message//multimedia_proxy` topic to the AI agents via the `message_from_device` method: + ```json + { + "jsonrpc": "2.0", + "method": "message_from_device", + "params": { + // The ID of the device that sent the message + "device_id": "device_id", + "payload": "payload" + } + } + ``` + +- **Send Message to Device**: + The AI agents can send arbitrary messages to the device via the multimedia proxy: + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "method": "message_to_device", + "params": { + // The ID of the device to send the message to + // The message will be sent to the device via the `$message/` MQTT topic + "device_id": "device_id", + // Or you can specify the topic manually to send message to any device + // If specified, the `device_id` field will be ignored + "topic": "topic/to/device", + "payload": "payload" + } + } + ``` + + The multimedia proxy will acknowledge the request with: + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "result": "ok" + } + ``` diff --git a/en_US/emqx-ai/multimedia-ai/overview.md b/en_US/emqx-ai/multimedia-ai/overview.md new file mode 100644 index 000000000..66a3a2570 --- /dev/null +++ b/en_US/emqx-ai/multimedia-ai/overview.md @@ -0,0 +1,23 @@ +# Multimedia Server + +The EMQX Multimedia Server is a high-performance multimedia processing platform built on WebRTC technology. It can receive RTP/SRTP audio and video streams from clients and integrates various AI features such as Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and image understanding. Leveraging large model capabilities, the EMQX Multimedia Server supports complex voice conversations and tool invocation, providing robust technical support for AI applications requiring audio and video capabilities. + +## Core Features + +- **Real-time Audio and Video Processing**: Supports high-quality audio and video streaming, ensuring low latency and highly reliable communication. +- **Automatic Speech Recognition (ASR)**: Delivers accurate speech-to-text functionality, suitable for voice assistants, intelligent customer service, and more. +- **Text-to-Speech (TTS)**: Offers text-to-speech services in multiple languages and voice styles, enhancing user interaction experiences. +- **Image Understanding**: Integrates advanced image recognition and analysis technologies, supporting various image processing applications. +- **Large Model Support**: Utilizes large model capabilities to enable complex voice conversations and tool invocation, meeting diverse business needs. +- **Highly Flexible Architecture**: Adapts to applications of different scales and complexities, supporting horizontal scaling and customizable configurations. +- **High Reliability**: Employs distributed architecture design to ensure high availability and system stability. +- **Low Latency**: Optimized network transmission and large model processing mechanisms guarantee smooth real-time interactions. + +## Application Scenarios + +EMQX Multimedia Server is suitable for the following scenarios: + +- **Emotional Companionship**: Provides personalized emotional companionship services through voice conversations and emotion recognition technologies. +- **Intelligent Customer Service**: Uses ASR and TTS technologies to enable efficient voice interactions and enhance customer service experiences. +- **Intelligent Device Control**: Enables convenient control of smart devices through voice commands and image recognition. + diff --git a/en_US/emqx-ai/overview.md b/en_US/emqx-ai/overview.md new file mode 100644 index 000000000..45587a24a --- /dev/null +++ b/en_US/emqx-ai/overview.md @@ -0,0 +1,9 @@ +# EMQX AI + +EMQX AI is EMQX's innovative practice in the field of artificial intelligence, dedicated to providing efficient and reliable communication infrastructure for AI applications. It consists of the following two aspects: + +- **Message Communication:** Through the MCP over MQTT protocol, EMQX offers lightweight, low-latency tool invocation and message transmission for AI applications, meeting communication needs in scenarios such as model inference, real-time feedback, intelligent data collection, and intelligent control. See [MCP over MQTT](./mcp-over-mqtt/overview.md) for details. + +- **Multimedia Streaming:** Based on a WebRTC-powered multimedia server, EMQX provides real-time audio and video streaming capabilities for AI applications, suitable for scenarios like intelligent voice, video analysis, image understanding, and remote collaboration. See [Multimedia AI](./multimedia-ai/overview.md) for details. + +By combining these two solutions, EMQX builds a solid communication foundation for AI applications, helping developers quickly create intelligent and real-time AI solutions. diff --git a/en_US/emqx-ai/sdks/mcp-sdk-erlang.md b/en_US/emqx-ai/sdks/mcp-sdk-erlang.md new file mode 100644 index 000000000..5109cd5a4 --- /dev/null +++ b/en_US/emqx-ai/sdks/mcp-sdk-erlang.md @@ -0,0 +1,189 @@ +# Erlang SDK + +This document demonstrates how to use the [MCP over MQTT Erlang SDK](https://github.com/emqx/mcp-mqtt-erl) to create a simple MCP over MQTT server and client. + +## Example + +### Creating a Simple MCP Client + +```erlang +-module(mcp_mqtt_erl_client_demo). + +-behaviour(mcp_mqtt_erl_client_session). + +-export([ + client_name/0, + client_version/0, + client_capabilities/0, + received_non_mcp_message/3 +]). +-export([start_link/0]). + +%% The client name, version, and capabilities. These details will be sent to the server during MCP initialization. +client_name() -> + <<"emqx_tools/cli_demo">>. + +client_version() -> + <<"1.0">>. + +client_capabilities() -> #{}. + +%% Callback for non-MCP messages +received_non_mcp_message(MqttClient, Msg, State) -> + io:format("~p Received non-MCP message: ~p~n", [MqttClient, Msg]), + State. + +%% Start the MCP over MQTT client +start_link() -> + mcp_mqtt_erl_client:start_link( + #{ + server_name_filter => <<"#">>, + callback_mod => ?MODULE, + broker_address => {"127.0.0.1", 1883}, + mqtt_options => #{ + clientid => <<"emqx_tools_cli_demo">>, + username => <<"emqx">>, + password => <<"public">> + } + }). +``` + +Here, `server_name_filter` is used to subscribe to the MQTT topic filter for MCP servers, and `mqtt_options` are options passed to the underlying MQTT client. + +### Creating a Simple MCP Server + +Below is a simple MCP server implementation that supports two tools: `tool1` and `tool2`. + +```erlang +-module(mcp_mqtt_erl_server_demo). + +-behaviour(mcp_mqtt_erl_server_session). + +-include_lib("mcp_mqtt_erl/include/mcp_mqtt_erl_types.hrl"). +-include_lib("mcp_mqtt_erl/include/emqx_mcp_tools.hrl"). + +-export([ + start_link/2 +]). + +-export([ + server_name/0, + server_id/2, + server_version/0, + server_capabilities/0, + server_instructions/0, + server_meta/0 +]). + +-export([ + initialize/2, + list_resources/1, + read_resource/2, + call_tool/3, + list_tools/1 +]). + +-type loop_data() :: #{ + server_id => binary(), + client_info => map(), + client_capabilities => map(), + mcp_client_id => binary(), + _ => any() +}. + +-spec start_link(integer(), mcp_mqtt_erl_server:config()) -> gen_statem:start_ret(). +start_link(Idx, Conf) -> + mcp_mqtt_erl_server:start_link(Idx, Conf). + +server_version() -> + <>. + +server_name() -> + <<"emqx_tools/info_apis">>. + +server_id(ClientIdPrefix, Idx) -> + Idx1 = integer_to_binary(Idx), + Node = atom_to_binary(node()), + <>. + +server_capabilities() -> + #{ + resources => #{ + subscribe => true, + listChanged => true + }, + tools => #{ + listChanged => true + } + }. + +server_instructions() -> + <<"">>. + +server_meta() -> + #{ + authorization => #{ + roles => [<<"admin">>, <<"user">>] + } + }. + +-spec initialize(binary(), client_params()) -> {ok, loop_data()}. +initialize(ServerId, #{client_info := ClientInfo, client_capabilities := Capabilities, mcp_client_id := McpClientId}) -> + io:format("initialize --- server_id: ~p, client_info: ~p, client_capabilities: ~p, mcp_client_id: ~p~n", [ServerId, ClientInfo, Capabilities, McpClientId]), + {ok, #{ + server_id => ServerId, + client_info => ClientInfo, + client_capabilities => Capabilities, + mcp_client_id => McpClientId + }}. + +-spec call_tool(binary(), map(), loop_data()) -> {ok, call_tool_result() | [call_tool_result()], loop_data()}. +call_tool(ToolName, Args, LoopData) -> + io:format("call_tool --- tool_name: ~p, args: ~p~n", [ToolName, Args]), + Result = #{ + type => text, + text => <<"This is the result of the tool call">> + }, + {ok, Result, LoopData}. + +-spec list_tools(loop_data()) -> {ok, [tool_def()], loop_data()}. +list_tools(LoopData) -> + io:format("list_tools --- ~n", []), + Tools = [ + #{ + name => <<"tool1">>, + description => <<"This is tool 1">>, + inputSchema => #{ + type => <<"object">>, + properties => #{ + arg1 => #{ + type => <<"string">>, + description => <<"Argument 1">> + }, + arg2 => #{ + type => <<"integer">>, + description => <<"Argument 2">> + } + } + } + }, + #{ + name => <<"tool2">>, + description => <<"This is tool 2">>, + inputSchema => #{ + type => <<"object">>, + properties => #{ + arg1 => #{ + type => <<"string">>, + description => <<"Argument 1">> + }, + arg2 => #{ + type => <<"boolean">>, + description => <<"Argument 2">> + } + } + } + } + ], + {ok, Tools, LoopData}. +``` diff --git a/en_US/emqx-ai/sdks/mcp-sdk-esp32-c.md b/en_US/emqx-ai/sdks/mcp-sdk-esp32-c.md new file mode 100644 index 000000000..e69de29bb diff --git a/en_US/emqx-ai/sdks/mcp-sdk-paho-c.md b/en_US/emqx-ai/sdks/mcp-sdk-paho-c.md new file mode 100644 index 000000000..e69de29bb diff --git a/en_US/emqx-ai/sdks/mcp-sdk-python.md b/en_US/emqx-ai/sdks/mcp-sdk-python.md new file mode 100644 index 000000000..a850618c6 --- /dev/null +++ b/en_US/emqx-ai/sdks/mcp-sdk-python.md @@ -0,0 +1,130 @@ +# Python SDK + +This guide demonstrates how to use the [MCP over MQTT Python SDK](https://github.com/emqx/mcp-python-sdk) to create a simple MCP over MQTT server and client. + +## Create a Demo Project + +Let's use [uv](https://docs.astral.sh/uv/) to create a demo project: + +```bash +uv init mcp_over_mqtt_demo +cd mcp_over_mqtt_demo +``` + +## Create a Simple MCP Server + +In the `mcp_over_mqtt_demo` project, create a simple MCP server that exposes a calculator tool and some resources. Create a file named `demo_mcp_server.py` and add the following code: + +```python +# demo_mcp_server.py +from mcp.server.fastmcp import FastMCP + +# Create an MCP server +mcp = FastMCP( + "demo_mcp_server/calculator", + log_level="DEBUG", + mqtt_server_description="A simple FastMCP server that exposes a calculator tool", + mqtt_options={ + "host": "broker.emqx.io", + }, +) + +# Add an addition tool +@mcp.tool() +def add(a: int, b: int) -> int: + """Add two numbers""" + return a + b + +# Add a dynamic greeting resource +@mcp.resource("greeting://{name}") +def get_greeting(name: str) -> str: + """Get a personalized greeting""" + return f"Hello, {name}!" +``` + +## Create a Simple MCP Client + +In the same project, create a simple MCP client that connects to the server and lists available tools and resources. Create a file named `demo_mcp_client.py` and add the following code: + +```python +# demo_mcp_client.py +import logging +import anyio +import mcp.client.mqtt as mcp_mqtt +from mcp.shared.mqtt import configure_logging + +configure_logging(level="INFO") +logger = logging.getLogger(__name__) + +async def on_mcp_server_discovered(client, server_name): + logger.info(f"Discovered {server_name}, connecting ...") + await client.initialize_mcp_server(server_name) + +async def on_mcp_connect(client, server_name, connect_result): + success, init_result = connect_result + if success == 'error': + logger.error(f"Failed to connect to {server_name}: {init_result}") + return + logger.info(f"Connected to {server_name}, success={success}, init_result={init_result}") + capabilities = init_result.capabilities + if capabilities.prompts: + prompts = await client.list_prompts(server_name) + logger.info(f"Prompts of {server_name}: {prompts}") + if capabilities.resources: + resources = await client.list_resources(server_name) + logger.info(f"Resources of {server_name}: {resources}") + resource_templates = await client.list_resource_templates(server_name) + logger.info(f"Resources templates of {server_name}: {resource_templates}") + if capabilities.tools: + toolsResult = await client.list_tools(server_name) + tools = toolsResult.tools + logger.info(f"Tools of {server_name}: {tools}") + if tools[0].name == "add": + result = await client.call_tool(server_name, name = tools[0].name, arguments={"a": 1, "b": 2}) + logger.info(f"Calling the tool as add(a=1, b=2), result: {result}") + +async def on_mcp_disconnect(client, server_name): + logger.info(f"Disconnected from {server_name}") + +async def main(): + async with mcp_mqtt.MqttTransportClient( + "test_client", + auto_connect_to_mcp_server = True, + on_mcp_server_discovered = on_mcp_server_discovered, + on_mcp_connect = on_mcp_connect, + on_mcp_disconnect = on_mcp_disconnect, + mqtt_options = mcp_mqtt.MqttOptions( + host="broker.emqx.io", + ) + ) as client: + client.start() + while True: + ## Simulate other works while the MQTT transport client is running in the background... + await anyio.sleep(20) + +if __name__ == "__main__": + anyio.run(main) +``` + +## Run the Demo + +1. First, install the required dependencies: + +```bash +uv add git+https://github.com/emqx/mcp-python-sdk --branch main +uv add "mcp[cli]" +``` + +2. Now run the client: + +```bash +uv run demo_mcp_client.py +``` + +3. Open a new terminal and run the server: + +```bash +uv run mcp run --transport mqtt ./demo_mcp_server.py +``` + +Even if the client starts before the server, it will discover the server and connect to it. The client will list available tools and resources, and call the `add` tool with parameters `a=1` and `b=2`. diff --git a/en_US/emqx-ai/sdks/mcp-sdk-typescript.md b/en_US/emqx-ai/sdks/mcp-sdk-typescript.md new file mode 100644 index 000000000..e69de29bb diff --git a/en_US/emqx-ai/sdks/mcp-sdks-overview.md b/en_US/emqx-ai/sdks/mcp-sdks-overview.md new file mode 100644 index 000000000..e69de29bb diff --git a/en_US/emqx-ai/sdks/multimedia-ai/overview.md b/en_US/emqx-ai/sdks/multimedia-ai/overview.md new file mode 100644 index 000000000..d240fe096 --- /dev/null +++ b/en_US/emqx-ai/sdks/multimedia-ai/overview.md @@ -0,0 +1,11 @@ +# Clients Compatible with Multimedia Services + +Clients that support the WebRTC protocol can interact with multimedia AI services. Common clients include: + +- **Web Browsers**: Modern browsers (such as Chrome, Firefox, Edge, Safari) support WebRTC and can directly access multimedia AI services. + +- **Mobile Applications**: By integrating WebRTC SDKs (such as [Pion](https://pion.ly)), mobile apps can interact with multimedia AI services. + +- **Embedded Devices**: IoT devices can connect to multimedia AI services by integrating device-adapted WebRTC libraries, such as [ESP32 WebRTC](https://github.com/espressif/esp-webrtc-solution). + +Here we provide a sample client code based on a web browser, demonstrating how to interact with multimedia services: [Typescript WebRTC Example](./webrtc-typescript.md). diff --git a/en_US/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md b/en_US/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md new file mode 100644 index 000000000..53f4bbc98 --- /dev/null +++ b/en_US/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md @@ -0,0 +1 @@ +# Typescript WebRTC Example diff --git a/en_US/emqx-ai/sdks/overview.md b/en_US/emqx-ai/sdks/overview.md new file mode 100644 index 000000000..3f9097ed0 --- /dev/null +++ b/en_US/emqx-ai/sdks/overview.md @@ -0,0 +1,3 @@ +# MCP over MQTT and Multimedia Service SDKs + +EMQX provides SDKs for various platforms and programming languages to help developers quickly integrate MCP over MQTT and multimedia server features. These SDKs encapsulate the underlying communication details, allowing developers to focus on implementing business logic. diff --git a/gen.py b/gen.py index 76f1446ef..26fe8d301 100755 --- a/gen.py +++ b/gen.py @@ -16,7 +16,7 @@ ## check if the 'lang' field matches expected input ## when no 'lang' is defined, it matches both 'en', 'cn' and 'ja' def is_lang_match(i, lang): - if 'lang' in i: + if isinstance(i, dict) and ('lang' in i): return i['lang'] == lang else: return True diff --git a/ja_JP/emqx-ai/mcp-over-mqtt/architecture.md b/ja_JP/emqx-ai/mcp-over-mqtt/architecture.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/mcp-over-mqtt/overview.md b/ja_JP/emqx-ai/mcp-over-mqtt/overview.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/mcp-over-mqtt/specification.md b/ja_JP/emqx-ai/mcp-over-mqtt/specification.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/multimedia-ai/architecture.md b/ja_JP/emqx-ai/multimedia-ai/architecture.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/multimedia-ai/message-protocol.md b/ja_JP/emqx-ai/multimedia-ai/message-protocol.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/multimedia-ai/overview.md b/ja_JP/emqx-ai/multimedia-ai/overview.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/overview.md b/ja_JP/emqx-ai/overview.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/sdks/mcp-sdk-erlang.md b/ja_JP/emqx-ai/sdks/mcp-sdk-erlang.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/sdks/mcp-sdk-esp32-c.md b/ja_JP/emqx-ai/sdks/mcp-sdk-esp32-c.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/sdks/mcp-sdk-paho-c.md b/ja_JP/emqx-ai/sdks/mcp-sdk-paho-c.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/sdks/mcp-sdk-python.md b/ja_JP/emqx-ai/sdks/mcp-sdk-python.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/sdks/mcp-sdk-typescript.md b/ja_JP/emqx-ai/sdks/mcp-sdk-typescript.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/sdks/mcp-sdks-overview.md b/ja_JP/emqx-ai/sdks/mcp-sdks-overview.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/sdks/multimedia-ai/overview.md b/ja_JP/emqx-ai/sdks/multimedia-ai/overview.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md b/ja_JP/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md new file mode 100644 index 000000000..e69de29bb diff --git a/ja_JP/emqx-ai/sdks/overview.md b/ja_JP/emqx-ai/sdks/overview.md new file mode 100644 index 000000000..e69de29bb diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 000000000..dd813ee59 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,9 @@ +[project] +name = "emqx-docs" +version = "0.1.0" +description = "Add your description here" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + "pyyaml>=6.0.2", +] diff --git a/zh_CN/emqx-ai/mcp-over-mqtt/architecture.md b/zh_CN/emqx-ai/mcp-over-mqtt/architecture.md new file mode 100644 index 000000000..152c9a0e8 --- /dev/null +++ b/zh_CN/emqx-ai/mcp-over-mqtt/architecture.md @@ -0,0 +1,90 @@ +# 架构 + +## MQTT 传输的核心组件 + +MCP over MQTT 引入了中心化的 MQTT Broker，其他组件（Host、Client、Server）保持不变。 + +```mermaid +graph LR + subgraph "Application Host Process" + H[Host] + C1[Client 1] + C2[Client 2] + C3[Client 3] + H --> C1 + H --> C2 + H --> C3 + end + + subgraph "MQTT Broker" + B[Broker] + C1 --> B + C2 --> B + C3 --> B + end + + subgraph "Servers" + S1[Server A
External APIs] + R1[("Remote
Resource A")] + B --> S1 + S1 <--> R1 + end + + subgraph "Servers" + S2[Server B
External APIs] + R2[("Remote
Resource B")] + B --> S2 + S2 <--> R2 + end +``` + +### Host、Client 和 Server + +Host、Client 和 Server 组件保持不变： + +- Host 进程作为客户端的容器和协调者。 +- 每个 Client 由 Host 创建，并维护与 Server 的独立连接。 +- Server 提供专用的上下文和能力。 + +最大的区别是，Client 和 Server 现在通过 MQTT Broker 进行通信，而不是直接相互通信。另外，由于 MQTT Broker 的引入，Client 和 Server 变为多对多的关系，而不是一对一。 + +详见 [核心组件](https://modelcontextprotocol.io/docs/learn/architecture#concepts-of-mcp)。 + +### MQTT Broker + +MQTT Broker 作为中心化的消息路由器： +- 促进 Client 与 Server 之间的通信。 +- 支持服务发现和服务注册（通过保留消息）。 +- 对 Client 和 Server 进行认证和授权。 + +## 服务端负载均衡与可扩展性 + +为实现 MCP 服务端负载均衡与可扩展性，MCP Server 可以启动多个实例（进程），每个实例使用唯一的 `server-id` 作为 MQTT Client ID 建立独立的 MQTT 连接。所有 MCP Server 实例共享同一个 `server-name`。 + +Client 首先需要订阅服务发现主题，以获取指定 `server-name` 下的所有 `server-id` 列表。然后，Client 根据自定义的 Server 选择策略（如随机选择或轮询），向其中一个 `server-id` 发起 `initialize` 请求。初始化完成后，MCP Client 会在特定的 RPC 主题上与选定的 MCP Server 实例进行通信。 + +```mermaid +graph LR + + C1["MCP Client1"] + C2["MCP Client2"] + C3["MCP Client3"] + C4["MCP Client4"] + + subgraph "MCP Server Instances (server-name-a)" + S1[Server Instance 1] + S2[Server Instance 2] + end + + C1 <-- "RPC topic of client-1 and server instance 1" --> S1 + C2 <-- "RPC topic of client-2 and server instance 1" --> S1 + C3 <-- "RPC topic of client-3 and server instance 2" --> S2 + C4 <-- "RPC topic of client-4 and server instance 2" --> S2 + +``` + +这使我们能够实现 MCP 服务端的高可用性和可扩展性： + +- 扩容时，已有的 MCP 客户端会继续连接到旧的服务端实例，而新的 MCP 客户端则有机会向新的服务端实例发起初始化请求。 + +- 缩容时，MCP 客户端可以重新向 MCP 服务端发起初始化请求，从而连接到其他服务端实例。 \ No newline at end of file diff --git a/zh_CN/emqx-ai/mcp-over-mqtt/overview.md b/zh_CN/emqx-ai/mcp-over-mqtt/overview.md new file mode 100644 index 000000000..b2ff7006d --- /dev/null +++ b/zh_CN/emqx-ai/mcp-over-mqtt/overview.md @@ -0,0 +1,21 @@ +# MCP over MQTT + +MCP over MQTT 是一种基于 MQTT 协议的 [MCP](https://modelcontextprotocol.io/docs/getting-started/intro) 实现，旨在为 AI 应用提供高效、低延迟的工具调用以及消息通信能力。通过 MCP over MQTT，开发者可以轻松地在物联网系统中集成 AI 模型和服务，实现设备与 AI 之间的无缝交互。 + +## 为什么使用 MQTT 实现 MCP + +MQTT 是一种轻量级且广泛使用的 IoT 和边缘计算协议。它旨在应对不可靠的网络和低带宽的情况，因此非常适合边缘设备与云服务之间的通信。 + +通过使用 MQTT 作为 MCP 的传输层，我们将 MCP 的应用范围扩展到更广泛的场景中，包括边缘计算、物联网和云服务等任何需要使用 MQTT 的地方。 + +## 特性 + +MCP over MQTT 支持 MCP 的所有特性，并增加了以下特性： + +- **内置服务注册和发现**: MCP 客户端可以从 MQTT 代理发现可用的 MCP 服务器。 + +- **内置负载均衡和可扩展性**: MCP 服务器可以通过添加更多的 MCP 服务器实例进行水平扩展，同时保持 MCP 服务器端的状态。 + +- **支持集中式的认证和授权**: MCP over MQTT 可以利用 MQTT Broker 的认证和授权机制，确保只有经过授权的客户端可以访问 MCP 服务。 + +- **支持服务名管理和下发**: 在 MCP 的基础上，EMQX 增加了 MCP 服务名的概念，用于 MCP 服务的标识和分类管理。用户可以在 EMQX 上集中式地设计和下发 MCP 服务名，简化多 MCP 服务的管理和维护。 diff --git a/zh_CN/emqx-ai/mcp-over-mqtt/specification.md b/zh_CN/emqx-ai/mcp-over-mqtt/specification.md new file mode 100644 index 000000000..cdf103cc8 --- /dev/null +++ b/zh_CN/emqx-ai/mcp-over-mqtt/specification.md @@ -0,0 +1,489 @@ +# 协议规范 + +本文档定义了 MCP 协议在 MQTT 传输层下的特殊要求，包括 MQTT 主题、客户端 ID 格式等，并描述了 MQTT 传输层的生命周期，包括服务发现、初始化、能力列表变更、资源更新和关闭流程。 + +请结合 [MCP 规范](https://modelcontextprotocol.io/specification/2025-06-18) 一起阅读。 + +## 术语 + +- **server-name**：MCP 服务器的标识符，将包含在主题中。 + + 多个使用相同 `server-name` 的连接视为同一个 MCP 服务器的多个实例，提供完全相同的服务。MCP 客户端发送初始化消息时，应根据客户端侧策略选择其中一个。 + + 不同 `server-name` 的 MCP 服务器可能提供类似功能，此时客户端可根据权限、LLM 推荐或用户选择，决定初始化连接的服务器。 + + 连接 MQTT Broker 后，Broker 可通过 MQTT CONNECT 消息中的 `MCP-SERVER-NAME` 用户属性建议 MCP 服务器使用的 `server-name`。如有建议，MCP 服务器**必须**采用该名称；否则应根据自身功能使用默认名称。 + + `server-name` 必须采用以 `/` 分隔的层级主题风格，便于客户端通过 MQTT 通配符订阅特定类型的 MCP 服务器，例如：`server-type/sub-type/name`。 + + `server-name` 不应包含 `+` 和 `#` 字符，且应在所有 MCP 服务器中唯一。 + +- **server-name-filter**：用于匹配 `server-name` 的 MQTT 主题过滤器，可包含 `/`、`+`、`#`。详见 **server-name** 说明。 + + 连接 MQTT Broker 后，Broker 可通过 MQTT CONNACK 消息中的 `MCP-SERVER-NAME-FILTERS` 用户属性建议 MCP 客户端使用的过滤器。该属性值为字符串数组，每个字符串为 MQTT 主题过滤器。若无建议，客户端应根据自身功能使用默认过滤器。 + +- **server-id**：MCP 服务器实例的 MQTT Client ID，除 `/`、`+`、`#` 外任意字符串。必须全局唯一，并包含在主题中。 + +- **mcp-client-id**：客户端的 MQTT Client ID，除 `/`、`+`、`#` 外任意字符串。必须全局唯一，并包含在主题中。每次初始化请求需使用不同的 client-id。 + +## 消息主题 + +MCP 通过 MQTT 主题传递消息，协议包含如下主题： + +| 主题名称 | 主题格式 | 说明 | +|-------------------------------|-------------------------------------------------------------------|-----------------------------------------------------------------------------| +| 服务器控制主题 | `$mcp-server/{server-id}/{server-name}` | 初始化及其他控制消息。 | +| 服务器能力变更主题 | `$mcp-server/capability/{server-id}/{server-name}` | 能力列表变更或资源更新通知。 | +| 服务器在线状态主题 | `$mcp-server/presence/{server-id}/{server-name}` | 服务器在线/离线状态消息。 | +| 客户端在线状态主题 | `$mcp-client/presence/{mcp-client-id}` | 客户端在线/离线状态消息。 | +| 客户端能力变更主题 | `$mcp-client/capability/{mcp-client-id}` | 客户端能力列表变更通知。 | +| RPC 主题 | `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` | RPC 请求/响应及通知消息。 | + +## MQTT 协议版本 + +MCP 服务器和客户端**必须**使用 MQTT 5.0 协议。 + +## 用户属性 + +CONNECT 消息**必须**设置如下用户属性： +- `MCP-COMPONENT-TYPE`：`mcp-client` 或 `mcp-server` +- `MCP-META`：包含 MCP 组件元数据的 JSON 对象，如版本、实现信息、位置等。Broker 可据此建议 server-name 或 server-name-filter。 + +CONNACK 消息（Broker 发送）**可选**用户属性： +- `MCP-SERVER-NAME`：Broker 建议的 MCP 服务器名称，仅服务器连接时存在。 +- `MCP-RBAC`：JSON 数组，包含服务器名称及对应角色名，客户端可据此判断权限。每个元素为包含 `server_name` 和 `role_name` 的对象，仅客户端连接时存在。 +- `MCP-SERVER-NAME-FILTERS`：Broker 建议的服务器名称过滤器，字符串数组，每个为 MQTT 主题过滤器，仅客户端连接时存在。 + +PUBLISH 消息**必须**设置如下用户属性： +- `MCP-COMPONENT-TYPE`：`mcp-client` 或 `mcp-server` +- `MCP-MQTT-CLIENT-ID`：发送方的 MQTT Client ID + +## 会话过期时间 + +会话过期时间**必须**设为 0，客户端断开时会话立即清理。 + +## MQTT Client ID + +### MCP 服务器 + +MCP 服务器的 Client ID（`server-id`）为除 `/`、`+`、`#` 外的任意字符串。 + +### MCP 客户端 + +MCP 客户端的 Client ID（`mcp-client-id`）为除 `/`、`+`、`#` 外的任意字符串，每次初始化请求需使用不同的 client-id。 + +## MQTT 主题与过滤器 + +### MCP 服务器订阅 + +| 主题过滤器 | 说明 | +|----------------------------------------------------|----------------------------------------------------------------------| +| `$mcp-server/{server-id}/{server-name}` | 服务器控制主题，接收控制消息。 | +| `$mcp-client/capability/{mcp-client-id}` | 客户端能力变更主题，接收客户端能力列表变更通知。 | +| `$mcp-client/presence/{mcp-client-id}` | 客户端在线状态主题，接收客户端断开通知。 | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`| RPC 主题，接收来自客户端的 RPC 请求、响应和通知。 | + +::: info +- 服务器订阅 RPC 主题时**必须**设置 **No Local** 选项，避免收到自身消息。 +::: + +### MCP 服务器发布 + +| 主题名称 | 消息内容 | +|-----------------------------------------------------|--------------------------------------------------------------------------| +| `$mcp-server/capability/{server-id}/{server-name}` | 能力列表变更或资源更新通知。 | +| `$mcp-server/presence/{server-id}/{server-name}` | 服务器在线状态消息，详见[服务发现](#service-discovery)。 | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`| RPC 请求、响应和通知。 | + +::: info +- 服务器发布在线状态消息时，**必须**将 `$mcp-server/presence/{server-id}/{server-name}` 的 RETAIN 标志设为 True。 +- 连接 MQTT Broker 时，服务器**必须**将 `$mcp-server/presence/{server-id}/{server-name}` 设为遗嘱主题，负载为空，用于异常断开时清除保留消息。 +::: + +### MCP 客户端订阅 + +| 主题过滤器 | 说明 | +|----------------------------------------------------|----------------------------------------------------------------------| +| `$mcp-server/capability/{server-id}/{server-name-filter}`| 能力变更主题，接收服务器能力列表变更或资源更新通知。 | +| `$mcp-server/presence/+/{server-name-filter}` | 服务器在线状态主题，接收服务器在线状态消息。 | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}`| RPC 主题，接收服务器发送的 RPC 请求、响应和通知。 | + +::: info +- 客户端订阅 RPC 主题时**必须**设置 **No Local** 选项，避免收到自身消息。 +::: + +### MCP 客户端发布 + +| 主题名称 | 消息内容 | +|--------------------------------------------|------------------------------------------------------| +| `$mcp-server/{server-id}/{server-name}` | 发送控制消息，如初始化请求。 | +| `$mcp-client/capability/{mcp-client-id}` | 发送客户端能力列表变更通知。 | +| `$mcp-client/presence/{mcp-client-id}` | 发送客户端断开通知。 | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`| 向指定服务器发送 RPC 请求/响应。 | + +::: info +- 连接 MQTT Broker 时，客户端**必须**将 `$mcp-client/presence/{mcp-client-id}` 设为遗嘱主题，负载为 "disconnected" 通知，用于异常断开时通知服务器。 +::: + +## 服务发现 + +### 服务注册 + +MCP 服务器启动后，向 MQTT Broker 注册服务。服务发现与注册主题为：`$mcp-server/presence/{server-id}/{server-name}`。 + +服务器**必须**在启动时向服务在线状态主题发布 "server/online" 通知，并设置 RETAIN 标志为 True。 + +"server/online" 通知**应**只包含简要信息，避免消息过大。客户端可在初始化后请求详细信息。 + +- MCP 服务器功能简述，便于客户端选择初始化对象。 +- 元数据，如角色与权限，帮助客户端理解访问控制策略。`rbac` 字段可包含角色列表，每个角色含名称、描述、允许的方法、工具和资源，Broker 可据此实现 RBAC。 + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/server/online", + "params": { + "server_name": "example/server", + "description": "简要描述 MCP 服务器功能，便于客户端按需选择。若提供工具，仅说明工具类型，不含参数，减少消息体积。", + "meta": { + "rbac": { + "roles": [ + { + "name": "admin", + "description": "管理员角色，拥有全部权限", + "allowed_methods": [ + "notifications/initialized", + "ping", "tools/list", "tools/call", "resources/list", "resources/read", + "resources/subscribe", "resources/unsubscribe" + ], + "allowed_tools": "all", + "allowed_resources": "all" + }, + { + "name": "user", + "description": "普通用户角色，权限有限", + "allowed_methods": [ + "notifications/initialized", + "ping", "tools/list", "tools/call", "resources/list", "resources/read" + ], + "allowed_tools": [ + "get_vehicle_status", "get_vehicle_location" + ], + "allowed_resources": [ + "file:///vehicle/telemetry.data" + ] + } + ] + } + } + } +} +``` + +详细信息如工具参数，**应**由客户端按需通过 `**/list` 请求获取。 + +客户端可随时订阅 `$mcp-server/presence/+/{server-name-filter}`，其中 `{server-name-filter}` 为服务器名称过滤器。 + +例如，若服务器名称为 `{server-type}/{sub-type}/{name}`，客户端仅有权限访问 `{server-type}/{sub-type}` 类型服务器，则可订阅 `$mcp-server/presence/+/{server-type}/{sub-type}/#`，一次性获取所有 `{sub-type}` 类型服务器的在线状态。 + +虽然客户端可订阅 `$mcp-server/presence/+/#` 获取所有类型服务器，但管理员可通过 MQTT Broker 的 ACL 限制其只能在如 `$mcp-rpc/{mcp-client-id}/{server-id}/{server-type}/{sub-type}/#` 等主题收发消息。因此，过宽的订阅无实际意义。合理设计 `{server-name-filter}` 可减少无关信息干扰。 + +### 服务注销 + +连接 MQTT Broker 时，服务器**必须**将 `$mcp-server/presence/{server-id}/{server-name}` 设为遗嘱主题，负载为空，用于异常断开时清除注册信息。 + +主动断开前，服务器**必须**向 `$mcp-server/presence/{server-id}/{server-name}` 发送空消息，清除注册信息。 + +在 `$mcp-server/presence/{server-id}/{server-name}` 主题上： + +- 客户端收到 `server/online` 通知时，应记录 `{server-id}` 为该 `{server-name}` 的一个实例。 +- 客户端收到空消息时，应清除缓存的 `{server-id}`。只要有任一实例在线，客户端应认为 MCP 服务器在线。 + +服务注册与注销消息流程如下： + +```mermaid +sequenceDiagram + participant MCP_Server as MCP 服务器 + participant MQTT_Broker as MQTT Broker + participant MCP_Client as MCP 客户端 + + MCP_Server ->> MQTT_Broker: 注册服务
主题: $mcp-server/presence/{server-id}/{server-name}
Retain: True + Note right of MQTT_Broker: 存储保留消息 + + MCP_Client ->> MQTT_Broker: 订阅服务
主题过滤器: $mcp-server/presence/+/ {server-name-filter} + + MQTT_Broker ->> MCP_Client: 服务描述
主题: $mcp-server/presence/{server-id}/{server-name}
负载: "notifications/server/online" + Note left of MCP_Client: 记录 server-id + + MCP_Server ->> MQTT_Broker: 注销服务
主题: $mcp-server/presence/{server-id}/{server-name}
Retain: True
负载: 空 + Note right of MQTT_Broker: 清除保留消息 + + MQTT_Broker ->> MCP_Client: 服务描述
主题: $mcp-server/presence/{server-id}/{server-name}
负载: 空 + Note left of MCP_Client: 移除 server-id +``` + +## 初始化 + +本节仅描述 MQTT 传输相关的初始化流程，详细内容见 [生命周期](https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle)。 + +初始化阶段**必须**是客户端与服务器的首次交互。 + +客户端**必须**在发送初始化请求前，订阅 RPC 主题（`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`），并设置 **No Local** 选项。 + +服务器**必须**在响应初始化前，订阅 RPC 主题（`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`），并设置 **No Local** 选项。 + +```mermaid +sequenceDiagram + participant MCP_Client as MCP 客户端 + participant MCP_Server as MCP 服务器 + + Note right of MCP_Client: 订阅服务器 RPC 主题 + MCP_Client ->> MCP_Server: 初始化请求
主题: $mcp-server/{server-id}/{server-name} + Note left of MCP_Server: 订阅客户端 RPC 主题 + MCP_Server ->> MCP_Client: 初始化响应
主题: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + MCP_Client ->> MCP_Server: 已初始化通知
主题: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + MCP_Client ->> MCP_Server: RPC 请求/响应/通知
主题: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + MCP_Server ->> MCP_Client: RPC 请求/响应/通知
主题: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} +``` + +客户端**必须**通过 `$mcp-server/{server-id}/{server-name}` 主题发送 `initialize` 请求，内容包括： + +- 支持的协议版本 +- 客户端能力 +- 客户端实现信息 + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "method": "initialize", + "params": { + "protocolVersion": "2024-11-05", + "capabilities": { + "roots": { + "listChanged": true + }, + "sampling": {} + }, + "clientInfo": { + "name": "ExampleClient", + "version": "1.0.0" + } + } +} +``` + +服务器**必须**通过 `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` 主题响应自身能力及信息： + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "result": { + "protocolVersion": "2024-11-05", + "capabilities": { + "logging": {}, + "prompts": { + "listChanged": true + }, + "resources": { + "subscribe": true, + "listChanged": true + }, + "tools": { + "listChanged": true + } + }, + "serverInfo": { + "name": "ExampleServer", + "version": "1.0.0" + } + } +} +``` + +初始化成功后，客户端**必须**通过 `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` 发送已初始化通知： + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/initialized" +} +``` + +## 能力列表变更 + +客户端在发送初始化请求前，**必须**订阅服务器能力列表变更主题：`$mcp-server/capability/{server-id}/{server-name-filter}`。 + +服务器在响应初始化请求前，**必须**订阅客户端能力列表变更主题：`$mcp-client/capability/{mcp-client-id}`。 + +后续能力列表更新时： + +- 服务器向 `$mcp-server/capability/{server-id}/{server-name}` 发送通知 +- 客户端向 `$mcp-client/capability/{mcp-client-id}` 发送通知 + +能力列表变更通知负载依赖具体能力，如工具变更为 "notifications/tools/list_changed"。收到通知后，需通过 RPC 获取最新能力列表，详见相关能力文档。 + +```mermaid + +sequenceDiagram + participant MCP_Client as MCP 客户端 + participant MCP_Server as MCP 服务器 + + Note right of MCP_Client: 客户端订阅服务器能力变更主题 + MCP_Client ->> MCP_Server: 初始化 + + Note left of MCP_Server: 服务器订阅客户端能力变更主题 + MCP_Server ->> MCP_Client: 初始化响应 + MCP_Client ->> MCP_Server: 已初始化 + + MCP_Server -->> MCP_Client: 能力列表变更
主题: $mcp-server/capability/{server-id}/{server-name} + + MCP_Client ->> MCP_Server: 获取能力列表
主题: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + + MCP_Server -->> MCP_Client: 能力列表响应
$mcp-rpc/{mcp-client-id}/{server-id}/{server-name} +``` + +## 资源更新 + +MCP 协议允许客户端订阅特定资源的变更。 + +若服务器支持资源订阅能力，客户端可在发送已初始化通知前订阅资源变更主题。 + +客户端订阅资源变更主题为：`$mcp-server/capability/{server-id}/{server-name}`。 + +资源变更时，服务器**应**向 `$mcp-server/capability/{server-id}/{server-name}` 发送通知。 + +```mermaid + +sequenceDiagram + participant MCP_Client as MCP 客户端 + participant MCP_Server as MCP 服务器 + + MCP_Client ->> MCP_Server: 初始化 + MCP_Server ->> MCP_Client: 初始化响应 + Note right of MCP_Client: 客户端订阅服务器资源更新主题 + MCP_Client ->> MCP_Server: 已初始化 + + MCP_Client ->> MCP_Server: 获取资源列表
主题: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + + MCP_Server -->> MCP_Client: 资源列表响应
主题: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name}
URIs: [{resource-uri}, {resource-uri}, ...] + + MCP_Server -->> MCP_Client: 资源更新通知
主题: $mcp-server/capability/{server-id}/{server-name}
URI: {resource-uri} + + MCP_Client ->> MCP_Server: 读取资源
主题: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name}
URI: {resource-uri} + + MCP_Server -->> MCP_Client: 读取资源响应
主题: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name}
URI: {resource-uri} +``` + +## 关闭流程 + +### 服务器断开 + +服务器**必须**连接时设置遗嘱消息，通知客户端异常断开，遗嘱主题为 `$mcp-server/presence/{server-id}/{server-name}`，负载为空。 + +主动断开前，服务器**必须**向该主题发送空消息。 + +服务器如需与客户端“去初始化”但保持与 Broker 的连接，**必须**向 RPC 主题 `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` 发送 "disconnected" 通知，并取消订阅如下主题： +- `$mcp-client/capability/{mcp-client-id}` +- `$mcp-client/presence/{mcp-client-id}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` + +"disconnected" 通知格式： + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/disconnected" +} +``` + +客户端收到服务器在线状态主题的空消息或 RPC 主题的 "disconnected" 通知时，**必须**视服务器为离线，清除缓存的 `{server-id}`，并取消订阅如下主题： +- `$mcp-server/capability/{server-id}/{server-name-filter}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}` + +### 客户端断开 + +服务器**必须**在响应初始化前订阅客户端在线状态主题（`$mcp-client/presence/{mcp-client-id}`）。 + +客户端**必须**连接时设置遗嘱消息，通知服务器异常断开，遗嘱主题为 `$mcp-client/presence/{mcp-client-id}`，负载为 "disconnected" 通知。 + +主动断开前，客户端**必须**向该主题发送 "disconnected" 通知。 + +客户端如需与服务器“去初始化”但保持与 Broker 的连接，**必须**向 RPC 主题 `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` 发送 "disconnected" 通知，并取消订阅如下主题： +- `$mcp-server/capability/{server-id}/{server-name-filter}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}` + +服务器收到 "disconnected" 通知后，**必须**取消订阅如下主题： +- `$mcp-client/capability/{mcp-client-id}` +- `$mcp-client/presence/{mcp-client-id}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` + +"disconnected" 通知格式： + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/disconnected" +} +``` + +## 健康检查 + +客户端或服务器**可**随时发送 `ping` 请求检查对方健康状态。 + +- 客户端若未在合理时间收到服务器 `ping` 响应，**必须**向 `$mcp-client/presence/{mcp-client-id}` 发送 "disconnected" 通知并断开自身连接。 +- 服务器若未及时收到客户端 `ping` 响应，**必须**向客户端发送其他 RPC 请求。 + +详见 [Ping](https://modelcontextprotocol.io/specification/2025-06-18/basic/utilities/ping)。 + +## 超时设置 + +所有 RPC 请求均通过 MQTT 异步发送，需考虑超时。不同请求超时时间可配置，推荐默认值如下： + +- "initialize"：30 秒 +- "ping"：10 秒 +- "roots/list"：30 秒 +- "resources/list"：30 秒 +- "tools/list"：30 秒 +- "prompts/list"：30 秒 +- "prompts/get"：30 秒 +- "sampling/createMessage"：60 秒 +- "resources/read"：30 秒 +- "resources/templates/list"：30 秒 +- "resources/subscribe"：30 秒 +- "tools/call"：60 秒 +- "completion/complete"：60 秒 +- "logging/setLevel"：30 秒 + + + +## 错误处理 + +实现应准备处理如下错误场景： + +- 协议版本不匹配 +- 必需能力协商失败 +- 初始化请求超时 +- 关闭超时 + +实现应为所有请求设置合理超时，防止连接挂起和资源耗尽。 + +初始化错误示例： + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "error": { + "code": -32602, + "message": "不支持的协议版本", + "data": { + "supported": ["2025-03-26"], + "requested": "1.0.0" + } + } +} +``` diff --git a/zh_CN/emqx-ai/multimedia-ai/architecture.md b/zh_CN/emqx-ai/multimedia-ai/architecture.md new file mode 100644 index 000000000..e1701ee6b --- /dev/null +++ b/zh_CN/emqx-ai/multimedia-ai/architecture.md @@ -0,0 +1,55 @@ +# 架构 + +下图展示了使用多媒体服务器的 AI 系统的整体架构： + +```mermaid +flowchart LR + 设备 <-- WebRTC --> 多媒体服务器 + 多媒体服务器 <-- STDIO --> AI代理 + 多媒体服务器 <--> ASRTTS + AI代理 <--> LLM + + 设备[设备] + 多媒体服务器[多媒体服务器] + AI代理[AI代理] + ASRTTS[ASR/TTS] + LLM[LLM] +``` + +- 设备: 通过 WebRTC 与多媒体服务器进行音视频数据的交互。 +- 多媒体服务器: 负责处理来自设备的音视频数据，提供 ASR（自动语音识别）、 TTS（文本转语音）等功能，并与 AI 代理进行通信。 +- AI 代理: 通过 STDIO 接受多媒体服务器传递的 ASR 结果，包含了 AI 应用的核心业务逻辑，调用 LLM（大语言模型）处理文本自然语言，并调用多媒体服务提供的 API 发送文字或音频流给设备。 +- ASR/TTS: 语音和视觉模型提供商，提供自动语音识别和文本转语音服务。 + +## 工作流程 + +下图展示了各组件之间的交互流程： + +```mermaid +sequenceDiagram + actor Customer as 设备 + participant Media as 多媒体服务器 + participant P1 as AI代理 + participant P2 as LLM + + Customer ->>+ Media: WebRTC Audio + Media ->> Media: ASR + Media ->> P1: ASR Results + P1 ->> P2: Process ASR with MCP Tools + P2 ->> P1: LLM Results + P1 ->> P1: Process LLM Results + P1 ->> Media: TTS and send to Device + Media ->> Media: TTS + Media ->> Customer: WebRTC Audio + Customer ->> Media: WebRTC Video + P1 ->> Media: Realtime Image Analysis + Media ->> Media: Image Analysis + Media ->> P1: Image Analysis Result + P1 ->> P2: Summary the Analysis Reuslt + P2 ->> P1: Summary + P1 ->> Media: TTS and send to Device + Media ->> Customer: WebRTC Audio + P1 ->> P1: Some other processing + P1 ->> Media: Send message to Device + Media ->> Customer: MQTT message +``` diff --git a/zh_CN/emqx-ai/multimedia-ai/message-protocol.md b/zh_CN/emqx-ai/multimedia-ai/message-protocol.md new file mode 100644 index 000000000..8cd35a90b --- /dev/null +++ b/zh_CN/emqx-ai/multimedia-ai/message-protocol.md @@ -0,0 +1,369 @@ +# 消息协议 + +本文档描述了多媒体服务器与客户端（设备）和 AI 代理之间交互所使用的消息协议。 + +## 通过 MQTT 发送和接收 WebRTC 信令 + +建立 MQTT 连接后，客户端需使用以下 MQTT 主题来建立 WebRTC 连接： + +- `$webrtc//multimedia_proxy`：客户端与多媒体服务器之间用于 WebRTC 连接建立的信令消息 MQTT 主题。客户端应订阅此主题以接收来自多媒体服务器的信令消息。 +- `$webrtc/`：设备用于接收信令消息的 MQTT 主题。 + +客户端应将 `offer` 和 `candidate` 消息发送到 `$webrtc//multimedia_proxy` 主题，并在 `$webrtc/` 主题上等待来自多媒体服务器的 `answer` 和 `candidate` 消息，以建立 WebRTC 连接。 + +WebRTC 连接信令消息格式如下： + +```json +{ + "type": "sdp_offer", + "data": { + "sdp": , + "type": "offer" + } +} +``` + +```json +{ + "type": "sdp_answer", + "data": { + "sdp": , + "type": "answer" + } +} +``` + +```json +{ + "type": "ice_candidate", + "data": { + "candidate": , + "sdpMid": , + "sdpMLineIndex": , + "usernameFragment": + } +} +``` + +上述 `data` 字段可通过浏览器中的 [RTCPeerConnection API](https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection) 生成，例如： + +```javascript +// 创建 offer +const offer = await pc.createOffer(); +await pc.setLocalDescription(offer); +const message = { + type: "sdp_offer", + data: offer +}; +// 通过 MQTT 向多媒体服务器发送消息 +mqttClient.publish(`$webrtc/${deviceId}/multimedia_proxy`, JSON.stringify(message)); +``` + +```javascript +// 处理来自多媒体服务器的 answer +mqttClient.on('message', (topic, message) => { + const msg = JSON.parse(message.toString()); + if (msg.type === 'sdp_answer') { + const answer = msg.data; + pc.setRemoteDescription(new RTCSessionDescription(answer)); + } else if (msg.type === 'ice_candidate') { + const candidate = new RTCIceCandidate(msg.data); + pc.addIceCandidate(candidate); + } +}); +``` + +当 WebRTC 连接终止时，多媒体服务器会向客户端发送 `webrtc_terminated` 消息： + +```json +{ + "type": "webrtc_terminated", + "reason": "终止原因" +} +``` + +完整的客户端信令实现示例可参考 [signaling_mqtt.js](https://github.com/emqx/emqx-multimedia-proxy/blob/main/apps/emqx_media_proxy_web/assets/js/signaling_mqtt.js)。你可以访问 http://localhost:4000/webrtc_mqtt 体验演示。 + +## 通过 MQTT 发送普通消息 + +多媒体服务器可通过以下 MQTT 主题向设备发送普通消息： + +- `$message/`：多媒体服务器向设备发送普通消息的 MQTT 主题。 +- `$message//multimedia_proxy`：设备向多媒体服务器发送任意消息的 MQTT 主题，消息会通过 `message_from_device` 方法转发给 AI 代理。 + +### 发送给设备的普通消息格式 + +多媒体服务器可通过 `$message/` 主题向设备发送如下类型的消息： + +当有 ASR 结果时，发送 `asr_response` 消息： + +```json +{ + "type": "asr_response", + "format": "merged" | "raw", + "results": <如果为 merged，则为识别文本；如果为 raw，则为 ASR 结果的 JSON 数组> +} +``` + +当 TTS 任务开始时，发送 `tts_begin` 消息： + +```json +{ + "type": "tts_begin", + "task_id": "task_id" +} +``` + +当文本被转换为语音且文本也需发送给设备时，发送 `tts_text` 消息： + +```json +{ + "type": "tts_text", + "task_id": "task_id", + "text": "text" +} +``` + +当 TTS 任务完成时，发送 `tts_complete` 消息： + +```json +{ + "type": "tts_complete", + "task_id": "task_id" +} +``` + +当 TTS 任务结束或被终止时，发送 `tts_terminate` 消息： + +```json +{ + "type": "tts_terminate", + "task_id": "task_id" +} +``` + +当代理向设备发送任意消息（通过 `message_to_device` 方法）时，发送 `message` 消息： + +```json +{ + "type": "message", + "payload": <任意格式的内容> +} +``` + +### 设备发送给多媒体服务器的任意消息格式 + +设备可通过 `$message//multimedia_proxy` 主题向多媒体服务器发送任意消息，格式如下： + +```json +{ + "type": "message", + "payload": <任意格式的内容> +} +``` + +## 多媒体服务器与 AI 代理的交互协议 + +通过 AI 代理可增强多媒体服务器的能力，例如根据业务逻辑处理 ASR 结果或向设备发送任意格式的消息。 + +多媒体服务器与 AI 代理之间通过简单的 JSON RPC 2.0 协议进行交互，消息通过 STDIO（标准输入输出）发送。消息以换行符（`\n`）分隔，且消息内容不得包含嵌入的换行符。 + +- **初始化**： + 建立 STDIO 连接后，代理需向多媒体服务器发送初始化消息，协商协议版本和配置： + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "method": "init", + "params": { + "protocol_version": "1.0", + "configs": { + "asr": { + // 若启用，服务器会在每次有新 ASR 结果时发送合并后的文本，否则由代理负责合并 ASR 结果 + "auto_merge": false + } + } + } + } + ``` + + 多媒体服务器会回复确认消息： + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "result": "ok" + } + ``` + +- **ASR 结果**： + 多媒体服务器会以通知形式向 AI 代理发送 ASR 结果，格式如下： + ```json + { + "jsonrpc": "2.0", + "method": "asr_result", + "params": { + // 当前设备 ID + "device_id": "device_id", + "text": "识别文本" + } + } + ``` + +- **TTS 及发送**： + + AI 代理可请求多媒体服务器进行 TTS 并将音频发送给指定设备。 + + 首先代理需发送 `tts_and_send_start` 消息以启动 TTS 任务，然后发送一个或多个 `tts_and_send` 消息以发送待转换为语音的文本。相同任务的文本可批量或分多次发送，但需使用相同的 `task_id`。最后，代理需发送 `tts_and_send_finish` 消息表示 TTS 任务结束。 + + 启动消息： + ```json + { + "jsonrpc": "2.0", + "id": "3", + "method": "tts_and_send_start", + "params": { + // 发送音频的设备 ID + "device_id": "device_id", + // + "task_id": "aaa", + "text": "待转换为语音的文本" + } + } + ``` + + 待转换文本可批量发送： + ```json + [ + { + "jsonrpc": "2.0", + "id": "4", + "method": "tts_and_send", + "params": { + // 发送音频的设备 ID + "device_id": "device_id", + // + "task_id": "aaa", + "text": "待转换为语音的文本" + } + }, + { + "jsonrpc": "2.0", + "id": "5", + "method": "tts_and_send", + "params": { + // 发送音频的设备 ID + "device_id": "device_id", + // + "task_id": "aaa", + "text": "，更多文本可批量发送" + } + }, + { + "jsonrpc": "2.0", + "id": "6", + "method": "tts_and_send_finish", + "params": { + // 发送音频的设备 ID + "device_id": "device_id", + // TTS 任务的 task_id + "task_id": "aaa" + } + } + ] + ``` + + `tts_and_send_start` 和 `tts_and_send_finish` 消息可与 `tts_and_send` 消息一起批量发送，也可单独发送。 + + 多媒体服务器会以 "ok" 或错误进行确认： + ```json + [ + { + "jsonrpc": "2.0", + "id": "4", + "result": "ok" + }, + { + "jsonrpc": "2.0", + "id": "5", + "result": "ok" + }, + { + "jsonrpc": "2.0", + "id": "6", + "result": "ok" + } + ] + ``` + +- **图像分析**： + AI 代理可请求多媒体服务器进行图像分析： + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "method": "image_analysis", + "params": { + // 要采集图像的设备 ID + "device_id": "device_id", + // 要采集和分析的图像数量 + "image_count": 2, + "capture_interval": 1000, // 采集间隔（毫秒） + "image_format": "jpeg", // 图像格式 + "user_prompt": "分析图像并提供见解" + } + } + ``` + + 多媒体服务器会返回分析结果： + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "result": { + "analysis_result": "分析结果" + } + } + ``` + +- **转发设备消息**： + 多媒体服务器会将收到的 `$message//multimedia_proxy` 主题消息通过 `message_from_device` 方法转发给 AI 代理： + ```json + { + "jsonrpc": "2.0", + "method": "message_from_device", + "params": { + // 发送消息的设备 ID + "device_id": "device_id", + "payload": "内容" + } + } + ``` + +- **发送消息到设备**： + AI 代理可通过多媒体服务器向设备发送任意消息： + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "method": "message_to_device", + "params": { + // 要发送消息的设备 ID + // 消息将通过 `$message/` MQTT 主题发送给设备 + "device_id": "device_id", + // 也可手动指定主题向任意设备发送消息 + // 若指定，则忽略 device_id 字段 + "topic": "topic/to/device", + "payload": "内容" + } + } + ``` + + 多媒体服务器会回复确认消息： + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "result": "ok" + } + ``` diff --git a/zh_CN/emqx-ai/multimedia-ai/overview.md b/zh_CN/emqx-ai/multimedia-ai/overview.md new file mode 100644 index 000000000..a9953bc34 --- /dev/null +++ b/zh_CN/emqx-ai/multimedia-ai/overview.md @@ -0,0 +1,22 @@ +# 多媒体服务器 + +EMQX 多媒体服务器是一个基于 WebRTC 技术构建的高性能多媒体处理平台。它能够接收来自客户端的 RTP/SRTP 音视频流，并集成了多种 AI 功能，如自动语音识别（ASR）、文本转语音（TTS）以及图像理解等。通过利用大模型能力，EMQX 多媒体服务器支持复杂的语音对话和工具调用，为需要音视频能力的 AI 应用提供了强大的技术支持。 + +## 核心功能 + +- **实时音视频处理**: 支持高质量的音视频流传输，确保低延迟和高可靠性的通信体验。 +- **自动语音识别（ASR）**: 提供精准的语音转文本功能，适用于语音助手、智能客服等应用。 +- **文本转语音（TTS）**: 支持多种语言和声音风格的文本转语音服务，提升用户交互体验。 +- **图像理解**: 集成先进的图像识别和分析技术，支持多种图像处理应用。 +- **大模型支持**: 利用大模型能力，实现复杂的语音对话和工具调用，满足多样化的业务需求。 +- **高度灵活的架构**: 适应不同规模和复杂度的应用场景，支持横向扩展和定制化配置。 +- **高可靠性**: 采用分布式架构设计，确保系统的高可用性和稳定性。 +- **低延迟**: 优化的网络传输和大模型处理机制，确保实时交互的流畅性。 + +## 应用场景 + +EMQX 多媒体服务器适用于以下场景： + +- **情感陪伴**: 通过语音对话和情感识别技术，为用户提供个性化的情感陪伴服务。 +- **智能客服**: 利用 ASR 和 TTS 技术，实现高效的语音交互，提升客户服务体验。 +- **设备智能控制**: 通过语音指令和图像识别，实现对智能设备的便捷控制。 diff --git a/zh_CN/emqx-ai/overview.md b/zh_CN/emqx-ai/overview.md new file mode 100644 index 000000000..438ff7441 --- /dev/null +++ b/zh_CN/emqx-ai/overview.md @@ -0,0 +1,9 @@ +# EMQX AI + +EMQX AI 是 EMQX 在人工智能领域的创新实践，致力于为 AI 应用提供高效、可靠的通信基础设施，分为以下两个方面： + +- **消息通信：** 通过 MCP over MQTT 协议，EMQX 为 AI 应用提供了轻量、低延迟的工具调用和消息传递方式，满足 AI 应用在模型推理、实时反馈、智能化数据采集和智能化控制等场景下的通信需求。详见 [MCP over MQTT](./mcp-over-mqtt/overview.md)。 + +- **多媒体流传输：** 通过基于 WebRTC 的多媒体服务器，EMQX 为 AI 应用提供了实时音视频流的传输能力，适用于智能语音、视频分析、图像理解、远程协作等场景。详见 [多媒体智能](./multimedia-ai/overview.md)。 + +以上两个方案相结合，为 AI 应用构建了坚实的通信底座，助力开发者快速搭建智能化、实时化的 AI 应用。 diff --git a/zh_CN/emqx-ai/sdks/mcp-sdk-erlang.md b/zh_CN/emqx-ai/sdks/mcp-sdk-erlang.md new file mode 100644 index 000000000..64f2d5635 --- /dev/null +++ b/zh_CN/emqx-ai/sdks/mcp-sdk-erlang.md @@ -0,0 +1,189 @@ +# Erlang SDK + +本文将使用 [MCP over MQTT Erlang SDK](https://github.com/emqx/mcp-mqtt-erl) 创建一个简单的 MCP over MQTT 服务端和客户端。 + +## 示例 + +### 创建一个简单的 MCP 客户端 + +```erlang +-module(mcp_mqtt_erl_client_demo). + +-behaviour(mcp_mqtt_erl_client_session). + +-export([ + client_name/0, + client_version/0, + client_capabilities/0, + received_non_mcp_message/3 +]). +-export([start_link/0]). + +%% The client name, version and capabilities. These info will be sent to the server during the MCP initialization. +client_name() -> + <<"emqx_tools/cli_demo">>. + +client_version() -> + <<"1.0">>. + +client_capabilities() -> #{}. + +%% Callbacks for non-MCP messages +received_non_mcp_message(MqttClient, Msg, State) -> + io:format("~p Received non-MCP message: ~p~n", [MqttClient, Msg]), + State. + +%% Start the MCP over MQTT client +start_link() -> + mcp_mqtt_erl_client:start_link( + #{ + server_name_filter => <<"#">>, + callback_mod => ?MODULE, + broker_address => {"127.0.0.1", 1883}, + mqtt_options => #{ + clientid => <<"emqx_tools_cli_demo">>, + username => <<"emqx">>, + password => <<"public">> + } + }). +``` + +其中 `server_name_filter` 是用于订阅 MCP 服务器的 MQTT 主题过滤器，`mqtt_options` 是传递给底层 MQTT 客户端的选项。 + +### 创建一个简单的 MCP 服务器 + +以下是一个简单的 MCP 服务器实现，支持两个工具：`tool1` 和 `tool2`。 + +```erlang +-module(mcp_mqtt_erl_server_demo). + +-behaviour(mcp_mqtt_erl_server_session). + +-include_lib("mcp_mqtt_erl/include/mcp_mqtt_erl_types.hrl"). +-include_lib("mcp_mqtt_erl/include/emqx_mcp_tools.hrl"). + +-export([ + start_link/2 +]). + +-export([ + server_name/0, + server_id/2, + server_version/0, + server_capabilities/0, + server_instructions/0, + server_meta/0 +]). + +-export([ + initialize/2, + list_resources/1, + read_resource/2, + call_tool/3, + list_tools/1 +]). + +-type loop_data() :: #{ + server_id => binary(), + client_info => map(), + client_capabilities => map(), + mcp_client_id => binary(), + _ => any() +}. + +-spec start_link(integer(), mcp_mqtt_erl_server:config()) -> gen_statem:start_ret(). +start_link(Idx, Conf) -> + mcp_mqtt_erl_server:start_link(Idx, Conf). + +server_version() -> + <>. + +server_name() -> + <<"emqx_tools/info_apis">>. + +server_id(ClientIdPrefix, Idx) -> + Idx1 = integer_to_binary(Idx), + Node = atom_to_binary(node()), + <>. + +server_capabilities() -> + #{ + resources => #{ + subscribe => true, + listChanged => true + }, + tools => #{ + listChanged => true + } + }. + +server_instructions() -> + <<"">>. + +server_meta() -> + #{ + authorization => #{ + roles => [<<"admin">>, <<"user">>] + } + }. + +-spec initialize(binary(), client_params()) -> {ok, loop_data()}. +initialize(ServerId, #{client_info := ClientInfo, client_capabilities := Capabilities, mcp_client_id := McpClientId}) -> + io:format("initialize --- server_id: ~p, client_info: ~p, client_capabilities: ~p, mcp_client_id: ~p~n", [ServerId, ClientInfo, Capabilities, McpClientId]), + {ok, #{ + server_id => ServerId, + client_info => ClientInfo, + client_capabilities => Capabilities, + mcp_client_id => McpClientId + }}. + +-spec call_tool(binary(), map(), loop_data()) -> {ok, call_tool_result() | [call_tool_result()], loop_data()}. +call_tool(ToolName, Args, LoopData) -> + io:format("call_tool --- tool_name: ~p, args: ~p~n", [ToolName, Args]), + Result = #{ + type => text, + text => <<"This is the result of the tool call">> + }, + {ok, Result, LoopData}. + +-spec list_tools(loop_data()) -> {ok, [tool_def()], loop_data()}. +list_tools(LoopData) -> + io:format("list_tools --- ~n", []), + Tools = [ + #{ + name => <<"tool1">>, + description => <<"This is tool 1">>, + inputSchema => #{ + type => <<"object">>, + properties => #{ + arg1 => #{ + type => <<"string">>, + description => <<"Argument 1">> + }, + arg2 => #{ + type => <<"integer">>, + description => <<"Argument 2">> + } + } + } + }, + #{ + name => <<"tool2">>, + description => <<"This is tool 2">>, + inputSchema => #{ + type => <<"object">>, + properties => #{ + arg1 => #{ + type => <<"string">>, + description => <<"Argument 1">> + }, + arg2 => #{ + type => <<"boolean">>, + description => <<"Argument 2">> + } + } + } + } + ], + {ok, Tools, LoopData}. +``` diff --git a/zh_CN/emqx-ai/sdks/mcp-sdk-esp32-c.md b/zh_CN/emqx-ai/sdks/mcp-sdk-esp32-c.md new file mode 100644 index 000000000..e69de29bb diff --git a/zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md b/zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md new file mode 100644 index 000000000..e69de29bb diff --git a/zh_CN/emqx-ai/sdks/mcp-sdk-python.md b/zh_CN/emqx-ai/sdks/mcp-sdk-python.md new file mode 100644 index 000000000..c713ce90f --- /dev/null +++ b/zh_CN/emqx-ai/sdks/mcp-sdk-python.md @@ -0,0 +1,130 @@ +# Python SDK + +本文将使用 [MCP over MQTT Python SDK](https://github.com/emqx/mcp-python-sdk) 创建一个简单的 MCP over MQTT 服务端和客户端。 + +## 创建一个演示项目 + +使用 [uv](https://docs.astral.sh/uv/) 创建一个演示项目： + +```bash +uv init mcp_over_mqtt_demo +cd mcp_over_mqtt_demo +``` + +## 创建一个简单的 MCP 服务器 + +在 `mcp_over_mqtt_demo` 项目中，让我们创建一个简单的 MCP 服务器，暴露一个计算器工具和一些资源。创建一个名为 `demo_mcp_server.py` 的文件，并添加以下代码： + +```python +# demo_mcp_server.py +from mcp.server.fastmcp import FastMCP + +# Create an MCP server +mcp = FastMCP( + "demo_mcp_server/calculator", + log_level="DEBUG", + mqtt_server_description="A simple FastMCP server that exposes a calculator tool", + mqtt_options={ + "host": "broker.emqx.io", + }, +) + +# Add an addition tool +@mcp.tool() +def add(a: int, b: int) -> int: + """Add two numbers""" + return a + b + +# Add a dynamic greeting resource +@mcp.resource("greeting://{name}") +def get_greeting(name: str) -> str: + """Get a personalized greeting""" + return f"Hello, {name}!" +``` + +## 创建一个简单的 MCP 客户端 + +在同一个项目中，让我们创建一个简单的 MCP 客户端，连接到服务器并列出可用的工具和资源。创建一个名为 `demo_mcp_client.py` 的文件，并添加以下代码： + +```python +# demo_mcp_client.py +import logging +import anyio +import mcp.client.mqtt as mcp_mqtt +from mcp.shared.mqtt import configure_logging + +configure_logging(level="INFO") +logger = logging.getLogger(__name__) + +async def on_mcp_server_discovered(client, server_name): + logger.info(f"Discovered {server_name}, connecting ...") + await client.initialize_mcp_server(server_name) + +async def on_mcp_connect(client, server_name, connect_result): + success, init_result = connect_result + if success == 'error': + logger.error(f"Failed to connect to {server_name}: {init_result}") + return + logger.info(f"Connected to {server_name}, success={success}, init_result={init_result}") + capabilities = init_result.capabilities + if capabilities.prompts: + prompts = await client.list_prompts(server_name) + logger.info(f"Prompts of {server_name}: {prompts}") + if capabilities.resources: + resources = await client.list_resources(server_name) + logger.info(f"Resources of {server_name}: {resources}") + resource_templates = await client.list_resource_templates(server_name) + logger.info(f"Resources templates of {server_name}: {resource_templates}") + if capabilities.tools: + toolsResult = await client.list_tools(server_name) + tools = toolsResult.tools + logger.info(f"Tools of {server_name}: {tools}") + if tools[0].name == "add": + result = await client.call_tool(server_name, name = tools[0].name, arguments={"a": 1, "b": 2}) + logger.info(f"Calling the tool as add(a=1, b=2), result: {result}") + +async def on_mcp_disconnect(client, server_name): + logger.info(f"Disconnected from {server_name}") + +async def main(): + async with mcp_mqtt.MqttTransportClient( + "test_client", + auto_connect_to_mcp_server = True, + on_mcp_server_discovered = on_mcp_server_discovered, + on_mcp_connect = on_mcp_connect, + on_mcp_disconnect = on_mcp_disconnect, + mqtt_options = mcp_mqtt.MqttOptions( + host="broker.emqx.io", + ) + ) as client: + client.start() + while True: + ## Simulate other works while the MQTT transport client is running in the background... + await anyio.sleep(20) + +if __name__ == "__main__": + anyio.run(main) +``` + +## 运行演示 + +1. 首先，安装所需的依赖项： + +```bash +uv add git+https://github.com/emqx/mcp-python-sdk --branch main +uv add "mcp[cli]" +``` + +2. 现在运行客户端： + +```bash +uv run demo_mcp_client.py +``` + +3. 打开一个新终端并运行服务器: + +```bash +uv run mcp run --transport mqtt ./demo_mcp_server.py +``` + +即便客户端先于服务器启动，它也会发现服务器并连接到它。客户端将列出可用的工具和资源，并使用参数 `a=1` 和 `b=2` 调用 `add` 工具。 diff --git a/zh_CN/emqx-ai/sdks/mcp-sdk-typescript.md b/zh_CN/emqx-ai/sdks/mcp-sdk-typescript.md new file mode 100644 index 000000000..e69de29bb diff --git a/zh_CN/emqx-ai/sdks/mcp-sdks-overview.md b/zh_CN/emqx-ai/sdks/mcp-sdks-overview.md new file mode 100644 index 000000000..e69de29bb diff --git a/zh_CN/emqx-ai/sdks/multimedia-ai/overview.md b/zh_CN/emqx-ai/sdks/multimedia-ai/overview.md new file mode 100644 index 000000000..d8c64ea20 --- /dev/null +++ b/zh_CN/emqx-ai/sdks/multimedia-ai/overview.md @@ -0,0 +1,11 @@ +# 与多媒体服务适配的客户端 + +支持 WebRTC 协议的客户端均可与多媒体 AI 服务进行交互。常见的客户端包括： + +- **Web 浏览器**: 现代浏览器（如 Chrome、Firefox、Edge、Safari）均支持 WebRTC，可以直接通过浏览器访问多媒体 AI 服务。 + +- **移动应用**: 通过集成 WebRTC SDK（如 [Pion](https://pion.ly)）来实现与多媒体 AI 服务的交互。 + +- **嵌入式设备**: 物联网设备可以通过集成与设备适配的 WebRTC 库来实现与多媒体 AI 服务的连接。如 [ESP32 WebRTC](https://github.com/espressif/esp-webrtc-solution) + +这里我们提供了基于 Web 浏览器的客户端代码示例，演示如何与多媒体服务进行交互：[Typescript WebRTC 示例](./webrtc-typescript.md)。 diff --git a/zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md b/zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md new file mode 100644 index 000000000..2f2cae53b --- /dev/null +++ b/zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md @@ -0,0 +1 @@ +# Typescript WebRTC 示例 diff --git a/zh_CN/emqx-ai/sdks/overview.md b/zh_CN/emqx-ai/sdks/overview.md new file mode 100644 index 000000000..bfb490aae --- /dev/null +++ b/zh_CN/emqx-ai/sdks/overview.md @@ -0,0 +1,3 @@ +# MCP over MQTT 和多媒体服务 SDK + +EMQX 提供了多种平台和编程语言的 SDK，帮助开发者快速集成 MCP over MQTT 和多媒体服务器的功能。这些 SDK 封装了底层的通信细节，使开发者能够专注于业务逻辑的实现。 From e7d0b260d05e78626b714642db512428a2612e9c Mon Sep 17 00:00:00 2001 From: fengzero Date: Thu, 11 Sep 2025 06:43:54 +0000 Subject: [PATCH 07/49] mcp c sdk --- zh_CN/emqx-ai/sdks/mcp-sdk-esp32-c.md | 57 ++++++++++++++++++++++++ zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md | 64 +++++++++++++++++++++++++++ 2 files changed, 121 insertions(+) diff --git a/zh_CN/emqx-ai/sdks/mcp-sdk-esp32-c.md b/zh_CN/emqx-ai/sdks/mcp-sdk-esp32-c.md index e69de29bb..5d486648f 100644 --- a/zh_CN/emqx-ai/sdks/mcp-sdk-esp32-c.md +++ b/zh_CN/emqx-ai/sdks/mcp-sdk-esp32-c.md @@ -0,0 +1,57 @@ +# ESP32 C SDK + +本文将使用 [MCP over MQTT C SDK for ESP32](https://github.com/mqtt-ai/esp-mcp-over-mqtt) 创建一个简单的 MCP over MQTT 服务端。目前仅支持 MCP 服务端，可以通过 Python SDK 创建 MCP 客户端进行交互。 + +此 SDK 使用的 MQTT 库是 ESP-IDF 中自带的 MQTT 库，适用于 ESP32 设备，因此必须在 ESP-IDF 环境中使用。 + +## 创建 MCP 服务器 + +参照 [ESP32 C SDK](https://github.com/mqtt-ai/esp-mcp-over-mqtt) README 中的说明，在基于 ESP-IDF 创建的 ESP32 项目中，创建一个名为 `mcp_server_example.c` 的文件，并添加以下代码： + +```c +#include "mcp_server.h" + +const char* get_temperature_callback(int n_args, property_t *args) { + // Read sensor data + float temp = read_temperature_sensor(); + + // Return JSON formatted result + static char result[64]; + snprintf(result, sizeof(result), "{\"temperature\": %.2f}", temp); + return result; +} + +// Define MCP tools +mcp_tool_t my_tools[] = { + { + .name = "get_temperature", + .description = "Get device temperature", + .property_count = 0, + .properties = NULL, + .call = get_temperature_callback + } +}; + +// Initialize MCP server +mcp_server_t *server = mcp_server_init( + "esp32_sensor", // Server name + "ESP32 Sensor MCP Server", // Description + "mqtt://broker.example.com", // MQTT Broker URI + "esp32_client_001", // Client ID + "username", // Username + "password", // Password + NULL // Certificate (optional) +); + +// Register tools +mcp_server_register_tool(server, 1, my_tools); + +// Start server +mcp_server_run(server); +``` + +## 基于 ESP-IDF 构建的项目中使用 MCP 服务器 + +详细使用方法请参考 [ESP32 MCP Demo](https://github.com/mqtt-ai/esp32-mcp-mqtt-tutorial/tree/main/samples/blog_3) 示例项目，该项目展示了如何在 ESP-IDF 项目中集成 MCP over MQTT C SDK for ESP32，并创建一个 MCP 服务器，以及如何通过 Python SDK 创建 MCP 客户端进行交互。 + +基于 ESP-IDF 构建好项目后，烧录到 ESP32 设备上运行，即可启动 MCP 服务器。 \ No newline at end of file diff --git a/zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md b/zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md index e69de29bb..9ca69e428 100644 --- a/zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md +++ b/zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md @@ -0,0 +1,64 @@ +# C SDK with Paho MQTT + +本文将使用 [MCP over MQTT C SDK with Paho MQTT](https://github.com/mqtt-ai/paho-mcp-over-mqtt) 创建一个简单的 MCP over MQTT 服务端。目前仅支持 MCP 服务端，可以通过 Python SDK 创建 MCP 客户端进行交互。 + +## 创建 MCP 服务器 + +参照 [C SDK with Paho MQTT](https://github.com/mqtt-ai/paho-mcp-over-mqtt) README 中的说明，安装完相关依赖以及 SDK 后，创建一个名为 `demo_mcp_server.c` 的文件，并添加以下代码： + +```c +#include "mcp_server.h" + +// Callback function to get temperature +const char* get_temperature_callback(int n_args, property_t *args) { + // Read sensor data + float temp = read_temperature_sensor(); + + // Return JSON formatted result + static char result[64]; + snprintf(result, sizeof(result), "{\"temperature\": %.2f}", temp); + return result; +} + +// Define MCP tools +mcp_tool_t my_tools[] = { + { + .name = "get_temperature", + .description = "Get device temperature", + .property_count = 0, + .properties = NULL, + .call = get_temperature_callback + } +}; + +// Initialize MCP server +mcp_server_t *server = mcp_server_init( + "sensor", // Server name + "Sensor MCP Server", // Description + "mqtt://broker.example.com", // MQTT Broker URI + "client_001", // Client ID + "username", // Username + "password", // Password + NULL // Certificate (optional) +); + +// Register tools +mcp_server_register_tool(server, 1, my_tools); + +// Start server +mcp_server_run(server); +``` + +## 使用 Python SDK 创建 MCP 客户端并运行 + +参考 [Python SDK](./mcp-sdk-python.md) 中的说明，创建一个 MCP 客户端连接到上述 C SDK 创建的 MCP 服务器，并调用 `get_temperature` 工具。 + +## 使用 CMake 构建并编译 MCP 服务器 + +CMake 使用可以参考 [paho-mcp-over-mqtt](https://github.com/mqtt-ai/paho-mcp-over-mqtt) 中的示例。 + +构建后，运行生成的可执行文件，即可启动 MCP 服务器。 + +```bash +./demo_mcp_server +``` \ No newline at end of file From 8880df13eef2be68defdacbd6e6d5957f87f53dc Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Fri, 12 Sep 2025 09:36:10 +0800 Subject: [PATCH 08/49] Release notes for v5.10.1 --- en_US/changes/breaking-changes-5.10.md | 6 + en_US/changes/changes-ee-v5.md | 153 +++++++++++++++++++++++++ 2 files changed, 159 insertions(+) diff --git a/en_US/changes/breaking-changes-5.10.md b/en_US/changes/breaking-changes-5.10.md index 84ac5bb39..fd80c7434 100644 --- a/en_US/changes/breaking-changes-5.10.md +++ b/en_US/changes/breaking-changes-5.10.md @@ -1,5 +1,11 @@ # Incompatible Changes in EMQX 5.10 +## 5.10.1 + +- [#15752](https://github.com/emqx/emqx/pull/15752) Listener connection rate limits (`max_conn_rate` and `max_conn_burst`) are now enforced per listener rather than per acceptor, restoring the pre-5.9.0 behavior. As a result, configurations from versions 5.9.0, 5.9.1, and 5.10.0 are incompatible: specified rates must be scaled up by the number of acceptors configured for respective listeners. + +- [#15753](https://github.com/emqx/emqx/pull/15753) Listener connection rate limits (`max_conn_rate` and `max_conn_burst`) are now enforced per listener rather than per acceptor, restoring the pre-5.9.0 behavior. As a result, configurations from versions 5.9.0 and 5.9.1 are incompatible: specified rates must be scaled up by the number of acceptors configured for respective listeners. + ## 5.10.0 - [#15289](https://github.com/emqx/emqx/pull/15289) Added a new `resource_opts.health_check_timeout` configuration to all Connectors, Actions, and Sources, with a default value of 60 seconds. If a health check takes more than this to return a response, the Connector/Action/Source will be deemed `disconnected`. diff --git a/en_US/changes/changes-ee-v5.md b/en_US/changes/changes-ee-v5.md index 552974f70..0bfe0ba0a 100644 --- a/en_US/changes/changes-ee-v5.md +++ b/en_US/changes/changes-ee-v5.md @@ -1,5 +1,158 @@ # EMQX Enterprise Version 5 +## 5.10.1 + +*Release Date: 2025-09-17* + +Make sure to check the breaking changes and known issues before upgrading to EMQX 5.10.0. + +### Enhancements + +- [#15499](https://github.com/emqx/emqx/pull/15499) Added a force deactivate alarm API endpoint to allow administrators to forcibly deactivate active alarms. + +- [#15294](https://github.com/emqx/emqx/pull/15294) Enhance LDAP authentication and authorization. + LDAP authorization now supports extended ACL rules in JSON format. + LDAP authenticaton now can fetch ACL rules from LDAP. These rules are cached in the client's metadata, so authorization is performed without additional LDAP queries. + +- [#15349](https://github.com/emqx/emqx/pull/15349) Optimize external resource management for authentication and authorization. Previously, EMQX could remain connected to a resource configured for a disabled authentication or authorization provider. + +- [#15360](https://github.com/emqx/emqx/pull/15360) Added support for writing data files in Parquet format for S3Tables Action. + +- [#15364](https://github.com/emqx/emqx/pull/15364) Add HTTP header configuration items to the OpenTelemetry integration to adapt to collectors with HTTP authentication. + +- [#15387](https://github.com/emqx/emqx/pull/15387) Improved Kinesis Producer Connector and Action health checks to mitigate the occurrence of rate limiting when calling `ListStreams` and `DescribeStream` APIs. Now, we limit the the calls per Connector to such APIs to 5/s and 10/s, respectively. If a Connector or Action cannot call their health check API before timing out, they will simply maintain their current status. If they receive a throttling response (e.g.: `LimitExceededException`), they will also maintain their current status. + + Introduced a new `resource_opts.health_check_interval_jitter` configuration to add an uniform random delay to `resource_opts.health_check_interval`, so that multiple Actions under the same Connector will seldom run their health checks simultaneously. + +- [#15399](https://github.com/emqx/emqx/pull/15399) Now, `node_dump` will export the current system configuration in HOCON format with redacted secrets. + +- [#15542](https://github.com/emqx/emqx/pull/15542) Upgraded our `erlcoud` library to `3.8.3.0`. This allows one to setup a S3 Connector without specifying Access Key Id and Secret Access Key, so long as the EC2 instance EMQX is running in has the correct IAM permissions to read/write to the configured bucket(s). + +- [#15585](https://github.com/emqx/emqx/pull/15585) Updated our `brod` client to version 4.4.4. This expands the supported Kafka API ranges, in particular due to the `JoinGroups` API `v0`-`v1` being deprecated. + +- [#15773](https://github.com/emqx/emqx/pull/15773) Throttle client ID registration during reconnects + + Added throttling for client ID registration if the previous session cleanup is still in progress. + This prevents instability when clients reconnect aggressively with the same client ID. + Affected clients will receive reason code `137` (Server Busy) in `CONNACK` with Reason-String "THROTTLED". + Clients should retry after the previous session cleanup has completed. + + Also fixed the reason code from `133` to `137` when there is another connection in the middle of registering the same client ID. + +- [#15845](https://github.com/emqx/emqx/pull/15845) Extended the `static_clientids` configuration of MQTT Connector to allow specifying usernames and passwords associated with each clientid. + +- [#15536](https://github.com/emqx/emqx/pull/15536) Disable the `node.global_gc_interval` configuration by default. + +- [#15539](https://github.com/emqx/emqx/pull/15539) Optimize Erlang VM parameters. + + - Increase the buffer sizes for distributed channels to 32MB to avoid `busy_dist_port` alarms during intensive Mnesia operations: `+zdbbl 32768` + - Disable scheduler busy-waiting to reduce CPU usage observed by the operating system: `+sbwt none +sbwtdcpu none +sbwtdio none` + - Set scheduler binding type to `db` to reduce message latency: `+stbt db` + +### Bug Fixes + +- [#15383](https://github.com/emqx/emqx/pull/15383) Fix a potential resource leak in MQTT bridge when the bridge fails to start. Previously, the topic index table was not properly cleaned up when the bridge failed to start. + +- [#15547](https://github.com/emqx/emqx/pull/15547) Fixed error when an HTTP request with a large body is sent. + +- [#15639](https://github.com/emqx/emqx/pull/15639) Fix incorrect counting of the packets.subscribe.auth_error metric. + +- [#15679](https://github.com/emqx/emqx/pull/15679) Fixed a bug where incorrect grouping of built-in authentication data for ExProto, JT/T 808, + GB/T 32960, and OCPP gateways caused mutual interference. + +- [#15699](https://github.com/emqx/emqx/pull/15699) Fixed issue that caused the built-in authentication data for gateways is incorrectly purged when a node is stopped or shutdown. + +- [#15785](https://github.com/emqx/emqx/pull/15785) Fixed a crash triggered by MQTT usernames containing non-ASCII characters when formatting network congestion alarms. + +- [#15797](https://github.com/emqx/emqx/pull/15797) Add an `encoding` alias for the `payload_encoding` parameter in the HTTP PUBLISH interface. + +- [#15342](https://github.com/emqx/emqx/pull/15342) Fixed NATS gateway crash when clientinfo override templates contain undefined packet fields by returning empty binary instead of undefined atom. + + +- [#15361](https://github.com/emqx/emqx/pull/15361) Fixed a function clause error when parsing a malformed `User-Property` pair where the pair length is wrong (too short). + +- [#15394](https://github.com/emqx/emqx/pull/15394) Fixed a very rare race condition in which Action metrics could end up in an inconsistent state. + +- [#15396](https://github.com/emqx/emqx/pull/15396) Removed redundant cleanup operations for shared subscriptions of disconnected clients, which were prone to crashes under high disconnect volume, resulting in potential inconsistencies in the global broker state. + +- [#15416](https://github.com/emqx/emqx/pull/15416) Fixed occasional warning-level log events and crashes during session expiration of WebSocket connections, introduced by recent WebSocket performance improvements. These had no impact on broker capacity, but produced log entries like the following: + * `error: {function_clause,[{gen_tcp,send,[closed,[]],[{file,“gen_tcp.erl”},{line,966}]},{cowboy_websocket_linger,commands,3,[{file,“cowboy_websocket_linger.erl”},{line,665}]},...` + * `message: {tcp,#Port<0.364>,<<136,130,...>>}, msg: emqx_session_mem_unknown_message` + +- [#15553](https://github.com/emqx/emqx/pull/15553) Fixes an issue with helm chart when all nodes except one will be crashing if the chart is deployed with default values. + +- [#15580](https://github.com/emqx/emqx/pull/15580) Add emqxLicenseSecretRef variable to EMQX Enterprise helm chart, allowing users to specify a Kubernetes secret containing the EMQX license key. This fixes the issue with defunct emqxLicenseSecretName variable. + +- [#15581](https://github.com/emqx/emqx/pull/15581) Upgrade OTP version from 26.2.5.2 to 26.2.5.14 + + This upgrade includes two TLS-related fixes relevant to EMQX: + + - Fixed a crash in TLS connections caused by a race condition during certificate renewal. + - Added support for RSA certificates signed with PSS parameters. Previously TLS handshake may fail with `invalid_signature`. + + +- [#15603](https://github.com/emqx/emqx/pull/15603) Fixed an issue with the MQTT bridge when a stale connection was displayed as `Connected' and the connection was not re-established. + +- [#15616](https://github.com/emqx/emqx/pull/15616) Consider Kafka connection healthy if `topic_authorization_failed` is received for the default probing topic. + +- [#15683](https://github.com/emqx/emqx/pull/15683) Fix exhook TLS options to allow verify server host name. + +- [#15706](https://github.com/emqx/emqx/pull/15706) Fixed an issue that could make Message Transformations and Schema Validations behave inconsistently. If one deleted a Message Transformation / Schema Validation, and then disabled one of the items that followed the deleted one, it would remain enabled. + +- [#15708](https://github.com/emqx/emqx/pull/15708) Previously, when restarting a node which had external schema registries configured, they would not load properly. + +- [#15712](https://github.com/emqx/emqx/pull/15712) Fix node boot-up failure during rolling upgrade from older versions (before 5.9) + + In previous EMQX versions (before 5.9), a bug in the ZIP timestamp encoder could store an invalid “seconds” value in archive entries (values corresponding to the 30th or 31st 2-second slot in DOS time format). + +- [#15788](https://github.com/emqx/emqx/pull/15788) Etcd cluster discovery issue + + Resolved an issue where EMQX nodes from different clusters could mistakenly join each other when using a shared etcd server. + This was caused by a bug in the etcd client library. + +- [#15794](https://github.com/emqx/emqx/pull/15794) Ensure that any changes to connection rate limits take effect immediately after the listener update has completed. Previously, parts of internal limiter state were not directly affected by configuration changes. For example, after increasing the burst rate, the effective rate limit could appear stricter than expected. + +- [#15810](https://github.com/emqx/emqx/pull/15810) The original `sparkplug_{en,de}code` rule functions, when handling `bytes_value` metrics values, did not base64 decode/encode the data, respectively, thus not following the Protobuf spec. + + https://protobuf.dev/programming-guides/json/ + + Thus, here, we introduce new `spb_{en,de}code` rule functions that translate such fields, to avoid breaking backwards compatibility, and deprecate the old `sparkplug_{en,de}code` rule functions. + +- [#15818](https://github.com/emqx/emqx/pull/15818) Corrected handling of `{allow|deny, all}` ACL rules. + + Previously, these rules were internally translated to match `#`, which incorrectly failed to match topics prefixed with `$` (e.g. `$testtopic/1`) due to MQTT spec restrictions. + Now, a special internal value is used to ensure `{allow|deny, all}` rules correctly match any topic, including `$`-prefixed ones. + +- [#15822](https://github.com/emqx/emqx/pull/15822) Fixed an issue where the OCPP connection would crash after sending a certain number of messages. + +- [#15826](https://github.com/emqx/emqx/pull/15826) Previously, if the user used in a Kafka Consumer Connector did not have permissions to read the special `____emqx_consumer_probe` group used for health checks, the health check would fail. Now, if the Kafka broker returns an ACL denied response, the connection is considered healthy. + +- [#15827](https://github.com/emqx/emqx/pull/15827) Fixed atom and process leaks in the GreptimeDB driver. + + Fixed a `function_clause` error that could arise if certain incorrect write syntaxes were used in GreptimeDB Actions. + +- [#15836](https://github.com/emqx/emqx/pull/15836) Enriched the returned information when a Kafka Consumer Source fails to be added, for example, due to denied topic ACLs. + +- [#15844](https://github.com/emqx/emqx/pull/15844) Added validation to forbid adding empty usernames to the built-in database authenticator. Such users cannot be deleted via the HTTP API later, since they mess up the API path. + + If you have such an user and wish to delete it, run the following in an EMQX console: + + ```erlang + mria:transaction(emqx_authn_shard, fun() -> mnesia:delete(emqx_authn_mnesia, {'mqtt:global',<<>>}, write) end). + ``` + +- [#15850](https://github.com/emqx/emqx/pull/15850) Fixed an issue with the MQTT bridge when a stale connection was displayed as `Connected' and the connection was not re-established. + +- [#15863](https://github.com/emqx/emqx/pull/15863) Fix license quota alarm text. + +- [#15866](https://github.com/emqx/emqx/pull/15866) Upgrade Kafka producer lib wollf to 4.0.12 to improve handling of temporarily missing partitions in Kafka metadata responses. + + In rare race conditions, Kafka may return an incomplete partition list. + Previously, this was only handled when a topic was recreated with fewer partitions, but not when partitions were temporarily missing. + This gap could cause the partition producer to stall and block shutdown indefinitely. + +- [#15872](https://github.com/emqx/emqx/pull/15872) Eliminate warning log 'unclean_terminate' when disconnected after CONNACK is sent with a non-zero reason code. + ## 5.10.0 *Release Date: 2025-06-10* From 4d253d7861b04a123ad952a8c0c4f1836544df84 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Fri, 12 Sep 2025 17:30:09 +0800 Subject: [PATCH 09/49] Removed some PRs not for 5.10.1 --- en_US/changes/changes-ee-v5.md | 98 +++++++++++++--------------------- 1 file changed, 37 insertions(+), 61 deletions(-) diff --git a/en_US/changes/changes-ee-v5.md b/en_US/changes/changes-ee-v5.md index 0bfe0ba0a..77fef2c98 100644 --- a/en_US/changes/changes-ee-v5.md +++ b/en_US/changes/changes-ee-v5.md @@ -8,103 +8,79 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ ### Enhancements +#### Observability + - [#15499](https://github.com/emqx/emqx/pull/15499) Added a force deactivate alarm API endpoint to allow administrators to forcibly deactivate active alarms. +- [#15364](https://github.com/emqx/emqx/pull/15364) Added HTTP header configuration items to the OpenTelemetry integration to adapt to collectors with HTTP authentication. -- [#15294](https://github.com/emqx/emqx/pull/15294) Enhance LDAP authentication and authorization. - LDAP authorization now supports extended ACL rules in JSON format. - LDAP authenticaton now can fetch ACL rules from LDAP. These rules are cached in the client's metadata, so authorization is performed without additional LDAP queries. +#### Access Control -- [#15349](https://github.com/emqx/emqx/pull/15349) Optimize external resource management for authentication and authorization. Previously, EMQX could remain connected to a resource configured for a disabled authentication or authorization provider. +- [#15294](https://github.com/emqx/emqx/pull/15294) Enhanced LDAP authentication and authorization. + LDAP authorization now supports an extended ACL rule format using JSON, in addition to the existing simple topic list. ACL rules can also be fetched from LDAP during authentication based on client information, and are cached in the client’s metadata to avoid repeated LDAP queries during authorization. +- [#15349](https://github.com/emqx/emqx/pull/15349) Optimized external resource management for authentication and authorization. Previously, EMQX could remain connected to a resource configured for a disabled authentication or authorization provider. -- [#15360](https://github.com/emqx/emqx/pull/15360) Added support for writing data files in Parquet format for S3Tables Action. +#### Data Integration -- [#15364](https://github.com/emqx/emqx/pull/15364) Add HTTP header configuration items to the OpenTelemetry integration to adapt to collectors with HTTP authentication. +- [#15360](https://github.com/emqx/emqx/pull/15360) Added support for writing data files in Parquet format for Amazon S3 Tables Action. -- [#15387](https://github.com/emqx/emqx/pull/15387) Improved Kinesis Producer Connector and Action health checks to mitigate the occurrence of rate limiting when calling `ListStreams` and `DescribeStream` APIs. Now, we limit the the calls per Connector to such APIs to 5/s and 10/s, respectively. If a Connector or Action cannot call their health check API before timing out, they will simply maintain their current status. If they receive a throttling response (e.g.: `LimitExceededException`), they will also maintain their current status. +- [#15387](https://github.com/emqx/emqx/pull/15387) Added rate limiting to Kinesis Producer Connector and Action health checks to comply with AWS API quotas and improve cluster behavior. - Introduced a new `resource_opts.health_check_interval_jitter` configuration to add an uniform random delay to `resource_opts.health_check_interval`, so that multiple Actions under the same Connector will seldom run their health checks simultaneously. + - Health check calls to `ListStreams` and `DescribeStream` are now limited to 5/s and 10/s per Connector, respectively, matching AWS rate limits. + - A distributed limiter is coordinated by a core node in the cluster to enforce these limits consistently. + - If a health check is throttled or times out, the Connector or Action will now retain its previous status instead of being marked as disconnected. -- [#15399](https://github.com/emqx/emqx/pull/15399) Now, `node_dump` will export the current system configuration in HOCON format with redacted secrets. + Also introduced a new `resource_opts.health_check_interval_jitter`, which adds a uniform random delay to `resource_opts.health_check_interval` to reduce the chance of multiple Actions under the same Connector running health checks at the same time. - [#15542](https://github.com/emqx/emqx/pull/15542) Upgraded our `erlcoud` library to `3.8.3.0`. This allows one to setup a S3 Connector without specifying Access Key Id and Secret Access Key, so long as the EC2 instance EMQX is running in has the correct IAM permissions to read/write to the configured bucket(s). +- [#15845](https://github.com/emqx/emqx/pull/15845) The `static_clientids` configuration for the MQTT Connector now supports specifying a username and password for each client ID. This is particularly useful for scenarios like connecting to Azure IoT Hub, where each device (client ID) requires a unique set of credentials. This enhancement helps ensure successful connections across multiple nodes in a clustered environment. -- [#15585](https://github.com/emqx/emqx/pull/15585) Updated our `brod` client to version 4.4.4. This expands the supported Kafka API ranges, in particular due to the `JoinGroups` API `v0`-`v1` being deprecated. - -- [#15773](https://github.com/emqx/emqx/pull/15773) Throttle client ID registration during reconnects - - Added throttling for client ID registration if the previous session cleanup is still in progress. - This prevents instability when clients reconnect aggressively with the same client ID. - Affected clients will receive reason code `137` (Server Busy) in `CONNACK` with Reason-String "THROTTLED". - Clients should retry after the previous session cleanup has completed. - - Also fixed the reason code from `133` to `137` when there is another connection in the middle of registering the same client ID. - -- [#15845](https://github.com/emqx/emqx/pull/15845) Extended the `static_clientids` configuration of MQTT Connector to allow specifying usernames and passwords associated with each clientid. - -- [#15536](https://github.com/emqx/emqx/pull/15536) Disable the `node.global_gc_interval` configuration by default. - -- [#15539](https://github.com/emqx/emqx/pull/15539) Optimize Erlang VM parameters. +#### CLI - - Increase the buffer sizes for distributed channels to 32MB to avoid `busy_dist_port` alarms during intensive Mnesia operations: `+zdbbl 32768` - - Disable scheduler busy-waiting to reduce CPU usage observed by the operating system: `+sbwt none +sbwtdcpu none +sbwtdio none` - - Set scheduler binding type to `db` to reduce message latency: `+stbt db` +- [#15399](https://github.com/emqx/emqx/pull/15399) The `node_dump` tool now exports the current system configuration in HOCON format, with sensitive information (such as passwords and secrets) automatically redacted for security. ### Bug Fixes -- [#15383](https://github.com/emqx/emqx/pull/15383) Fix a potential resource leak in MQTT bridge when the bridge fails to start. Previously, the topic index table was not properly cleaned up when the bridge failed to start. - -- [#15547](https://github.com/emqx/emqx/pull/15547) Fixed error when an HTTP request with a large body is sent. - -- [#15639](https://github.com/emqx/emqx/pull/15639) Fix incorrect counting of the packets.subscribe.auth_error metric. - -- [#15679](https://github.com/emqx/emqx/pull/15679) Fixed a bug where incorrect grouping of built-in authentication data for ExProto, JT/T 808, - GB/T 32960, and OCPP gateways caused mutual interference. - -- [#15699](https://github.com/emqx/emqx/pull/15699) Fixed issue that caused the built-in authentication data for gateways is incorrectly purged when a node is stopped or shutdown. +#### API -- [#15785](https://github.com/emqx/emqx/pull/15785) Fixed a crash triggered by MQTT usernames containing non-ASCII characters when formatting network congestion alarms. +- [#15547](https://github.com/emqx/emqx/pull/15547) Resolved an issue where EMQX would fail to process HTTP requests with large bodies (e.g., 10MB) in the REST API. +- [#15797](https://github.com/emqx/emqx/pull/15797) To improve compatibility with EMQX 4.x, the `encoding` parameter has been reintroduced in the batch publish HTTP API (`/api/v5/publish/bulk`) as an alias for `payload_encoding`. This change addresses migration issues for users relying on the original `encoding` parameter, and ensures existing integrations using EMQX v4 APIs can continue working without requiring software-level changes. -- [#15797](https://github.com/emqx/emqx/pull/15797) Add an `encoding` alias for the `payload_encoding` parameter in the HTTP PUBLISH interface. +#### Observability -- [#15342](https://github.com/emqx/emqx/pull/15342) Fixed NATS gateway crash when clientinfo override templates contain undefined packet fields by returning empty binary instead of undefined atom. +- [#15785](https://github.com/emqx/emqx/pull/15785) Resolved a crash that occurred when MQTT usernames containing non-ASCII characters were used in formatting network congestion alarm messages. +#### Gateway -- [#15361](https://github.com/emqx/emqx/pull/15361) Fixed a function clause error when parsing a malformed `User-Property` pair where the pair length is wrong (too short). +- [#15342](https://github.com/emqx/emqx/pull/15342) Fixed a crash in the NATS gateway caused by client info override templates referencing undefined packet fields. The system now returns an empty binary instead of undefined atom. -- [#15394](https://github.com/emqx/emqx/pull/15394) Fixed a very rare race condition in which Action metrics could end up in an inconsistent state. +#### Core MQTT Functions -- [#15396](https://github.com/emqx/emqx/pull/15396) Removed redundant cleanup operations for shared subscriptions of disconnected clients, which were prone to crashes under high disconnect volume, resulting in potential inconsistencies in the global broker state. -- [#15416](https://github.com/emqx/emqx/pull/15416) Fixed occasional warning-level log events and crashes during session expiration of WebSocket connections, introduced by recent WebSocket performance improvements. These had no impact on broker capacity, but produced log entries like the following: +- [#15361](https://github.com/emqx/emqx/pull/15361) Fixed a `function_clause` error when parsing a malformed `User-Property` pair with invalid (too short) length. +- [#15396](https://github.com/emqx/emqx/pull/15396) Removed redundant cleanup operations for shared subscriptions of disconnected clients. These operations were prone to crashes under high disconnect volumes and could lead to inconsistencies in the global broker state. +- [#15416](https://github.com/emqx/emqx/pull/15416) Fixed occasional warning-level log events and crashes during session expiration of WebSocket connections. This issue was introduced by recent WebSocket performance improvements. If did not affect broker capacity, but produced log entries like the following: * `error: {function_clause,[{gen_tcp,send,[closed,[]],[{file,“gen_tcp.erl”},{line,966}]},{cowboy_websocket_linger,commands,3,[{file,“cowboy_websocket_linger.erl”},{line,665}]},...` * `message: {tcp,#Port<0.364>,<<136,130,...>>}, msg: emqx_session_mem_unknown_message` -- [#15553](https://github.com/emqx/emqx/pull/15553) Fixes an issue with helm chart when all nodes except one will be crashing if the chart is deployed with default values. - -- [#15580](https://github.com/emqx/emqx/pull/15580) Add emqxLicenseSecretRef variable to EMQX Enterprise helm chart, allowing users to specify a Kubernetes secret containing the EMQX license key. This fixes the issue with defunct emqxLicenseSecretName variable. - -- [#15581](https://github.com/emqx/emqx/pull/15581) Upgrade OTP version from 26.2.5.2 to 26.2.5.14 - - This upgrade includes two TLS-related fixes relevant to EMQX: - - - Fixed a crash in TLS connections caused by a race condition during certificate renewal. - - Added support for RSA certificates signed with PSS parameters. Previously TLS handshake may fail with `invalid_signature`. +#### Data Integration -- [#15603](https://github.com/emqx/emqx/pull/15603) Fixed an issue with the MQTT bridge when a stale connection was displayed as `Connected' and the connection was not re-established. +- [#15394](https://github.com/emqx/emqx/pull/15394) Fixed a rare race condition where Action metrics could become inconsistent due to unexpected asynchronous replies. +- [#15603](https://github.com/emqx/emqx/pull/15603) Fixed an issue in the MQTT bridge where a stale connection could be shown as `Connected` and would not automatically reconnect. -- [#15616](https://github.com/emqx/emqx/pull/15616) Consider Kafka connection healthy if `topic_authorization_failed` is received for the default probing topic. +#### Deployment -- [#15683](https://github.com/emqx/emqx/pull/15683) Fix exhook TLS options to allow verify server host name. -- [#15706](https://github.com/emqx/emqx/pull/15706) Fixed an issue that could make Message Transformations and Schema Validations behave inconsistently. If one deleted a Message Transformation / Schema Validation, and then disabled one of the items that followed the deleted one, it would remain enabled. +- [#15553](https://github.com/emqx/emqx/pull/15553) Fixed an issue in the Helm chart where deploying EMQX with default values started multiple replicas and caused all nodes except one to crash. The chart now defaults to a single replica, since clustered deployments require an Commercial License. -- [#15708](https://github.com/emqx/emqx/pull/15708) Previously, when restarting a node which had external schema registries configured, they would not load properly. - [#15712](https://github.com/emqx/emqx/pull/15712) Fix node boot-up failure during rolling upgrade from older versions (before 5.9) In previous EMQX versions (before 5.9), a bug in the ZIP timestamp encoder could store an invalid “seconds” value in archive entries (values corresponding to the 30th or 31st 2-second slot in DOS time format). +#### Clustering + + - [#15788](https://github.com/emqx/emqx/pull/15788) Etcd cluster discovery issue Resolved an issue where EMQX nodes from different clusters could mistakenly join each other when using a shared etcd server. @@ -123,7 +99,7 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ Previously, these rules were internally translated to match `#`, which incorrectly failed to match topics prefixed with `$` (e.g. `$testtopic/1`) due to MQTT spec restrictions. Now, a special internal value is used to ensure `{allow|deny, all}` rules correctly match any topic, including `$`-prefixed ones. -- [#15822](https://github.com/emqx/emqx/pull/15822) Fixed an issue where the OCPP connection would crash after sending a certain number of messages. + - [#15826](https://github.com/emqx/emqx/pull/15826) Previously, if the user used in a Kafka Consumer Connector did not have permissions to read the special `____emqx_consumer_probe` group used for health checks, the health check would fail. Now, if the Kafka broker returns an ACL denied response, the connection is considered healthy. From f3f253746dcfc23677bc3efe211629fabc0ce095 Mon Sep 17 00:00:00 2001 From: zmstone Date: Fri, 12 Sep 2025 11:40:36 +0200 Subject: [PATCH 10/49] chore: delete stale file --- zh_CN/configuration/configuration.json | 23216 ----------------------- 1 file changed, 23216 deletions(-) delete mode 100644 zh_CN/configuration/configuration.json diff --git a/zh_CN/configuration/configuration.json b/zh_CN/configuration/configuration.json deleted file mode 100644 index 9056df3cf..000000000 --- a/zh_CN/configuration/configuration.json +++ /dev/null @@ -1,23216 +0,0 @@ -[ - { - "tags" : [ - - ], - "paths" : [ - - ], - "full_name" : "Root Config Keys", - "fields" : [ - { - "type" : { - "name" : "broker:listeners", - "kind" : "struct" - }, - "name" : "listeners", - "importance" : "high", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:mqtt", - "kind" : "struct" - }, - "name" : "mqtt", - "importance" : "medium", - "desc" : "Global MQTT configuration.\nThe configs here work as default values which can be overridden in zone configs", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "broker:zone", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "zones", - "importance" : "low", - "desc" : "A zone is a set of configs grouped by the zone name.\nFor flexible configuration mapping, the name can be set to a listener's zone config.\nNOTE: A built-in zone named default is auto created and can not be deleted.", - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "members" : [ - { - "name" : "authn-builtin_db:authentication", - "kind" : "struct" - }, - { - "name" : "authn-mysql:authentication", - "kind" : "struct" - }, - { - "name" : "authn-postgresql:authentication", - "kind" : "struct" - }, - { - "name" : "authn-mongodb:standalone", - "kind" : "struct" - }, - { - "name" : "authn-mongodb:replica-set", - "kind" : "struct" - }, - { - "name" : "authn-mongodb:sharded-cluster", - "kind" : "struct" - }, - { - "name" : "authn-redis:standalone", - "kind" : "struct" - }, - { - "name" : "authn-redis:cluster", - "kind" : "struct" - }, - { - "name" : "authn-redis:sentinel", - "kind" : "struct" - }, - { - "name" : "authn-http:get", - "kind" : "struct" - }, - { - "name" : "authn-http:post", - "kind" : "struct" - }, - { - "name" : "authn-jwt:hmac-based", - "kind" : "struct" - }, - { - "name" : "authn-jwt:public-key", - "kind" : "struct" - }, - { - "name" : "authn-jwt:jwks", - "kind" : "struct" - }, - { - "name" : "authn-scram-builtin_db:authentication", - "kind" : "struct" - } - ], - "kind" : "union" - } - }, - "raw_default" : [ - - ], - "name" : "authentication", - "importance" : "low", - "desc" : "Default authentication configs for all MQTT listeners.\n\nFor per-listener overrides see authentication in listener configs\n\nThis option can be configured with:\n
    \n
  • []: The default value, it allows *ALL* logins
    • \n
  • one: For example {enable:true,backend:\"built_in_database\",mechanism=\"password_based\"}
    • \n
  • chain: An array of structs.
    • \n
\n\nWhen a chain is configured, the login credentials are checked against the backends per the configured order, until an 'allow' or 'deny' decision can be made.\n\nIf there is no decision after a full chain exhaustion, the login is rejected.", - "default" : { - "oneliner" : true, - "hocon" : "[]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "authorization", - "kind" : "struct" - }, - "name" : "authorization", - "importance" : "high", - "desc" : "Authorization a.k.a. ACL.
\nIn EMQX, MQTT client access control is extremely flexible.
\nAn out-of-the-box set of authorization data sources are supported.\nFor example,
\n'file' source is to support concise and yet generic ACL rules in a file;
\n'built_in_database' source can be used to store per-client customizable rule sets,\nnatively in the EMQX node;
\n'http' source to make EMQX call an external HTTP API to make the decision;
\n'PostgreSQL' etc. to look up clients or rules from external databases", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "node", - "kind" : "struct" - }, - "name" : "node", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "cluster", - "kind" : "struct" - }, - "name" : "cluster", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "log", - "kind" : "struct" - }, - "name" : "log", - "importance" : "high", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "rpc", - "kind" : "struct" - }, - "name" : "rpc", - "importance" : "low", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker", - "kind" : "struct" - }, - "name" : "broker", - "desc" : "Message broker options.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:sys_topics", - "kind" : "struct" - }, - "name" : "sys_topics", - "desc" : "System topics configuration.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:force_shutdown", - "kind" : "struct" - }, - "name" : "force_shutdown", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:force_gc", - "kind" : "struct" - }, - "name" : "force_gc", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:sysmon", - "kind" : "struct" - }, - "name" : "sysmon", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:alarm", - "kind" : "struct" - }, - "name" : "alarm", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge:bridges", - "kind" : "struct" - }, - "name" : "bridges", - "importance" : "low", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "retainer", - "kind" : "struct" - }, - "name" : "retainer", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "statsd", - "kind" : "struct" - }, - "name" : "statsd", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "modules:delayed", - "kind" : "struct" - }, - "name" : "delayed", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "modules:telemetry", - "kind" : "struct" - }, - "name" : "telemetry", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "plugin:plugins", - "kind" : "struct" - }, - "name" : "plugins", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "dashboard", - "kind" : "struct" - }, - "name" : "dashboard", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "prometheus", - "kind" : "struct" - }, - "name" : "prometheus", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "rule_engine", - "kind" : "struct" - }, - "name" : "rule_engine", - "importance" : "low", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "exhook", - "kind" : "struct" - }, - "name" : "exhook", - "importance" : "low", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "authn-psk:psk_authentication", - "kind" : "struct" - }, - "name" : "psk_authentication", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter", - "kind" : "struct" - }, - "name" : "limiter", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "api_key", - "kind" : "struct" - }, - "name" : "api_key", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "key_license", - "kind" : "struct" - }, - "name" : "license", - "desc" : "Defines the EMQX Enterprise license. \n\n\nThe default license has 100 connections limit, it is issued on 2023-01-09 and valid for 5 years (1825 days).\n\nEMQX comes with a default trial license. For production use, please \nvisit https://www.emqx.com/apply-licenses/emqx to apply.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "schema_registry", - "kind" : "struct" - }, - "name" : "schema_registry", - "aliases" : [ - - ] - } - ] - }, - { - "tags" : [ - - ], - "paths" : [ - "api_key" - ], - "full_name" : "api_key", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "bootstrap_file", - "desc" : "Bootstrap file is used to add an api_key when emqx is launched,\n the format is:\n ```\n 7e729ae70d23144b:2QILI9AcQ9BYlVqLDHQNWN2saIjBV4egr1CZneTNKr9CpK\n ec3907f865805db0:Ee3taYltUKtoBVD9C3XjQl9C6NXheip8Z9B69BpUv5JxVHL\n ```", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "API Key, can be used to request API other than the management API key and the Dashboard user management API" - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:file", - "fields" : [ - { - "type" : { - "name" : "file", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "path", - "desc" : "Path to the file which contains the ACL rules.\nIf the file provisioned before starting EMQX node,\nit can be placed anywhere as long as EMQX has read access to it.\nThat is, EMQX will treat it as read only.\n\nIn case the rule-set is created or updated from EMQX Dashboard or HTTP API,\na new file will be created and placed in `authz` subdirectory inside EMQX's `data_dir`,\nand the old file will not be used anymore.", - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using a static file." - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:http_get", - "fields" : [ - { - "type" : { - "name" : "http", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "url", - "desc" : "URL of the auth server.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "30s", - "name" : "request_timeout", - "desc" : "HTTP request timeout.", - "default" : { - "oneliner" : true, - "hocon" : "\"30s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "name" : "body", - "desc" : "HTTP request body.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "connect_timeout", - "desc" : "The timeout when connecting to the HTTP server.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "enable_pipelining", - "desc" : "A positive integer. Whether to send HTTP requests continuously, when set to 1, it means that after each HTTP request is sent, you need to wait for the server to return and then continue to send the next request.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "name" : "max_retries", - "desc" : "Deprecated since 5.0.4.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "The pool size.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "connector-http:request", - "kind" : "struct" - }, - "name" : "request", - "desc" : "Configure HTTP request parameters.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "name" : "retry_interval", - "desc" : "Deprecated since 5.0.4.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "get", - "kind" : "singleton" - }, - "name" : "method", - "desc" : "HTTP method.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "[{binary(), binary()}]", - "kind" : "primitive" - }, - "raw_default" : { - "keep-alive" : "timeout=30, max=1000", - "connection" : "keep-alive", - "cache-control" : "no-cache", - "accept" : "application/json" - }, - "name" : "headers", - "desc" : "List of HTTP headers (without content-type).", - "default" : { - "oneliner" : false, - "hocon" : "{\n accept = \"application/json\"\n \"cache-control\" = \"no-cache\"\n connection = \"keep-alive\"\n \"keep-alive\" = \"timeout=30, max=1000\"\n}\n" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using an external HTTP server (via GET requests)." - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:http_post", - "fields" : [ - { - "type" : { - "name" : "http", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "url", - "desc" : "URL of the auth server.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "30s", - "name" : "request_timeout", - "desc" : "HTTP request timeout.", - "default" : { - "oneliner" : true, - "hocon" : "\"30s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "name" : "body", - "desc" : "HTTP request body.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "connect_timeout", - "desc" : "The timeout when connecting to the HTTP server.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "enable_pipelining", - "desc" : "A positive integer. Whether to send HTTP requests continuously, when set to 1, it means that after each HTTP request is sent, you need to wait for the server to return and then continue to send the next request.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "name" : "max_retries", - "desc" : "Deprecated since 5.0.4.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "The pool size.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "connector-http:request", - "kind" : "struct" - }, - "name" : "request", - "desc" : "Configure HTTP request parameters.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "name" : "retry_interval", - "desc" : "Deprecated since 5.0.4.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "post", - "kind" : "singleton" - }, - "name" : "method", - "desc" : "HTTP method.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "[{binary(), binary()}]", - "kind" : "primitive" - }, - "raw_default" : { - "keep-alive" : "timeout=30, max=1000", - "content-type" : "application/json", - "connection" : "keep-alive", - "cache-control" : "no-cache", - "accept" : "application/json" - }, - "name" : "headers", - "desc" : "List of HTTP Headers.", - "default" : { - "oneliner" : false, - "hocon" : "{\n accept = \"application/json\"\n \"cache-control\" = \"no-cache\"\n connection = \"keep-alive\"\n \"content-type\" = \"application/json\"\n \"keep-alive\" = \"timeout=30, max=1000\"\n}\n" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using an external HTTP server (via POST requests)." - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:mnesia", - "fields" : [ - { - "type" : { - "name" : "built_in_database", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using a built-in database (mnesia)." - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:mongo_rs", - "fields" : [ - { - "type" : { - "name" : "mongodb", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "collection", - "desc" : "`MongoDB` collection containing the authorization data.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "filter", - "desc" : "Conditional expression that defines the filter condition in the query.\nFilter supports the following placeholders
\n - ${username}: Will be replaced at runtime with Username used by the client when connecting
\n - ${clientid}: Will be replaced at runtime with Client ID used by the client when connecting", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "rs", - "kind" : "singleton" - }, - "raw_default" : "rs", - "name" : "mongo_type", - "desc" : "Replica set. Must be set to 'rs' when MongoDB server is running in 'replica set' mode.", - "default" : { - "oneliner" : true, - "hocon" : "rs" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe MongoDB default port 27017 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "unsafe", - "safe" - ], - "kind" : "enum" - }, - "raw_default" : "unsafe", - "name" : "w_mode", - "desc" : "Write mode.", - "default" : { - "oneliner" : true, - "hocon" : "unsafe" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "master", - "slave_ok" - ], - "kind" : "enum" - }, - "raw_default" : "master", - "name" : "r_mode", - "desc" : "Read mode.", - "default" : { - "oneliner" : true, - "hocon" : "master" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "replica_set_name", - "desc" : "Name of the replica set.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "srv_record", - "desc" : "Use DNS SRV record.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "auth_source", - "desc" : "Database name associated with the user's credentials.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "topology", - "kind" : "struct" - }, - "name" : "topology", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using a MongoDB replica set." - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:mongo_sharded", - "fields" : [ - { - "type" : { - "name" : "mongodb", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "collection", - "desc" : "`MongoDB` collection containing the authorization data.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "filter", - "desc" : "Conditional expression that defines the filter condition in the query.\nFilter supports the following placeholders
\n - ${username}: Will be replaced at runtime with Username used by the client when connecting
\n - ${clientid}: Will be replaced at runtime with Client ID used by the client when connecting", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "sharded", - "kind" : "singleton" - }, - "raw_default" : "sharded", - "name" : "mongo_type", - "desc" : "Sharded cluster. Must be set to 'sharded' when MongoDB server is running in 'sharded' mode.", - "default" : { - "oneliner" : true, - "hocon" : "sharded" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe MongoDB default port 27017 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "unsafe", - "safe" - ], - "kind" : "enum" - }, - "raw_default" : "unsafe", - "name" : "w_mode", - "desc" : "Write mode.", - "default" : { - "oneliner" : true, - "hocon" : "unsafe" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "srv_record", - "desc" : "Use DNS SRV record.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "auth_source", - "desc" : "Database name associated with the user's credentials.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "topology", - "kind" : "struct" - }, - "name" : "topology", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using a sharded MongoDB cluster." - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:mongo_single", - "fields" : [ - { - "type" : { - "name" : "mongodb", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "collection", - "desc" : "`MongoDB` collection containing the authorization data.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "filter", - "desc" : "Conditional expression that defines the filter condition in the query.\nFilter supports the following placeholders
\n - ${username}: Will be replaced at runtime with Username used by the client when connecting
\n - ${clientid}: Will be replaced at runtime with Client ID used by the client when connecting", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "single", - "kind" : "singleton" - }, - "raw_default" : "single", - "name" : "mongo_type", - "desc" : "Standalone instance. Must be set to 'single' when MongoDB server is running in standalone mode.", - "default" : { - "oneliner" : true, - "hocon" : "single" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe MongoDB default port 27017 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "unsafe", - "safe" - ], - "kind" : "enum" - }, - "raw_default" : "unsafe", - "name" : "w_mode", - "desc" : "Write mode.", - "default" : { - "oneliner" : true, - "hocon" : "unsafe" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "srv_record", - "desc" : "Use DNS SRV record.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "auth_source", - "desc" : "Database name associated with the user's credentials.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "topology", - "kind" : "struct" - }, - "name" : "topology", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using a single MongoDB instance." - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:mysql", - "fields" : [ - { - "type" : { - "name" : "mysql", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe MySQL default port 3306 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "root", - "name" : "username", - "desc" : "EMQX's username in the external database.", - "default" : { - "oneliner" : true, - "hocon" : "\"root\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "name" : "prepare_statement", - "desc" : "Key-value list of SQL prepared statements.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "query", - "desc" : "Database query used to retrieve authorization data.", - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using a MySQL database." - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:postgresql", - "fields" : [ - { - "type" : { - "name" : "postgresql", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe PostgreSQL default port 5432 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "name" : "prepare_statement", - "desc" : "Key-value list of SQL prepared statements.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "query", - "desc" : "Database query used to retrieve authorization data.", - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using a PostgreSQL database." - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:redis_cluster", - "fields" : [ - { - "type" : { - "name" : "redis", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe Redis default port 6379 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "cluster", - "kind" : "singleton" - }, - "raw_default" : "cluster", - "name" : "redis_type", - "desc" : "Cluster mode. Must be set to 'cluster' when Redis server is running in clustered mode.", - "default" : { - "oneliner" : true, - "hocon" : "cluster" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "cmd", - "desc" : "Database query used to retrieve authorization data.", - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using a Redis cluster." - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:redis_sentinel", - "fields" : [ - { - "type" : { - "name" : "redis", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe Redis default port 6379 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "sentinel", - "kind" : "singleton" - }, - "raw_default" : "sentinel", - "name" : "redis_type", - "desc" : "Sentinel mode. Must be set to 'sentinel' when Redis server is running in sentinel mode.", - "default" : { - "oneliner" : true, - "hocon" : "sentinel" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "sentinel", - "desc" : "The cluster name in Redis sentinel mode.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "database", - "desc" : "Redis database ID.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "cmd", - "desc" : "Database query used to retrieve authorization data.", - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using a Redis Sentinel." - }, - { - "tags" : [ - "Authorization" - ], - "paths" : [ - "authorization.sources.$INDEX" - ], - "full_name" : "authz:redis_single", - "fields" : [ - { - "type" : { - "name" : "redis", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this ACL provider", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe Redis default port 6379 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "single", - "kind" : "singleton" - }, - "raw_default" : "single", - "name" : "redis_type", - "desc" : "Single mode. Must be set to 'single' when Redis server is running in single mode.", - "default" : { - "oneliner" : true, - "hocon" : "single" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "database", - "desc" : "Redis database ID.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "cmd", - "desc" : "Database query used to retrieve authorization data.", - "aliases" : [ - - ] - } - ], - "desc" : "Authorization using a single Redis instance." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "alarm" - ], - "full_name" : "broker:alarm", - "fields" : [ - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "atom()", - "kind" : "primitive" - } - }, - "raw_default" : [ - "log", - "publish" - ], - "name" : "actions", - "examples" : [ - [ - "log", - "publish" - ] - ], - "desc" : "The actions triggered when the alarm is activated.
Currently, the following actions are supported: log and publish.\nlog is to write the alarm to log (console or file).\npublish is to publish the alarm as an MQTT message to the system topics:\n$SYS/brokers/emqx@xx.xx.xx.x/alarms/activate and\n$SYS/brokers/emqx@xx.xx.xx.x/alarms/deactivate", - "default" : { - "oneliner" : true, - "hocon" : "[log, publish]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1..3000", - "kind" : "primitive" - }, - "raw_default" : 1000, - "name" : "size_limit", - "examples" : [ - 1000 - ], - "desc" : "The maximum total number of deactivated alarms to keep as history.
When this limit is exceeded, the oldest deactivated alarms are deleted to cap the total number.", - "default" : { - "oneliner" : true, - "hocon" : "1000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "24h", - "name" : "validity_period", - "examples" : [ - [ - 50, - 52, - 104 - ] - ], - "desc" : "Retention time of deactivated alarms. Alarms are not deleted immediately\nwhen deactivated, but after the retention time.", - "default" : { - "oneliner" : true, - "hocon" : "\"24h\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the alarms." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "authorization.cache" - ], - "full_name" : "broker:authz_cache", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable the authorization cache.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1..1048576", - "kind" : "primitive" - }, - "raw_default" : 32, - "name" : "max_size", - "desc" : "Maximum number of cached items.", - "default" : { - "oneliner" : true, - "hocon" : "32" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "1m", - "name" : "ttl", - "desc" : "Time to live for the cached data.", - "default" : { - "oneliner" : true, - "hocon" : "\"1m\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the authorization cache." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "broker" - ], - "full_name" : "broker", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable_session_registry", - "desc" : "Enable session registry", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "local", - "leader", - "quorum", - "all" - ], - "kind" : "enum" - }, - "raw_default" : "quorum", - "name" : "session_locking_strategy", - "desc" : "Session locking strategy in a cluster.\n - `local`: only lock the session on the current node\n - `one`: select only one remote node to lock the session\n - `quorum`: select some nodes to lock the session\n - `all`: lock the session on all the nodes in the cluster", - "default" : { - "oneliner" : true, - "hocon" : "quorum" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "random", - "round_robin", - "round_robin_per_group", - "sticky", - "local", - "hash_topic", - "hash_clientid" - ], - "kind" : "enum" - }, - "raw_default" : "round_robin", - "name" : "shared_subscription_strategy", - "desc" : "Dispatch strategy for shared subscription.\n - `random`: dispatch the message to a random selected subscriber\n - `round_robin`: select the subscribers in a round-robin manner\n - `round_robin_per_group`: select the subscribers in round-robin fashion within each shared subscriber group\n - `local`: select random local subscriber otherwise select random cluster-wide\n - `sticky`: always use the last selected subscriber to dispatch, until the subscriber disconnects.\n - `hash_clientid`: select the subscribers by hashing the `clientIds`\n - `hash_topic`: select the subscribers by hashing the source topic", - "default" : { - "oneliner" : true, - "hocon" : "round_robin" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "shared_dispatch_ack_enabled", - "desc" : "Deprecated, will be removed in 5.1.\nEnable/disable shared dispatch acknowledgement for QoS 1 and QoS 2 messages.\nThis should allow messages to be dispatched to a different subscriber in the group in case the picked (based on `shared_subscription_strategy`) subscriber is offline.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "route_batch_clean", - "desc" : "Enable batch clean for deleted routes.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Message broker options." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.ws.$name.websocket.deflate_opts", - "listeners.wss.$name.websocket.deflate_opts" - ], - "full_name" : "broker:deflate_opts", - "fields" : [ - { - "type" : { - "symbols" : [ - "none", - "default", - "best_compression", - "best_speed" - ], - "kind" : "enum" - }, - "name" : "level", - "desc" : "Compression level.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1..9", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "mem_level", - "desc" : "Specifies the size of the compression state.
\nLower values decrease memory usage per connection.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "default", - "filtered", - "huffman_only", - "rle" - ], - "kind" : "enum" - }, - "raw_default" : "default", - "name" : "strategy", - "desc" : "Specifies the compression strategy.", - "default" : { - "oneliner" : true, - "hocon" : "default" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "takeover", - "no_takeover" - ], - "kind" : "enum" - }, - "raw_default" : "takeover", - "name" : "server_context_takeover", - "desc" : "Takeover means the compression state is retained between server messages.", - "default" : { - "oneliner" : true, - "hocon" : "takeover" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "takeover", - "no_takeover" - ], - "kind" : "enum" - }, - "raw_default" : "takeover", - "name" : "client_context_takeover", - "desc" : "Takeover means the compression state is retained between client messages.", - "default" : { - "oneliner" : true, - "hocon" : "takeover" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "8..15", - "kind" : "primitive" - }, - "raw_default" : 15, - "name" : "server_max_window_bits", - "desc" : "Specifies the size of the compression context for the server.", - "default" : { - "oneliner" : true, - "hocon" : "15" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "8..15", - "kind" : "primitive" - }, - "raw_default" : 15, - "name" : "client_max_window_bits", - "desc" : "Specifies the size of the compression context for the client.", - "default" : { - "oneliner" : true, - "hocon" : "15" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Compression options." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "sys_topics.sys_event_messages" - ], - "full_name" : "broker:event_names", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "client_connected", - "desc" : "Enable to publish client connected event messages", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "client_disconnected", - "desc" : "Enable to publish client disconnected event messages.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "client_subscribed", - "desc" : "Enable to publish event message that client subscribed a topic successfully.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "client_unsubscribed", - "desc" : "Enable to publish event message that client unsubscribed a topic successfully.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Enable or disable client lifecycle event publishing.\n\nThe following options affect MQTT clients as well as\ngateway clients. The types of the clients\nare distinguished by the topic prefix:\n\n- For the MQTT clients, the format is:\n`$SYS/broker//clients//`\n- For the Gateway clients, it is\n`$SYS/broker//gateway//clients//`\n" - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "force_gc" - ], - "full_name" : "broker:force_gc", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable forced garbage collection.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "0..inf", - "kind" : "primitive" - }, - "raw_default" : 16000, - "name" : "count", - "desc" : "GC the process after this many received messages.", - "default" : { - "oneliner" : true, - "hocon" : "16000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "16MB", - "name" : "bytes", - "desc" : "GC the process after specified number of bytes have passed through.", - "default" : { - "oneliner" : true, - "hocon" : "\"16MB\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Force garbage collection in MQTT connection process after\n they process certain number of messages or bytes of data." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "force_shutdown" - ], - "full_name" : "broker:force_shutdown", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable `force_shutdown` feature.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "0..inf", - "kind" : "primitive" - }, - "raw_default" : 1000, - "name" : "max_message_queue_len", - "desc" : "Maximum message queue length.", - "default" : { - "oneliner" : true, - "hocon" : "1000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:wordsize()", - "kind" : "primitive" - }, - "raw_default" : "32MB", - "name" : "max_heap_size", - "desc" : "Total heap size", - "default" : { - "oneliner" : true, - "hocon" : "\"32MB\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "When the process message queue length, or the memory bytes\nreaches a certain value, the process is forced to close.\n\nNote: \"message queue\" here refers to the \"message mailbox\"\nof the Erlang process, not the `mqueue` of QoS 1 and QoS 2." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.quic.$name.ssl_options" - ], - "full_name" : "broker:listener_quic_ssl_opts", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "cacertfile", - "desc" : "Trusted PEM format CA certificates bundle file.
\nThe certificates in this file are used to verify the TLS peer's certificates.\nAppend new certificates to the file if new CAs are to be trusted.\nThere is no need to restart EMQX to have the updated file loaded, because\nthe system regularly checks if file has been updated (and reload).
\nNOTE: invalidating (deleting) a certificate from the file will not affect\nalready established connections.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "certfile", - "desc" : "PEM format certificates chain file.
\nThe certificates in this file should be in reversed order of the certificate\nissue chain. That is, the host's certificate should be placed in the beginning\nof the file, followed by the immediate issuer certificate and so on.\nAlthough the root CA certificate is optional, it should be placed at the end of\nthe file if it is to be added.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "keyfile", - "desc" : "PEM format private key file.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "verify_peer", - "verify_none" - ], - "kind" : "enum" - }, - "raw_default" : "verify_none", - "name" : "verify", - "desc" : "Enable or disable peer verification.", - "default" : { - "oneliner" : true, - "hocon" : "verify_none" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "reuse_sessions", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "name" : "depth", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "password", - "examples" : [ - "" - ], - "desc" : "String containing the user's password. Only used if the private key file is password-protected.", - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "atom()", - "kind" : "primitive" - } - }, - "name" : "versions", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "string()", - "kind" : "primitive" - } - }, - "name" : "ciphers", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "secure_renegotiate", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "name" : "hibernate_after", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "dhfile", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "fail_if_no_peer_cert", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "honor_cipher_order", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "client_renegotiation", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "name" : "handshake_timeout", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "gc_after_handshake", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ocsp", - "kind" : "struct" - }, - "name" : "ocsp", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_crl_check", - "importance" : "medium", - "desc" : "Deprecated since 5.0.20.", - "aliases" : [ - - ] - } - ], - "desc" : "TLS options for QUIC transport." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.ssl.$name.ssl_options" - ], - "full_name" : "broker:listener_ssl_opts", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "cacertfile", - "desc" : "Trusted PEM format CA certificates bundle file.
\nThe certificates in this file are used to verify the TLS peer's certificates.\nAppend new certificates to the file if new CAs are to be trusted.\nThere is no need to restart EMQX to have the updated file loaded, because\nthe system regularly checks if file has been updated (and reload).
\nNOTE: invalidating (deleting) a certificate from the file will not affect\nalready established connections.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "certfile", - "desc" : "PEM format certificates chain file.
\nThe certificates in this file should be in reversed order of the certificate\nissue chain. That is, the host's certificate should be placed in the beginning\nof the file, followed by the immediate issuer certificate and so on.\nAlthough the root CA certificate is optional, it should be placed at the end of\nthe file if it is to be added.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "keyfile", - "desc" : "PEM format private key file.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "verify_peer", - "verify_none" - ], - "kind" : "enum" - }, - "raw_default" : "verify_none", - "name" : "verify", - "desc" : "Enable or disable peer verification.", - "default" : { - "oneliner" : true, - "hocon" : "verify_none" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "reuse_sessions", - "desc" : "Enable TLS session reuse.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 10, - "name" : "depth", - "desc" : "Maximum number of non-self-issued intermediate certificates that can follow the peer certificate in a valid certification path.\nSo, if depth is 0 the PEER must be signed by the trusted ROOT-CA directly;
\nif 1 the path can be PEER, Intermediate-CA, ROOT-CA;
\nif 2 the path can be PEER, Intermediate-CA1, Intermediate-CA2, ROOT-CA.", - "default" : { - "oneliner" : true, - "hocon" : "10" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "password", - "examples" : [ - "" - ], - "desc" : "String containing the user's password. Only used if the private key file is password-protected.", - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "atom()", - "kind" : "primitive" - } - }, - "raw_default" : [ - "tlsv1.3", - "tlsv1.2", - "tlsv1.1", - "tlsv1" - ], - "name" : "versions", - "desc" : "All TLS/DTLS versions to be supported.
\nNOTE: PSK ciphers are suppressed by 'tlsv1.3' version config.
\nIn case PSK cipher suites are intended, make sure to configure\n['tlsv1.2', 'tlsv1.1'] here.", - "default" : { - "oneliner" : true, - "hocon" : "[tlsv1.3, tlsv1.2, tlsv1.1, tlsv1]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "string()", - "kind" : "primitive" - } - }, - "raw_default" : [ - - ], - "name" : "ciphers", - "desc" : "This config holds TLS cipher suite names separated by comma,\nor as an array of strings. e.g.\n\"TLS_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256\" or\n[\"TLS_AES_256_GCM_SHA384\",\"TLS_AES_128_GCM_SHA256\"].\n
\nCiphers (and their ordering) define the way in which the\nclient and server encrypts information over the network connection.\nSelecting a good cipher suite is critical for the\napplication's data security, confidentiality and performance.\n\nThe names should be in OpenSSL string format (not RFC format).\nAll default values and examples provided by EMQX config\ndocumentation are all in OpenSSL format.
\n\nNOTE: Certain cipher suites are only compatible with\nspecific TLS versions ('tlsv1.1', 'tlsv1.2' or 'tlsv1.3')\nincompatible cipher suites will be silently dropped.\nFor instance, if only 'tlsv1.3' is given in the versions,\nconfiguring cipher suites for other versions will have no effect.\n
\n\nNOTE: PSK ciphers are suppressed by 'tlsv1.3' version config
\nIf PSK cipher suites are intended, 'tlsv1.3' should be disabled from versions.
\nPSK cipher suites: \"RSA-PSK-AES256-GCM-SHA384,RSA-PSK-AES256-CBC-SHA384,\nRSA-PSK-AES128-GCM-SHA256,RSA-PSK-AES128-CBC-SHA256,\nRSA-PSK-AES256-CBC-SHA,RSA-PSK-AES128-CBC-SHA,\nRSA-PSK-DES-CBC3-SHA,RSA-PSK-RC4-SHA\"", - "default" : { - "oneliner" : true, - "hocon" : "[]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "secure_renegotiate", - "desc" : "SSL parameter renegotiation is a feature that allows a client and a server\nto renegotiate the parameters of the SSL connection on the fly.\nRFC 5746 defines a more secure way of doing this. By enabling secure renegotiation,\nyou drop support for the insecure renegotiation, prone to MitM attacks.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "hibernate_after", - "desc" : "Hibernate the SSL process after idling for amount of time reducing its memory footprint.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "dhfile", - "desc" : "Path to a file containing PEM-encoded Diffie-Hellman parameters\nto be used by the server if a cipher suite using Diffie-Hellman\nkey exchange is negotiated. If not specified, default parameters\nare used.
\nNOTE: The dhfile option is not supported by TLS 1.3.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "fail_if_no_peer_cert", - "desc" : "Used together with {verify, verify_peer} by an TLS/DTLS server.\nIf set to true, the server fails if the client does not have a\ncertificate to send, that is, sends an empty certificate.\nIf set to false, it fails only if the client sends an invalid\ncertificate (an empty certificate is considered valid).", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "honor_cipher_order", - "desc" : "An important security setting, it forces the cipher to be set based\n on the server-specified order instead of the client-specified order,\n hence enforcing the (usually more properly configured) security\n ordering of the server administrator.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "client_renegotiation", - "desc" : "In protocols that support client-initiated renegotiation,\nthe cost of resources of such an operation is higher for the server than the client.\nThis can act as a vector for denial of service attacks.\nThe SSL application already takes measures to counter-act such attempts,\nbut client-initiated renegotiation can be strictly disabled by setting this option to false.\nThe default value is true. Note that disabling renegotiation can result in\nlong-lived connections becoming unusable due to limits on\nthe number of messages the underlying cipher suite can encipher.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "handshake_timeout", - "desc" : "Maximum time duration allowed for the handshake to complete", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "gc_after_handshake", - "desc" : "Memory usage tuning. If enabled, will immediately perform a garbage collection after the TLS/SSL handshake.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ocsp", - "kind" : "struct" - }, - "name" : "ocsp", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "enable_crl_check", - "importance" : "medium", - "desc" : "Whether to enable CRL verification for this listener.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Socket options for SSL connections." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.wss.$name.ssl_options" - ], - "full_name" : "broker:listener_wss_opts", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "cacertfile", - "desc" : "Trusted PEM format CA certificates bundle file.
\nThe certificates in this file are used to verify the TLS peer's certificates.\nAppend new certificates to the file if new CAs are to be trusted.\nThere is no need to restart EMQX to have the updated file loaded, because\nthe system regularly checks if file has been updated (and reload).
\nNOTE: invalidating (deleting) a certificate from the file will not affect\nalready established connections.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "certfile", - "desc" : "PEM format certificates chain file.
\nThe certificates in this file should be in reversed order of the certificate\nissue chain. That is, the host's certificate should be placed in the beginning\nof the file, followed by the immediate issuer certificate and so on.\nAlthough the root CA certificate is optional, it should be placed at the end of\nthe file if it is to be added.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "keyfile", - "desc" : "PEM format private key file.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "verify_peer", - "verify_none" - ], - "kind" : "enum" - }, - "raw_default" : "verify_none", - "name" : "verify", - "desc" : "Enable or disable peer verification.", - "default" : { - "oneliner" : true, - "hocon" : "verify_none" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "reuse_sessions", - "desc" : "Enable TLS session reuse.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 10, - "name" : "depth", - "desc" : "Maximum number of non-self-issued intermediate certificates that can follow the peer certificate in a valid certification path.\nSo, if depth is 0 the PEER must be signed by the trusted ROOT-CA directly;
\nif 1 the path can be PEER, Intermediate-CA, ROOT-CA;
\nif 2 the path can be PEER, Intermediate-CA1, Intermediate-CA2, ROOT-CA.", - "default" : { - "oneliner" : true, - "hocon" : "10" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "password", - "examples" : [ - "" - ], - "desc" : "String containing the user's password. Only used if the private key file is password-protected.", - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "atom()", - "kind" : "primitive" - } - }, - "raw_default" : [ - "tlsv1.3", - "tlsv1.2", - "tlsv1.1", - "tlsv1" - ], - "name" : "versions", - "desc" : "All TLS/DTLS versions to be supported.
\nNOTE: PSK ciphers are suppressed by 'tlsv1.3' version config.
\nIn case PSK cipher suites are intended, make sure to configure\n['tlsv1.2', 'tlsv1.1'] here.", - "default" : { - "oneliner" : true, - "hocon" : "[tlsv1.3, tlsv1.2, tlsv1.1, tlsv1]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "string()", - "kind" : "primitive" - } - }, - "raw_default" : [ - - ], - "name" : "ciphers", - "desc" : "This config holds TLS cipher suite names separated by comma,\nor as an array of strings. e.g.\n\"TLS_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256\" or\n[\"TLS_AES_256_GCM_SHA384\",\"TLS_AES_128_GCM_SHA256\"].\n
\nCiphers (and their ordering) define the way in which the\nclient and server encrypts information over the network connection.\nSelecting a good cipher suite is critical for the\napplication's data security, confidentiality and performance.\n\nThe names should be in OpenSSL string format (not RFC format).\nAll default values and examples provided by EMQX config\ndocumentation are all in OpenSSL format.
\n\nNOTE: Certain cipher suites are only compatible with\nspecific TLS versions ('tlsv1.1', 'tlsv1.2' or 'tlsv1.3')\nincompatible cipher suites will be silently dropped.\nFor instance, if only 'tlsv1.3' is given in the versions,\nconfiguring cipher suites for other versions will have no effect.\n
\n\nNOTE: PSK ciphers are suppressed by 'tlsv1.3' version config
\nIf PSK cipher suites are intended, 'tlsv1.3' should be disabled from versions.
\nPSK cipher suites: \"RSA-PSK-AES256-GCM-SHA384,RSA-PSK-AES256-CBC-SHA384,\nRSA-PSK-AES128-GCM-SHA256,RSA-PSK-AES128-CBC-SHA256,\nRSA-PSK-AES256-CBC-SHA,RSA-PSK-AES128-CBC-SHA,\nRSA-PSK-DES-CBC3-SHA,RSA-PSK-RC4-SHA\"", - "default" : { - "oneliner" : true, - "hocon" : "[]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "secure_renegotiate", - "desc" : "SSL parameter renegotiation is a feature that allows a client and a server\nto renegotiate the parameters of the SSL connection on the fly.\nRFC 5746 defines a more secure way of doing this. By enabling secure renegotiation,\nyou drop support for the insecure renegotiation, prone to MitM attacks.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "hibernate_after", - "desc" : "Hibernate the SSL process after idling for amount of time reducing its memory footprint.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "dhfile", - "desc" : "Path to a file containing PEM-encoded Diffie-Hellman parameters\nto be used by the server if a cipher suite using Diffie-Hellman\nkey exchange is negotiated. If not specified, default parameters\nare used.
\nNOTE: The dhfile option is not supported by TLS 1.3.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "fail_if_no_peer_cert", - "desc" : "Used together with {verify, verify_peer} by an TLS/DTLS server.\nIf set to true, the server fails if the client does not have a\ncertificate to send, that is, sends an empty certificate.\nIf set to false, it fails only if the client sends an invalid\ncertificate (an empty certificate is considered valid).", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "honor_cipher_order", - "desc" : "An important security setting, it forces the cipher to be set based\n on the server-specified order instead of the client-specified order,\n hence enforcing the (usually more properly configured) security\n ordering of the server administrator.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "client_renegotiation", - "desc" : "In protocols that support client-initiated renegotiation,\nthe cost of resources of such an operation is higher for the server than the client.\nThis can act as a vector for denial of service attacks.\nThe SSL application already takes measures to counter-act such attempts,\nbut client-initiated renegotiation can be strictly disabled by setting this option to false.\nThe default value is true. Note that disabling renegotiation can result in\nlong-lived connections becoming unusable due to limits on\nthe number of messages the underlying cipher suite can encipher.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "handshake_timeout", - "desc" : "Maximum time duration allowed for the handshake to complete", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Socket options for WebSocket/SSL connections." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners" - ], - "full_name" : "broker:listeners", - "fields" : [ - { - "type" : { - "values" : { - "members" : [ - { - "name" : "broker:mqtt_tcp_listener", - "kind" : "struct" - }, - { - "name" : "marked_for_deletion", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "tcp", - "desc" : "TCP listeners.", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "members" : [ - { - "name" : "broker:mqtt_ssl_listener", - "kind" : "struct" - }, - { - "name" : "marked_for_deletion", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "ssl", - "desc" : "SSL listeners.", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "members" : [ - { - "name" : "broker:mqtt_ws_listener", - "kind" : "struct" - }, - { - "name" : "marked_for_deletion", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "ws", - "desc" : "HTTP websocket listeners.", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "members" : [ - { - "name" : "broker:mqtt_wss_listener", - "kind" : "struct" - }, - { - "name" : "marked_for_deletion", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "wss", - "desc" : "HTTPS websocket listeners.", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "members" : [ - { - "name" : "broker:mqtt_quic_listener", - "kind" : "struct" - }, - { - "name" : "marked_for_deletion", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "quic", - "desc" : "QUIC listeners.", - "aliases" : [ - - ] - } - ], - "desc" : "MQTT listeners identified by their protocol type and assigned names" - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "mqtt" - ], - "full_name" : "broker:mqtt", - "fields" : [ - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "15s", - "name" : "idle_timeout", - "desc" : "Configure the duration of time that a connection can remain idle (i.e., without any data transfer) before being:\n - Automatically disconnected if no CONNECT package is received from the client yet.\n - Put into hibernation mode to save resources if some CONNECT packages are already received.\nNote: Please set the parameter with caution as long idle time will lead to resource waste.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "1MB", - "name" : "max_packet_size", - "desc" : "Maximum MQTT packet size allowed.", - "default" : { - "oneliner" : true, - "hocon" : "\"1MB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "23..65535", - "kind" : "primitive" - }, - "raw_default" : 65535, - "name" : "max_clientid_len", - "desc" : "Maximum allowed length of MQTT Client ID.", - "default" : { - "oneliner" : true, - "hocon" : "65535" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1..65535", - "kind" : "primitive" - }, - "raw_default" : 128, - "name" : "max_topic_levels", - "desc" : "Maximum topic levels allowed.", - "default" : { - "oneliner" : true, - "hocon" : "128" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "qos()", - "kind" : "primitive" - }, - "raw_default" : 2, - "name" : "max_qos_allowed", - "desc" : "Maximum QoS allowed.", - "default" : { - "oneliner" : true, - "hocon" : "2" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "0..65535", - "kind" : "primitive" - }, - "raw_default" : 65535, - "name" : "max_topic_alias", - "desc" : "Maximum topic alias, 0 means no topic alias supported.", - "default" : { - "oneliner" : true, - "hocon" : "65535" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "retain_available", - "desc" : "Whether to enable support for MQTT retained message.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "wildcard_subscription", - "desc" : "Whether to enable support for MQTT wildcard subscription.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "shared_subscription", - "desc" : "Whether to enable support for MQTT shared subscription.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "exclusive_subscription", - "desc" : "Whether to enable support for MQTT exclusive subscription.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "ignore_loop_deliver", - "desc" : "Whether the messages sent by the MQTT v3.1.1/v3.1.0 client will be looped back to the publisher itself, similar to No Local in MQTT 5.0.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "strict_mode", - "desc" : "Whether to parse MQTT messages in strict mode.\nIn strict mode, invalid utf8 strings in for example client ID, topic name, etc. will cause the client to be disconnected.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "response_information", - "desc" : "UTF-8 string, for creating the response topic, for example, if set to reqrsp/, the publisher/subscriber will communicate using the topic prefix reqrsp/.\nTo disable this feature, input \"\" in the text box below. Only applicable to MQTT 5.0 clients.", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "integer()", - "kind" : "primitive" - }, - { - "name" : "disabled", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "raw_default" : "disabled", - "name" : "server_keepalive", - "desc" : "The keep alive duration required by EMQX. To use the setting from the client side, choose disabled from the drop-down list. Only applicable to MQTT 5.0 clients.", - "default" : { - "oneliner" : true, - "hocon" : "disabled" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "number()", - "kind" : "primitive" - }, - "raw_default" : 0.75, - "name" : "keepalive_backoff", - "desc" : "The coefficient EMQX uses to confirm whether the keep alive duration of the client expires. Formula: Keep Alive * Backoff * 2", - "default" : { - "oneliner" : true, - "hocon" : "0.75" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "1..inf", - "kind" : "primitive" - }, - { - "name" : "infinity", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "raw_default" : "infinity", - "name" : "max_subscriptions", - "desc" : "Maximum number of subscriptions allowed per client.", - "default" : { - "oneliner" : true, - "hocon" : "infinity" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "upgrade_qos", - "desc" : "Force upgrade of QoS level according to subscription.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1..65535", - "kind" : "primitive" - }, - "raw_default" : 32, - "name" : "max_inflight", - "desc" : "Maximum number of QoS 1 and QoS 2 messages that are allowed to be delivered simultaneously before completing the acknowledgment.", - "default" : { - "oneliner" : true, - "hocon" : "32" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "30s", - "name" : "retry_interval", - "desc" : "Retry interval for QoS 1/2 message delivering.", - "default" : { - "oneliner" : true, - "hocon" : "\"30s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "integer()", - "kind" : "primitive" - }, - { - "name" : "infinity", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "raw_default" : 100, - "name" : "max_awaiting_rel", - "desc" : "For each publisher session, the maximum number of outstanding QoS 2 messages pending on the client to send PUBREL. After reaching this limit, new QoS 2 PUBLISH requests will be rejected with `147(0x93)` until either PUBREL is received or timed out.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "300s", - "name" : "await_rel_timeout", - "desc" : "For client to broker QoS 2 message, the time limit for the broker to wait before the `PUBREL` message is received. The wait is aborted after timed out, meaning the packet ID is freed for new `PUBLISH` requests. Receiving a stale `PUBREL` causes a warning level log. Note, the message is delivered to subscribers before entering the wait for PUBREL.", - "default" : { - "oneliner" : true, - "hocon" : "\"300s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "2h", - "name" : "session_expiry_interval", - "desc" : "Specifies how long the session will expire after the connection is disconnected, only for non-MQTT 5.0 connections.", - "default" : { - "oneliner" : true, - "hocon" : "\"2h\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - { - "name" : "infinity", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "raw_default" : 1000, - "name" : "max_mqueue_len", - "desc" : "Maximum queue length. Enqueued messages when persistent client disconnected, or inflight window is full.", - "default" : { - "oneliner" : true, - "hocon" : "1000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "map()", - "kind" : "primitive" - }, - { - "name" : "disabled", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "raw_default" : "disabled", - "name" : "mqueue_priorities", - "desc" : "Topic priorities. Priority number [1-255]\nThere's no priority table by default, hence all messages are treated equal.\n\n**NOTE**: Comma and equal signs are not allowed for priority topic names.\n**NOTE**: Messages for topics not in the priority table are treated as either highest or lowest priority depending on the configured value for mqtt.mqueue_default_priority.\n\n**Examples**:\nTo configure \"topic/1\" > \"topic/2\":\nmqueue_priorities: {\"topic/1\": 10, \"topic/2\": 8}", - "default" : { - "oneliner" : true, - "hocon" : "disabled" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "highest", - "lowest" - ], - "kind" : "enum" - }, - "raw_default" : "lowest", - "name" : "mqueue_default_priority", - "desc" : "Default topic priority, which will be used by topics not in Topic Priorities (mqueue_priorities).", - "default" : { - "oneliner" : true, - "hocon" : "lowest" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "mqueue_store_qos0", - "desc" : "Specifies whether to store QoS 0 messages in the message queue while the connection is down but the session remains.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "use_username_as_clientid", - "desc" : "Whether to use Username as Client ID.\nThis setting takes effect later than Use Peer Certificate as Username and Use peer certificate as Client ID.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "disabled", - "cn", - "dn", - "crt", - "pem", - "md5" - ], - "kind" : "enum" - }, - "raw_default" : "disabled", - "name" : "peer_cert_as_username", - "desc" : "Use the CN, DN field in the peer certificate or the entire certificate content as Username. Only works for the TLS connection.\nSupported configurations are the following:\n- cn: CN field of the certificate\n- dn: DN field of the certificate\n- crt: Content of the DER or PEM certificate\n- pem: Convert DER certificate content to PEM format and use as Username\n- md5: MD5 value of the DER or PEM certificate", - "default" : { - "oneliner" : true, - "hocon" : "disabled" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "disabled", - "cn", - "dn", - "crt", - "pem", - "md5" - ], - "kind" : "enum" - }, - "raw_default" : "disabled", - "name" : "peer_cert_as_clientid", - "desc" : "Use the CN, DN field in the peer certificate or the entire certificate content as Client ID. Only works for the TLS connection.\nSupported configurations are the following:\n- cn: CN field of the certificate\n- dn: DN field of the certificate\n- crt: DER or PEM certificate\n- pem: Convert DER certificate content to PEM format and use as Client ID\n- md5: MD5 value of the DER or PEM certificate", - "default" : { - "oneliner" : true, - "hocon" : "disabled" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Global MQTT configuration.
The configs here work as default values which can be overridden\nin zone configs" - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.quic.$name" - ], - "full_name" : "broker:mqtt_quic_listener", - "fields" : [ - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "certfile", - "desc" : "Path to the certificate file. Will be deprecated in 5.1, use .ssl_options.certfile instead.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "keyfile", - "desc" : "Path to the secret key file. Will be deprecated in 5.1, use .ssl_options.keyfile instead.", - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "string()", - "kind" : "primitive" - } - }, - "raw_default" : [ - "TLS_AES_256_GCM_SHA384", - "TLS_AES_128_GCM_SHA256", - "TLS_CHACHA20_POLY1305_SHA256" - ], - "name" : "ciphers", - "desc" : "This config holds TLS cipher suite names separated by comma,\nor as an array of strings. e.g.\n\"TLS_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256\" or\n[\"TLS_AES_256_GCM_SHA384\",\"TLS_AES_128_GCM_SHA256\"].\n
\nCiphers (and their ordering) define the way in which the\nclient and server encrypts information over the network connection.\nSelecting a good cipher suite is critical for the\napplication's data security, confidentiality and performance.\n\nThe names should be in OpenSSL string format (not RFC format).\nAll default values and examples provided by EMQX config\ndocumentation are all in OpenSSL format.
\n\nNOTE: Certain cipher suites are only compatible with\nspecific TLS versions ('tlsv1.1', 'tlsv1.2' or 'tlsv1.3')\nincompatible cipher suites will be silently dropped.\nFor instance, if only 'tlsv1.3' is given in the versions,\nconfiguring cipher suites for other versions will have no effect.\n
\n\nNOTE: PSK ciphers are suppressed by 'tlsv1.3' version config
\nIf PSK cipher suites are intended, 'tlsv1.3' should be disabled from versions.
\nPSK cipher suites: \"RSA-PSK-AES256-GCM-SHA384,RSA-PSK-AES256-CBC-SHA384,\nRSA-PSK-AES128-GCM-SHA256,RSA-PSK-AES128-CBC-SHA256,\nRSA-PSK-AES256-CBC-SHA,RSA-PSK-AES128-CBC-SHA,\nRSA-PSK-DES-CBC3-SHA,RSA-PSK-RC4-SHA\"
\n\nNOTE: QUIC listener supports only 'tlsv1.3' ciphers", - "default" : { - "oneliner" : true, - "hocon" : "[\"TLS_AES_256_GCM_SHA384\", \"TLS_AES_128_GCM_SHA256\", \"TLS_CHACHA20_POLY1305_SHA256\"]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "idle_timeout", - "desc" : "How long a connection can go idle before it is gracefully shut down. 0 to disable", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "10s", - "name" : "handshake_idle_timeout", - "desc" : "How long a handshake can idle before it is discarded.", - "default" : { - "oneliner" : true, - "hocon" : "\"10s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "keep_alive_interval", - "desc" : "How often to send PING frames to keep a connection alive. 0 means disabled.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:listener_quic_ssl_opts", - "kind" : "struct" - }, - "name" : "ssl_options", - "desc" : "TLS options for QUIC transport", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enabled", - "desc" : "Enable listener.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "emqx_schema:ip_port()", - "kind" : "primitive" - }, - { - "name" : "integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 14567, - "name" : "bind", - "desc" : "IP address and port for the listening socket.", - "default" : { - "oneliner" : true, - "hocon" : "14567" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "acceptors", - "desc" : "The size of the listener's receiving pool.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "pos_integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 5000000, - "name" : "max_connections", - "desc" : "The maximum number of concurrent connections allowed by the listener.", - "default" : { - "oneliner" : true, - "hocon" : "5000000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "mountpoint", - "desc" : "When publishing or subscribing, prefix all topics with a mountpoint string.\nThe prefixed string will be removed from the topic name when the message\nis delivered to the subscriber. The mountpoint is a way that users can use\nto implement isolation of message routing between different listeners.\nFor example if a client A subscribes to `t` with `listeners.tcp.\\.mountpoint`\nset to `some_tenant`, then the client actually subscribes to the topic\n`some_tenant/t`. Similarly, if another client B (connected to the same listener\nas the client A) sends a message to topic `t`, the message is routed\nto all the clients subscribed `some_tenant/t`, so client A will receive the\nmessage, with topic name `t`.
\nSet to `\"\"` to disable the feature.
\n\nVariables in mountpoint string:\n - ${clientid}: clientid\n - ${username}: username", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "atom()", - "kind" : "primitive" - }, - "raw_default" : "default", - "name" : "zone", - "desc" : "The configuration zone to which the listener belongs.", - "default" : { - "oneliner" : true, - "hocon" : "default" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:listener_fields", - "kind" : "struct" - }, - "name" : "limiter", - "desc" : "Type of the rate limit.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "true", - "false", - "quick_deny_anonymous" - ], - "kind" : "enum" - }, - "raw_default" : true, - "name" : "enable_authn", - "desc" : "Set true (default) to enable client authentication on this listener, the authentication\nprocess goes through the configured authentication chain.\nWhen set to false to allow any clients with or without authentication information such as username or password to log in.\nWhen set to quick_deny_anonymous, it behaves like when set to true, but clients will be\ndenied immediately without going through any authenticators if username is not provided. This is useful to fence off\nanonymous clients early.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the MQTT over QUIC listener." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.ssl.$name" - ], - "full_name" : "broker:mqtt_ssl_listener", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enabled", - "desc" : "Enable listener.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "emqx_schema:ip_port()", - "kind" : "primitive" - }, - { - "name" : "integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 8883, - "name" : "bind", - "desc" : "IP address and port for the listening socket.", - "default" : { - "oneliner" : true, - "hocon" : "8883" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "acceptors", - "desc" : "The size of the listener's receiving pool.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "pos_integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 5000000, - "name" : "max_connections", - "desc" : "The maximum number of concurrent connections allowed by the listener.", - "default" : { - "oneliner" : true, - "hocon" : "5000000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "mountpoint", - "desc" : "When publishing or subscribing, prefix all topics with a mountpoint string.\nThe prefixed string will be removed from the topic name when the message\nis delivered to the subscriber. The mountpoint is a way that users can use\nto implement isolation of message routing between different listeners.\nFor example if a client A subscribes to `t` with `listeners.tcp.\\.mountpoint`\nset to `some_tenant`, then the client actually subscribes to the topic\n`some_tenant/t`. Similarly, if another client B (connected to the same listener\nas the client A) sends a message to topic `t`, the message is routed\nto all the clients subscribed `some_tenant/t`, so client A will receive the\nmessage, with topic name `t`.
\nSet to `\"\"` to disable the feature.
\n\nVariables in mountpoint string:\n - ${clientid}: clientid\n - ${username}: username", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "atom()", - "kind" : "primitive" - }, - "raw_default" : "default", - "name" : "zone", - "desc" : "The configuration zone to which the listener belongs.", - "default" : { - "oneliner" : true, - "hocon" : "default" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:listener_fields", - "kind" : "struct" - }, - "name" : "limiter", - "desc" : "Type of the rate limit.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "true", - "false", - "quick_deny_anonymous" - ], - "kind" : "enum" - }, - "raw_default" : true, - "name" : "enable_authn", - "desc" : "Set true (default) to enable client authentication on this listener, the authentication\nprocess goes through the configured authentication chain.\nWhen set to false to allow any clients with or without authentication information such as username or password to log in.\nWhen set to quick_deny_anonymous, it behaves like when set to true, but clients will be\ndenied immediately without going through any authenticators if username is not provided. This is useful to fence off\nanonymous clients early.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "string()", - "kind" : "primitive" - } - }, - "raw_default" : [ - "allow all" - ], - "name" : "access_rules", - "desc" : "The access control rules for this listener.
See: https://github.com/emqtt/esockd#allowdeny", - "default" : { - "oneliner" : true, - "hocon" : "[\"allow all\"]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "proxy_protocol", - "desc" : "Enable the Proxy Protocol V1/2 if the EMQX cluster is deployed behind HAProxy or Nginx.
\nSee: https://www.haproxy.com/blog/haproxy/proxy-protocol/", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "3s", - "name" : "proxy_protocol_timeout", - "desc" : "Timeout for proxy protocol. EMQX will close the TCP connection if proxy protocol packet is not received within the timeout.", - "default" : { - "oneliner" : true, - "hocon" : "\"3s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:tcp_opts", - "kind" : "struct" - }, - "name" : "tcp_options", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:listener_ssl_opts", - "kind" : "struct" - }, - "name" : "ssl_options", - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the MQTT over SSL listener." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.tcp.$name" - ], - "full_name" : "broker:mqtt_tcp_listener", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enabled", - "desc" : "Enable listener.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "emqx_schema:ip_port()", - "kind" : "primitive" - }, - { - "name" : "integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 1883, - "name" : "bind", - "desc" : "IP address and port for the listening socket.", - "default" : { - "oneliner" : true, - "hocon" : "1883" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "acceptors", - "desc" : "The size of the listener's receiving pool.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "pos_integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 5000000, - "name" : "max_connections", - "desc" : "The maximum number of concurrent connections allowed by the listener.", - "default" : { - "oneliner" : true, - "hocon" : "5000000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "mountpoint", - "desc" : "When publishing or subscribing, prefix all topics with a mountpoint string.\nThe prefixed string will be removed from the topic name when the message\nis delivered to the subscriber. The mountpoint is a way that users can use\nto implement isolation of message routing between different listeners.\nFor example if a client A subscribes to `t` with `listeners.tcp.\\.mountpoint`\nset to `some_tenant`, then the client actually subscribes to the topic\n`some_tenant/t`. Similarly, if another client B (connected to the same listener\nas the client A) sends a message to topic `t`, the message is routed\nto all the clients subscribed `some_tenant/t`, so client A will receive the\nmessage, with topic name `t`.
\nSet to `\"\"` to disable the feature.
\n\nVariables in mountpoint string:\n - ${clientid}: clientid\n - ${username}: username", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "atom()", - "kind" : "primitive" - }, - "raw_default" : "default", - "name" : "zone", - "desc" : "The configuration zone to which the listener belongs.", - "default" : { - "oneliner" : true, - "hocon" : "default" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:listener_fields", - "kind" : "struct" - }, - "name" : "limiter", - "desc" : "Type of the rate limit.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "true", - "false", - "quick_deny_anonymous" - ], - "kind" : "enum" - }, - "raw_default" : true, - "name" : "enable_authn", - "desc" : "Set true (default) to enable client authentication on this listener, the authentication\nprocess goes through the configured authentication chain.\nWhen set to false to allow any clients with or without authentication information such as username or password to log in.\nWhen set to quick_deny_anonymous, it behaves like when set to true, but clients will be\ndenied immediately without going through any authenticators if username is not provided. This is useful to fence off\nanonymous clients early.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "string()", - "kind" : "primitive" - } - }, - "raw_default" : [ - "allow all" - ], - "name" : "access_rules", - "desc" : "The access control rules for this listener.
See: https://github.com/emqtt/esockd#allowdeny", - "default" : { - "oneliner" : true, - "hocon" : "[\"allow all\"]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "proxy_protocol", - "desc" : "Enable the Proxy Protocol V1/2 if the EMQX cluster is deployed behind HAProxy or Nginx.
\nSee: https://www.haproxy.com/blog/haproxy/proxy-protocol/", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "3s", - "name" : "proxy_protocol_timeout", - "desc" : "Timeout for proxy protocol. EMQX will close the TCP connection if proxy protocol packet is not received within the timeout.", - "default" : { - "oneliner" : true, - "hocon" : "\"3s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:tcp_opts", - "kind" : "struct" - }, - "name" : "tcp_options", - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the MQTT over TCP listener." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.ws.$name" - ], - "full_name" : "broker:mqtt_ws_listener", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enabled", - "desc" : "Enable listener.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "emqx_schema:ip_port()", - "kind" : "primitive" - }, - { - "name" : "integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 8083, - "name" : "bind", - "desc" : "IP address and port for the listening socket.", - "default" : { - "oneliner" : true, - "hocon" : "8083" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "acceptors", - "desc" : "The size of the listener's receiving pool.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "pos_integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 5000000, - "name" : "max_connections", - "desc" : "The maximum number of concurrent connections allowed by the listener.", - "default" : { - "oneliner" : true, - "hocon" : "5000000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "mountpoint", - "desc" : "When publishing or subscribing, prefix all topics with a mountpoint string.\nThe prefixed string will be removed from the topic name when the message\nis delivered to the subscriber. The mountpoint is a way that users can use\nto implement isolation of message routing between different listeners.\nFor example if a client A subscribes to `t` with `listeners.tcp.\\.mountpoint`\nset to `some_tenant`, then the client actually subscribes to the topic\n`some_tenant/t`. Similarly, if another client B (connected to the same listener\nas the client A) sends a message to topic `t`, the message is routed\nto all the clients subscribed `some_tenant/t`, so client A will receive the\nmessage, with topic name `t`.
\nSet to `\"\"` to disable the feature.
\n\nVariables in mountpoint string:\n - ${clientid}: clientid\n - ${username}: username", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "atom()", - "kind" : "primitive" - }, - "raw_default" : "default", - "name" : "zone", - "desc" : "The configuration zone to which the listener belongs.", - "default" : { - "oneliner" : true, - "hocon" : "default" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:listener_fields", - "kind" : "struct" - }, - "name" : "limiter", - "desc" : "Type of the rate limit.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "true", - "false", - "quick_deny_anonymous" - ], - "kind" : "enum" - }, - "raw_default" : true, - "name" : "enable_authn", - "desc" : "Set true (default) to enable client authentication on this listener, the authentication\nprocess goes through the configured authentication chain.\nWhen set to false to allow any clients with or without authentication information such as username or password to log in.\nWhen set to quick_deny_anonymous, it behaves like when set to true, but clients will be\ndenied immediately without going through any authenticators if username is not provided. This is useful to fence off\nanonymous clients early.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "string()", - "kind" : "primitive" - } - }, - "raw_default" : [ - "allow all" - ], - "name" : "access_rules", - "desc" : "The access control rules for this listener.
See: https://github.com/emqtt/esockd#allowdeny", - "default" : { - "oneliner" : true, - "hocon" : "[\"allow all\"]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "proxy_protocol", - "desc" : "Enable the Proxy Protocol V1/2 if the EMQX cluster is deployed behind HAProxy or Nginx.
\nSee: https://www.haproxy.com/blog/haproxy/proxy-protocol/", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "3s", - "name" : "proxy_protocol_timeout", - "desc" : "Timeout for proxy protocol. EMQX will close the TCP connection if proxy protocol packet is not received within the timeout.", - "default" : { - "oneliner" : true, - "hocon" : "\"3s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:tcp_opts", - "kind" : "struct" - }, - "name" : "tcp_options", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ws_opts", - "kind" : "struct" - }, - "name" : "websocket", - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the MQTT over WebSocket listener." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.wss.$name" - ], - "full_name" : "broker:mqtt_wss_listener", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enabled", - "desc" : "Enable listener.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "emqx_schema:ip_port()", - "kind" : "primitive" - }, - { - "name" : "integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 8084, - "name" : "bind", - "desc" : "IP address and port for the listening socket.", - "default" : { - "oneliner" : true, - "hocon" : "8084" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "acceptors", - "desc" : "The size of the listener's receiving pool.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "pos_integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 5000000, - "name" : "max_connections", - "desc" : "The maximum number of concurrent connections allowed by the listener.", - "default" : { - "oneliner" : true, - "hocon" : "5000000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "mountpoint", - "desc" : "When publishing or subscribing, prefix all topics with a mountpoint string.\nThe prefixed string will be removed from the topic name when the message\nis delivered to the subscriber. The mountpoint is a way that users can use\nto implement isolation of message routing between different listeners.\nFor example if a client A subscribes to `t` with `listeners.tcp.\\.mountpoint`\nset to `some_tenant`, then the client actually subscribes to the topic\n`some_tenant/t`. Similarly, if another client B (connected to the same listener\nas the client A) sends a message to topic `t`, the message is routed\nto all the clients subscribed `some_tenant/t`, so client A will receive the\nmessage, with topic name `t`.
\nSet to `\"\"` to disable the feature.
\n\nVariables in mountpoint string:\n - ${clientid}: clientid\n - ${username}: username", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "atom()", - "kind" : "primitive" - }, - "raw_default" : "default", - "name" : "zone", - "desc" : "The configuration zone to which the listener belongs.", - "default" : { - "oneliner" : true, - "hocon" : "default" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:listener_fields", - "kind" : "struct" - }, - "name" : "limiter", - "desc" : "Type of the rate limit.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "true", - "false", - "quick_deny_anonymous" - ], - "kind" : "enum" - }, - "raw_default" : true, - "name" : "enable_authn", - "desc" : "Set true (default) to enable client authentication on this listener, the authentication\nprocess goes through the configured authentication chain.\nWhen set to false to allow any clients with or without authentication information such as username or password to log in.\nWhen set to quick_deny_anonymous, it behaves like when set to true, but clients will be\ndenied immediately without going through any authenticators if username is not provided. This is useful to fence off\nanonymous clients early.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "string()", - "kind" : "primitive" - } - }, - "raw_default" : [ - "allow all" - ], - "name" : "access_rules", - "desc" : "The access control rules for this listener.
See: https://github.com/emqtt/esockd#allowdeny", - "default" : { - "oneliner" : true, - "hocon" : "[\"allow all\"]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "proxy_protocol", - "desc" : "Enable the Proxy Protocol V1/2 if the EMQX cluster is deployed behind HAProxy or Nginx.
\nSee: https://www.haproxy.com/blog/haproxy/proxy-protocol/", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "3s", - "name" : "proxy_protocol_timeout", - "desc" : "Timeout for proxy protocol. EMQX will close the TCP connection if proxy protocol packet is not received within the timeout.", - "default" : { - "oneliner" : true, - "hocon" : "\"3s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:tcp_opts", - "kind" : "struct" - }, - "name" : "tcp_options", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:listener_wss_opts", - "kind" : "struct" - }, - "name" : "ssl_options", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ws_opts", - "kind" : "struct" - }, - "name" : "websocket", - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the MQTT over WebSocket/SSL listener." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.quic.$name.ssl_options.ocsp", - "listeners.ssl.$name.ssl_options.ocsp" - ], - "full_name" : "broker:ocsp", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "enable_ocsp_stapling", - "desc" : "Whether to enable Online Certificate Status Protocol (OCSP) stapling for the listener. If set to true, requires defining the OCSP responder URL and issuer PEM path.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:url()", - "kind" : "primitive" - }, - "name" : "responder_url", - "desc" : "URL for the OCSP responder to check the server certificate against.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "issuer_pem", - "desc" : "PEM-encoded certificate of the OCSP issuer for the server certificate.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5m", - "name" : "refresh_interval", - "desc" : "The period to refresh the OCSP response for the server.", - "default" : { - "oneliner" : true, - "hocon" : "\"5m\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "refresh_http_timeout", - "desc" : "The timeout for the HTTP request when checking OCSP responses.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Per listener OCSP Stapling configuration." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "authentication.$INDEX.ssl", - "authorization.sources.$INDEX.ssl", - "bridges.cassandra.$name.ssl", - "bridges.influxdb_api_v1.$name.ssl", - "bridges.influxdb_api_v2.$name.ssl", - "bridges.kafka.$name.ssl", - "bridges.kafka_consumer.$name.ssl", - "bridges.matrix.$name.ssl", - "bridges.mongodb_rs.$name.ssl", - "bridges.mongodb_sharded.$name.ssl", - "bridges.mongodb_single.$name.ssl", - "bridges.mqtt.$name.ssl", - "bridges.mysql.$name.ssl", - "bridges.pgsql.$name.ssl", - "bridges.redis_cluster.$name.ssl", - "bridges.redis_sentinel.$name.ssl", - "bridges.redis_single.$name.ssl", - "bridges.timescale.$name.ssl", - "bridges.webhook.$name.ssl", - "cluster.etcd.ssl" - ], - "full_name" : "broker:ssl_client_opts", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "cacertfile", - "desc" : "Trusted PEM format CA certificates bundle file.
\nThe certificates in this file are used to verify the TLS peer's certificates.\nAppend new certificates to the file if new CAs are to be trusted.\nThere is no need to restart EMQX to have the updated file loaded, because\nthe system regularly checks if file has been updated (and reload).
\nNOTE: invalidating (deleting) a certificate from the file will not affect\nalready established connections.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "certfile", - "desc" : "PEM format certificates chain file.
\nThe certificates in this file should be in reversed order of the certificate\nissue chain. That is, the host's certificate should be placed in the beginning\nof the file, followed by the immediate issuer certificate and so on.\nAlthough the root CA certificate is optional, it should be placed at the end of\nthe file if it is to be added.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "keyfile", - "desc" : "PEM format private key file.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "verify_peer", - "verify_none" - ], - "kind" : "enum" - }, - "raw_default" : "verify_none", - "name" : "verify", - "desc" : "Enable or disable peer verification.", - "default" : { - "oneliner" : true, - "hocon" : "verify_none" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "reuse_sessions", - "desc" : "Enable TLS session reuse.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 10, - "name" : "depth", - "desc" : "Maximum number of non-self-issued intermediate certificates that can follow the peer certificate in a valid certification path.\nSo, if depth is 0 the PEER must be signed by the trusted ROOT-CA directly;
\nif 1 the path can be PEER, Intermediate-CA, ROOT-CA;
\nif 2 the path can be PEER, Intermediate-CA1, Intermediate-CA2, ROOT-CA.", - "default" : { - "oneliner" : true, - "hocon" : "10" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "password", - "examples" : [ - "" - ], - "desc" : "String containing the user's password. Only used if the private key file is password-protected.", - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "atom()", - "kind" : "primitive" - } - }, - "raw_default" : [ - "tlsv1.3", - "tlsv1.2", - "tlsv1.1", - "tlsv1" - ], - "name" : "versions", - "desc" : "All TLS/DTLS versions to be supported.
\nNOTE: PSK ciphers are suppressed by 'tlsv1.3' version config.
\nIn case PSK cipher suites are intended, make sure to configure\n['tlsv1.2', 'tlsv1.1'] here.", - "default" : { - "oneliner" : true, - "hocon" : "[tlsv1.3, tlsv1.2, tlsv1.1, tlsv1]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "string()", - "kind" : "primitive" - } - }, - "raw_default" : [ - - ], - "name" : "ciphers", - "desc" : "This config holds TLS cipher suite names separated by comma,\nor as an array of strings. e.g.\n\"TLS_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256\" or\n[\"TLS_AES_256_GCM_SHA384\",\"TLS_AES_128_GCM_SHA256\"].\n
\nCiphers (and their ordering) define the way in which the\nclient and server encrypts information over the network connection.\nSelecting a good cipher suite is critical for the\napplication's data security, confidentiality and performance.\n\nThe names should be in OpenSSL string format (not RFC format).\nAll default values and examples provided by EMQX config\ndocumentation are all in OpenSSL format.
\n\nNOTE: Certain cipher suites are only compatible with\nspecific TLS versions ('tlsv1.1', 'tlsv1.2' or 'tlsv1.3')\nincompatible cipher suites will be silently dropped.\nFor instance, if only 'tlsv1.3' is given in the versions,\nconfiguring cipher suites for other versions will have no effect.\n
\n\nNOTE: PSK ciphers are suppressed by 'tlsv1.3' version config
\nIf PSK cipher suites are intended, 'tlsv1.3' should be disabled from versions.
\nPSK cipher suites: \"RSA-PSK-AES256-GCM-SHA384,RSA-PSK-AES256-CBC-SHA384,\nRSA-PSK-AES128-GCM-SHA256,RSA-PSK-AES128-CBC-SHA256,\nRSA-PSK-AES256-CBC-SHA,RSA-PSK-AES128-CBC-SHA,\nRSA-PSK-DES-CBC3-SHA,RSA-PSK-RC4-SHA\"", - "default" : { - "oneliner" : true, - "hocon" : "[]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "secure_renegotiate", - "desc" : "SSL parameter renegotiation is a feature that allows a client and a server\nto renegotiate the parameters of the SSL connection on the fly.\nRFC 5746 defines a more secure way of doing this. By enabling secure renegotiation,\nyou drop support for the insecure renegotiation, prone to MitM attacks.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "hibernate_after", - "desc" : "Hibernate the SSL process after idling for amount of time reducing its memory footprint.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "enable", - "desc" : "Enable TLS.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "disable", - "kind" : "singleton" - }, - { - "name" : "string()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "name" : "server_name_indication", - "examples" : [ - "disable" - ], - "desc" : "Specify the host name to be used in TLS Server Name Indication extension.
\nFor instance, when connecting to \"server.example.net\", the genuine server\nwhich accepts the connection and performs TLS handshake may differ from the\nhost the TLS client initially connects to, e.g. when connecting to an IP address\nor when the host has multiple resolvable DNS records
\nIf not specified, it will default to the host name string which is used\nto establish the connection, unless it is IP addressed used.
\nThe host name is then also used in the host name verification of the peer\ncertificate.
The special value 'disable' prevents the Server Name\nIndication extension from being sent and disables the hostname\nverification check.", - "aliases" : [ - - ] - } - ], - "desc" : "Socket options for SSL clients." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "sys_topics" - ], - "full_name" : "broker:sys_topics", - "fields" : [ - { - "type" : { - "members" : [ - { - "name" : "disabled", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "1m", - "name" : "sys_msg_interval", - "desc" : "Time interval of publishing `$SYS` messages.", - "default" : { - "oneliner" : true, - "hocon" : "\"1m\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "disabled", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "30s", - "name" : "sys_heartbeat_interval", - "desc" : "Time interval for publishing following heartbeat messages:\n - `$SYS/brokers//uptime`\n - `$SYS/brokers//datetime`", - "default" : { - "oneliner" : true, - "hocon" : "\"30s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:event_names", - "kind" : "struct" - }, - "name" : "sys_event_messages", - "desc" : "Client events messages.", - "aliases" : [ - - ] - } - ], - "desc" : "The EMQX Broker periodically publishes its own status, message statistics,\nclient online and offline events to the system topic starting with `$SYS/`.\n\nThe following options control the behavior of `$SYS` topics." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "sysmon" - ], - "full_name" : "broker:sysmon", - "fields" : [ - { - "type" : { - "name" : "broker:sysmon_vm", - "kind" : "struct" - }, - "name" : "vm", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:sysmon_os", - "kind" : "struct" - }, - "name" : "os", - "aliases" : [ - - ] - } - ], - "desc" : "Features related to system monitoring and introspection." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "sysmon.os" - ], - "full_name" : "broker:sysmon_os", - "fields" : [ - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "60s", - "name" : "cpu_check_interval", - "desc" : "The time interval for the periodic CPU check.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:percent()", - "kind" : "primitive" - }, - "raw_default" : "80%", - "name" : "cpu_high_watermark", - "desc" : "The threshold, as percentage of system CPU load,\n for how much system cpu can be used before the corresponding alarm is raised.", - "default" : { - "oneliner" : true, - "hocon" : "\"80%\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:percent()", - "kind" : "primitive" - }, - "raw_default" : "60%", - "name" : "cpu_low_watermark", - "desc" : "The threshold, as percentage of system CPU load,\n for how much system cpu can be used before the corresponding alarm is cleared.", - "default" : { - "oneliner" : true, - "hocon" : "\"60%\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "disabled", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "60s", - "name" : "mem_check_interval", - "desc" : "The time interval for the periodic memory check.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:percent()", - "kind" : "primitive" - }, - "raw_default" : "70%", - "name" : "sysmem_high_watermark", - "desc" : "The threshold, as percentage of system memory,\n for how much system memory can be allocated before the corresponding alarm is raised.", - "default" : { - "oneliner" : true, - "hocon" : "\"70%\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:percent()", - "kind" : "primitive" - }, - "raw_default" : "5%", - "name" : "procmem_high_watermark", - "desc" : "The threshold, as percentage of system memory,\n for how much system memory can be allocated by one Erlang process before\n the corresponding alarm is raised.", - "default" : { - "oneliner" : true, - "hocon" : "\"5%\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "This part of the configuration is responsible for monitoring\n the host OS health, such as free memory, disk space, CPU load, etc." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "sysmon.vm" - ], - "full_name" : "broker:sysmon_vm", - "fields" : [ - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "30s", - "name" : "process_check_interval", - "desc" : "The time interval for the periodic process limit check.", - "default" : { - "oneliner" : true, - "hocon" : "\"30s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:percent()", - "kind" : "primitive" - }, - "raw_default" : "80%", - "name" : "process_high_watermark", - "desc" : "The threshold, as percentage of processes, for how many\n processes can simultaneously exist at the local node before the corresponding\n alarm is raised.", - "default" : { - "oneliner" : true, - "hocon" : "\"80%\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:percent()", - "kind" : "primitive" - }, - "raw_default" : "60%", - "name" : "process_low_watermark", - "desc" : "The threshold, as percentage of processes, for how many\n processes can simultaneously exist at the local node before the corresponding\n alarm is cleared.", - "default" : { - "oneliner" : true, - "hocon" : "\"60%\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "disabled", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "disabled", - "name" : "long_gc", - "desc" : "When an Erlang process spends long time to perform garbage collection, a warning level long_gc log is emitted,\nand an MQTT message is published to the system topic $SYS/sysmon/long_gc.", - "default" : { - "oneliner" : true, - "hocon" : "disabled" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "disabled", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "240ms", - "name" : "long_schedule", - "desc" : "When the Erlang VM detect a task scheduled for too long, a warning level 'long_schedule' log is emitted,\nand an MQTT message is published to the system topic $SYS/sysmon/long_schedule.", - "default" : { - "oneliner" : true, - "hocon" : "\"240ms\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "disabled", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "32MB", - "name" : "large_heap", - "desc" : "When an Erlang process consumed a large amount of memory for its heap space,\nthe system will write a warning level large_heap log, and an MQTT message is published to\nthe system topic $SYS/sysmon/large_heap.", - "default" : { - "oneliner" : true, - "hocon" : "\"32MB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "busy_dist_port", - "desc" : "When the RPC connection used to communicate with other nodes in the cluster is overloaded,\nthere will be a busy_dist_port warning log,\nand an MQTT message is published to system topic $SYS/sysmon/busy_dist_port.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "busy_port", - "desc" : "When a port (e.g. TCP socket) is overloaded, there will be a busy_port warning log,\nand an MQTT message is published to the system topic $SYS/sysmon/busy_port.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "This part of the configuration is responsible for collecting\n BEAM VM events, such as long garbage collection, traffic congestion in the inter-broker\n communication, etc." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.ssl.$name.tcp_options", - "listeners.tcp.$name.tcp_options", - "listeners.ws.$name.tcp_options", - "listeners.wss.$name.tcp_options" - ], - "full_name" : "broker:tcp_opts", - "fields" : [ - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "active_n", - "desc" : "Specify the {active, N} option for this Socket.
\nSee: https://erlang.org/doc/man/inet.html#setopts-2", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 1024, - "name" : "backlog", - "desc" : "TCP backlog defines the maximum length that the queue of\npending connections can grow to.", - "default" : { - "oneliner" : true, - "hocon" : "1024" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "send_timeout", - "desc" : "The TCP send timeout for the connections.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "send_timeout_close", - "desc" : "Close the connection if send timeout.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "name" : "recbuf", - "examples" : [ - "2KB" - ], - "desc" : "The TCP receive buffer (OS kernel) for the connections.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "name" : "sndbuf", - "examples" : [ - "4KB" - ], - "desc" : "The TCP send buffer (OS kernel) for the connections.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "4KB", - "name" : "buffer", - "examples" : [ - "4KB" - ], - "desc" : "The size of the user-space buffer used by the driver.", - "default" : { - "oneliner" : true, - "hocon" : "\"4KB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "1MB", - "name" : "high_watermark", - "desc" : "The socket is set to a busy state when the amount of data queued internally\nby the VM socket implementation reaches this limit.", - "default" : { - "oneliner" : true, - "hocon" : "\"1MB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "nodelay", - "desc" : "The TCP_NODELAY flag for the connections.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "reuseaddr", - "desc" : "The SO_REUSEADDR flag for the connections.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "TCP listener options." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "listeners.ws.$name.websocket", - "listeners.wss.$name.websocket" - ], - "full_name" : "broker:ws_opts", - "fields" : [ - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "/mqtt", - "name" : "mqtt_path", - "desc" : "WebSocket's MQTT protocol path. So the address of EMQX Broker's WebSocket is:\nws://{ip}:{port}/mqtt", - "default" : { - "oneliner" : true, - "hocon" : "\"/mqtt\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "single", - "multiple" - ], - "kind" : "enum" - }, - "raw_default" : "multiple", - "name" : "mqtt_piggyback", - "desc" : "Whether a WebSocket message is allowed to contain multiple MQTT packets.", - "default" : { - "oneliner" : true, - "hocon" : "multiple" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "compress", - "desc" : "If true, compress WebSocket messages using zlib.
\nThe configuration items under deflate_opts belong to the compression-related parameter configuration.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "7200s", - "name" : "idle_timeout", - "desc" : "Close transport-layer connections from the clients that have not sent MQTT CONNECT message within this interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"7200s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "infinity", - "name" : "max_frame_size", - "desc" : "The maximum length of a single MQTT packet.", - "default" : { - "oneliner" : true, - "hocon" : "infinity" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "fail_if_no_subprotocol", - "desc" : "If true, the server will return an error when\n the client does not carry the Sec-WebSocket-Protocol field.\n
Note: WeChat applet needs to disable this verification.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:comma_separated_list()", - "kind" : "primitive" - }, - "raw_default" : "mqtt, mqtt-v3, mqtt-v3.1.1, mqtt-v5", - "name" : "supported_subprotocols", - "desc" : "Comma-separated list of supported subprotocols.", - "default" : { - "oneliner" : true, - "hocon" : "\"mqtt, mqtt-v3, mqtt-v3.1.1, mqtt-v5\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "check_origin_enable", - "desc" : "If true, origin HTTP header will be\n validated against the list of allowed origins configured in check_origins\n parameter.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "allow_origin_absence", - "desc" : "If false and check_origin_enable is\n true, the server will reject requests that don't have origin\n HTTP header.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:comma_separated_binary()", - "kind" : "primitive" - }, - "raw_default" : "http://localhost:18083, http://127.0.0.1:18083", - "name" : "check_origins", - "desc" : "List of allowed origins.
See check_origin_enable.", - "default" : { - "oneliner" : true, - "hocon" : "\"http://localhost:18083, http://127.0.0.1:18083\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "x-forwarded-for", - "name" : "proxy_address_header", - "desc" : "HTTP header used to pass information about the client IP address.\nRelevant when the EMQX cluster is deployed behind a load-balancer.", - "default" : { - "oneliner" : true, - "hocon" : "\"x-forwarded-for\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "x-forwarded-port", - "name" : "proxy_port_header", - "desc" : "HTTP header used to pass information about the client port. Relevant when the EMQX cluster is deployed behind a load-balancer.", - "default" : { - "oneliner" : true, - "hocon" : "\"x-forwarded-port\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:deflate_opts", - "kind" : "struct" - }, - "name" : "deflate_opts", - "aliases" : [ - - ] - } - ], - "desc" : "WebSocket listener options." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "zones.$name" - ], - "full_name" : "broker:zone", - "fields" : [ - { - "type" : { - "name" : "zone:mqtt", - "kind" : "struct" - }, - "name" : "mqtt", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "zone:force_shutdown", - "kind" : "struct" - }, - "name" : "force_shutdown", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "zone:force_gc", - "kind" : "struct" - }, - "name" : "force_gc", - "aliases" : [ - - ] - } - ], - "desc" : "A `Zone` defines a set of configuration items (such as the maximum number of connections) that can be shared between multiple listeners.\n\n`Listener` can refer to a `Zone` through the configuration item listener.\\.\\.zone.\n\nThe configs defined in the zones will override the global configs with the same key.\n\nFor example, given the following config:\n```\na {\n b: 1, c: 1\n}\nzone.my_zone {\n a {\n b:2\n }\n}\n```\n\nThe global config `a` is overridden by the configs `a` inside the zone `my_zone`.\n\nIf there is a listener using the zone `my_zone`, the value of config `a` will be: `{b:2, c: 1}`.\nNote that although the default value of `a.c` is `0`, the global value is used, i.e. configs in the zone have no default values. To override `a.c` one must configure it explicitly in the zone.\n\nAll the global configs that can be overridden in zones are:\n - `stats.*`\n - `mqtt.*`\n - `authorization.*`\n - `flapping_detect.*`\n - `force_shutdown.*`\n - `conn_congestion.*`\n - `force_gc.*`\n\n" - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.hstreamdb.$name.connector" - ], - "full_name" : "connector_hstreamdb:config", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "url", - "desc" : "HStreamDB Server URL", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "stream", - "desc" : "HStreamDB Stream Name", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "ordering_key", - "desc" : "HStreamDB Ordering Key", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "name" : "pool_size", - "desc" : "HStreamDB Pool Size", - "aliases" : [ - - ] - } - ], - "desc" : "HStreamDB connection config" - }, - { - "tags" : [ - - ], - "paths" : [ - "dashboard" - ], - "full_name" : "dashboard", - "fields" : [ - { - "type" : { - "name" : "dashboard:listeners", - "kind" : "struct" - }, - "name" : "listeners", - "desc" : "HTTP(s) listeners are identified by their protocol type and are\nused to serve dashboard UI and restful HTTP API.\nListeners must have a unique combination of port number and IP address.\nFor example, an HTTP listener can listen on all configured IP addresses\non a given port for a machine by specifying the IP address 0.0.0.0.\nAlternatively, the HTTP listener can specify a unique IP address for each listener,\nbut use the same port.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "60m", - "name" : "token_expired_time", - "desc" : "JWT token expiration time. Default is 60 minutes", - "default" : { - "oneliner" : true, - "hocon" : "\"60m\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "cors", - "desc" : "Support Cross-Origin Resource Sharing (CORS).\nAllows a server to indicate any origins (domain, scheme, or port) other than\nits own from which a browser should permit loading resources.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for EMQX dashboard." - }, - { - "tags" : [ - - ], - "paths" : [ - "dashboard.listeners.http" - ], - "full_name" : "dashboard:http", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Ignore or enable this listener", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - { - "name" : "emqx_schema:ip_port()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 18083, - "name" : "bind", - "examples" : [ - [ - 48, - 46, - 48, - 46, - 48, - 46, - 48, - 58, - 49, - 56, - 48, - 56, - 51 - ] - ], - "desc" : "Port without IP(18083) or port with specified IP(127.0.0.1:18083).", - "default" : { - "oneliner" : true, - "hocon" : "18083" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "num_acceptors", - "desc" : "Socket acceptor pool size for TCP protocols. Default is the number of schedulers online", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 512, - "name" : "max_connections", - "desc" : "Maximum number of simultaneous connections.", - "default" : { - "oneliner" : true, - "hocon" : "512" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 1024, - "name" : "backlog", - "desc" : "Defines the maximum length that the queue of pending connections can grow to.", - "default" : { - "oneliner" : true, - "hocon" : "1024" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "10s", - "name" : "send_timeout", - "desc" : "Send timeout for the socket.", - "default" : { - "oneliner" : true, - "hocon" : "\"10s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "inet6", - "desc" : "Enable IPv6 support, default is false, which means IPv4 only.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "ipv6_v6only", - "desc" : "Disable IPv4-to-IPv6 mapping for the listener.\nThe configuration is only valid when the inet6 is true.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "proxy_header", - "desc" : "Enable support for `HAProxy` header. Be aware once enabled regular HTTP requests can't be handled anymore.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for the dashboard listener (plaintext)." - }, - { - "tags" : [ - - ], - "paths" : [ - "dashboard.listeners.https" - ], - "full_name" : "dashboard:https", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "enable", - "desc" : "Ignore or enable this listener", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - { - "name" : "emqx_schema:ip_port()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 18084, - "name" : "bind", - "examples" : [ - [ - 48, - 46, - 48, - 46, - 48, - 46, - 48, - 58, - 49, - 56, - 48, - 56, - 52 - ] - ], - "desc" : "Port without IP(18083) or port with specified IP(127.0.0.1:18083).", - "default" : { - "oneliner" : true, - "hocon" : "18084" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "num_acceptors", - "desc" : "Socket acceptor pool size for TCP protocols. Default is the number of schedulers online", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 512, - "name" : "max_connections", - "desc" : "Maximum number of simultaneous connections.", - "default" : { - "oneliner" : true, - "hocon" : "512" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 1024, - "name" : "backlog", - "desc" : "Defines the maximum length that the queue of pending connections can grow to.", - "default" : { - "oneliner" : true, - "hocon" : "1024" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "10s", - "name" : "send_timeout", - "desc" : "Send timeout for the socket.", - "default" : { - "oneliner" : true, - "hocon" : "\"10s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "inet6", - "desc" : "Enable IPv6 support, default is false, which means IPv4 only.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "ipv6_v6only", - "desc" : "Disable IPv4-to-IPv6 mapping for the listener.\nThe configuration is only valid when the inet6 is true.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "proxy_header", - "desc" : "Enable support for `HAProxy` header. Be aware once enabled regular HTTP requests can't be handled anymore.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "cacertfile", - "desc" : "Trusted PEM format CA certificates bundle file.
\nThe certificates in this file are used to verify the TLS peer's certificates.\nAppend new certificates to the file if new CAs are to be trusted.\nThere is no need to restart EMQX to have the updated file loaded, because\nthe system regularly checks if file has been updated (and reload).
\nNOTE: invalidating (deleting) a certificate from the file will not affect\nalready established connections.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "certfile", - "desc" : "PEM format certificates chain file.
\nThe certificates in this file should be in reversed order of the certificate\nissue chain. That is, the host's certificate should be placed in the beginning\nof the file, followed by the immediate issuer certificate and so on.\nAlthough the root CA certificate is optional, it should be placed at the end of\nthe file if it is to be added.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "keyfile", - "desc" : "PEM format private key file.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "verify_peer", - "verify_none" - ], - "kind" : "enum" - }, - "raw_default" : "verify_none", - "name" : "verify", - "desc" : "Enable or disable peer verification.", - "default" : { - "oneliner" : true, - "hocon" : "verify_none" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "reuse_sessions", - "desc" : "Enable TLS session reuse.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 10, - "name" : "depth", - "desc" : "Maximum number of non-self-issued intermediate certificates that can follow the peer certificate in a valid certification path.\nSo, if depth is 0 the PEER must be signed by the trusted ROOT-CA directly;
\nif 1 the path can be PEER, Intermediate-CA, ROOT-CA;
\nif 2 the path can be PEER, Intermediate-CA1, Intermediate-CA2, ROOT-CA.", - "default" : { - "oneliner" : true, - "hocon" : "10" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "atom()", - "kind" : "primitive" - } - }, - "raw_default" : [ - "tlsv1.3", - "tlsv1.2", - "tlsv1.1", - "tlsv1" - ], - "name" : "versions", - "desc" : "All TLS/DTLS versions to be supported.
\nNOTE: PSK ciphers are suppressed by 'tlsv1.3' version config.
\nIn case PSK cipher suites are intended, make sure to configure\n['tlsv1.2', 'tlsv1.1'] here.", - "default" : { - "oneliner" : true, - "hocon" : "[tlsv1.3, tlsv1.2, tlsv1.1, tlsv1]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "string()", - "kind" : "primitive" - } - }, - "raw_default" : [ - - ], - "name" : "ciphers", - "desc" : "This config holds TLS cipher suite names separated by comma,\nor as an array of strings. e.g.\n\"TLS_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256\" or\n[\"TLS_AES_256_GCM_SHA384\",\"TLS_AES_128_GCM_SHA256\"].\n
\nCiphers (and their ordering) define the way in which the\nclient and server encrypts information over the network connection.\nSelecting a good cipher suite is critical for the\napplication's data security, confidentiality and performance.\n\nThe names should be in OpenSSL string format (not RFC format).\nAll default values and examples provided by EMQX config\ndocumentation are all in OpenSSL format.
\n\nNOTE: Certain cipher suites are only compatible with\nspecific TLS versions ('tlsv1.1', 'tlsv1.2' or 'tlsv1.3')\nincompatible cipher suites will be silently dropped.\nFor instance, if only 'tlsv1.3' is given in the versions,\nconfiguring cipher suites for other versions will have no effect.\n
\n\nNOTE: PSK ciphers are suppressed by 'tlsv1.3' version config
\nIf PSK cipher suites are intended, 'tlsv1.3' should be disabled from versions.
\nPSK cipher suites: \"RSA-PSK-AES256-GCM-SHA384,RSA-PSK-AES256-CBC-SHA384,\nRSA-PSK-AES128-GCM-SHA256,RSA-PSK-AES128-CBC-SHA256,\nRSA-PSK-AES256-CBC-SHA,RSA-PSK-AES128-CBC-SHA,\nRSA-PSK-DES-CBC3-SHA,RSA-PSK-RC4-SHA\"", - "default" : { - "oneliner" : true, - "hocon" : "[]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "secure_renegotiate", - "desc" : "SSL parameter renegotiation is a feature that allows a client and a server\nto renegotiate the parameters of the SSL connection on the fly.\nRFC 5746 defines a more secure way of doing this. By enabling secure renegotiation,\nyou drop support for the insecure renegotiation, prone to MitM attacks.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "hibernate_after", - "desc" : "Hibernate the SSL process after idling for amount of time reducing its memory footprint.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "dhfile", - "desc" : "Path to a file containing PEM-encoded Diffie-Hellman parameters\nto be used by the server if a cipher suite using Diffie-Hellman\nkey exchange is negotiated. If not specified, default parameters\nare used.
\nNOTE: The dhfile option is not supported by TLS 1.3.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "honor_cipher_order", - "desc" : "An important security setting, it forces the cipher to be set based\n on the server-specified order instead of the client-specified order,\n hence enforcing the (usually more properly configured) security\n ordering of the server administrator.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "client_renegotiation", - "desc" : "In protocols that support client-initiated renegotiation,\nthe cost of resources of such an operation is higher for the server than the client.\nThis can act as a vector for denial of service attacks.\nThe SSL application already takes measures to counter-act such attempts,\nbut client-initiated renegotiation can be strictly disabled by setting this option to false.\nThe default value is true. Note that disabling renegotiation can result in\nlong-lived connections becoming unusable due to limits on\nthe number of messages the underlying cipher suite can encipher.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "handshake_timeout", - "desc" : "Maximum time duration allowed for the handshake to complete", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for the dashboard listener (TLS)." - }, - { - "tags" : [ - - ], - "paths" : [ - "dashboard.listeners" - ], - "full_name" : "dashboard:listeners", - "fields" : [ - { - "type" : { - "name" : "dashboard:http", - "kind" : "struct" - }, - "name" : "http", - "desc" : "TCP listeners", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "dashboard:https", - "kind" : "struct" - }, - "name" : "https", - "desc" : "SSL listeners", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for the dashboard listener." - }, - { - "tags" : [ - - ], - "paths" : [ - "exhook" - ], - "full_name" : "exhook", - "fields" : [ - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "exhook:server", - "kind" : "struct" - } - }, - "raw_default" : [ - - ], - "name" : "servers", - "desc" : "List of exhook servers", - "default" : { - "oneliner" : true, - "hocon" : "[]" - }, - "aliases" : [ - - ] - } - ], - "desc" : "External hook (exhook) configuration." - }, - { - "tags" : [ - - ], - "paths" : [ - "exhook.servers.$INDEX" - ], - "full_name" : "exhook:server", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "name", - "examples" : [ - "default" - ], - "desc" : "Name of the exhook server", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable this Exhook server", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "url", - "examples" : [ - "http://127.0.0.1:9000" - ], - "desc" : "URL of the gRPC server", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "request_timeout", - "desc" : "The timeout of request gRPC server", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "deny", - "ignore" - ], - "kind" : "enum" - }, - "raw_default" : "deny", - "name" : "failed_action", - "desc" : "The value that is returned when the request to the gRPC server fails for any reason", - "default" : { - "oneliner" : true, - "hocon" : "deny" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "exhook:ssl_conf", - "kind" : "struct" - }, - "name" : "ssl", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "exhook:socket_options", - "kind" : "struct" - }, - "raw_default" : { - "nodelay" : true, - "keepalive" : true - }, - "name" : "socket_options", - "default" : { - "oneliner" : true, - "hocon" : "{keepalive = true, nodelay = true}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "false", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "60s", - "name" : "auto_reconnect", - "desc" : "Whether to automatically reconnect (initialize) the gRPC server.\nWhen gRPC is not available, Exhook tries to request the gRPC service at that interval and reinitialize the list of mounted hooks.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "The process pool size for gRPC client", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - } - ], - "desc" : "gRPC server configuration." - }, - { - "tags" : [ - - ], - "paths" : [ - "exhook.servers.$INDEX.socket_options" - ], - "full_name" : "exhook:socket_options", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "keepalive", - "desc" : "Enables/disables periodic transmission on a connected socket when no other data is exchanged.\nIf the other end does not respond, the connection is considered broken and an error message is sent to the controlling process.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "nodelay", - "desc" : "If true, option TCP_NODELAY is turned on for the socket,\nwhich means that also small amounts of data are sent immediately", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "name" : "recbuf", - "examples" : [ - "64KB" - ], - "desc" : "The minimum size of receive buffer to use for the socket", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "name" : "sndbuf", - "examples" : [ - "16KB" - ], - "desc" : "The minimum size of send buffer to use for the socket", - "aliases" : [ - - ] - } - ], - "desc" : "Connection socket options" - }, - { - "tags" : [ - - ], - "paths" : [ - "exhook.servers.$INDEX.ssl" - ], - "full_name" : "exhook:ssl_conf", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "cacertfile", - "desc" : "Trusted PEM format CA certificates bundle file.
\nThe certificates in this file are used to verify the TLS peer's certificates.\nAppend new certificates to the file if new CAs are to be trusted.\nThere is no need to restart EMQX to have the updated file loaded, because\nthe system regularly checks if file has been updated (and reload).
\nNOTE: invalidating (deleting) a certificate from the file will not affect\nalready established connections.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "certfile", - "desc" : "PEM format certificates chain file.
\nThe certificates in this file should be in reversed order of the certificate\nissue chain. That is, the host's certificate should be placed in the beginning\nof the file, followed by the immediate issuer certificate and so on.\nAlthough the root CA certificate is optional, it should be placed at the end of\nthe file if it is to be added.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "keyfile", - "desc" : "PEM format private key file.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "verify_peer", - "verify_none" - ], - "kind" : "enum" - }, - "raw_default" : "verify_none", - "name" : "verify", - "desc" : "Enable or disable peer verification.", - "default" : { - "oneliner" : true, - "hocon" : "verify_none" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "reuse_sessions", - "desc" : "Enable TLS session reuse.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 10, - "name" : "depth", - "desc" : "Maximum number of non-self-issued intermediate certificates that can follow the peer certificate in a valid certification path.\nSo, if depth is 0 the PEER must be signed by the trusted ROOT-CA directly;
\nif 1 the path can be PEER, Intermediate-CA, ROOT-CA;
\nif 2 the path can be PEER, Intermediate-CA1, Intermediate-CA2, ROOT-CA.", - "default" : { - "oneliner" : true, - "hocon" : "10" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "password", - "examples" : [ - "" - ], - "desc" : "String containing the user's password. Only used if the private key file is password-protected.", - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "atom()", - "kind" : "primitive" - } - }, - "raw_default" : [ - "tlsv1.3", - "tlsv1.2", - "tlsv1.1", - "tlsv1" - ], - "name" : "versions", - "desc" : "All TLS/DTLS versions to be supported.
\nNOTE: PSK ciphers are suppressed by 'tlsv1.3' version config.
\nIn case PSK cipher suites are intended, make sure to configure\n['tlsv1.2', 'tlsv1.1'] here.", - "default" : { - "oneliner" : true, - "hocon" : "[tlsv1.3, tlsv1.2, tlsv1.1, tlsv1]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "string()", - "kind" : "primitive" - } - }, - "raw_default" : [ - - ], - "name" : "ciphers", - "desc" : "This config holds TLS cipher suite names separated by comma,\nor as an array of strings. e.g.\n\"TLS_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256\" or\n[\"TLS_AES_256_GCM_SHA384\",\"TLS_AES_128_GCM_SHA256\"].\n
\nCiphers (and their ordering) define the way in which the\nclient and server encrypts information over the network connection.\nSelecting a good cipher suite is critical for the\napplication's data security, confidentiality and performance.\n\nThe names should be in OpenSSL string format (not RFC format).\nAll default values and examples provided by EMQX config\ndocumentation are all in OpenSSL format.
\n\nNOTE: Certain cipher suites are only compatible with\nspecific TLS versions ('tlsv1.1', 'tlsv1.2' or 'tlsv1.3')\nincompatible cipher suites will be silently dropped.\nFor instance, if only 'tlsv1.3' is given in the versions,\nconfiguring cipher suites for other versions will have no effect.\n
\n\nNOTE: PSK ciphers are suppressed by 'tlsv1.3' version config
\nIf PSK cipher suites are intended, 'tlsv1.3' should be disabled from versions.
\nPSK cipher suites: \"RSA-PSK-AES256-GCM-SHA384,RSA-PSK-AES256-CBC-SHA384,\nRSA-PSK-AES128-GCM-SHA256,RSA-PSK-AES128-CBC-SHA256,\nRSA-PSK-AES256-CBC-SHA,RSA-PSK-AES128-CBC-SHA,\nRSA-PSK-DES-CBC3-SHA,RSA-PSK-RC4-SHA\"", - "default" : { - "oneliner" : true, - "hocon" : "[]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "secure_renegotiate", - "desc" : "SSL parameter renegotiation is a feature that allows a client and a server\nto renegotiate the parameters of the SSL connection on the fly.\nRFC 5746 defines a more secure way of doing this. By enabling secure renegotiation,\nyou drop support for the insecure renegotiation, prone to MitM attacks.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "hibernate_after", - "desc" : "Hibernate the SSL process after idling for amount of time reducing its memory footprint.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "enable", - "desc" : "Enable TLS.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "disable", - "kind" : "singleton" - }, - { - "name" : "string()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "name" : "server_name_indication", - "examples" : [ - "disable" - ], - "desc" : "Specify the host name to be used in TLS Server Name Indication extension.
\nFor instance, when connecting to \"server.example.net\", the genuine server\nwhich accepts the connection and performs TLS handshake may differ from the\nhost the TLS client initially connects to, e.g. when connecting to an IP address\nor when the host has multiple resolvable DNS records
\nIf not specified, it will default to the host name string which is used\nto establish the connection, unless it is IP addressed used.
\nThe host name is then also used in the host name verification of the peer\ncertificate.
The special value 'disable' prevents the Server Name\nIndication extension from being sent and disables the hostname\nverification check.", - "aliases" : [ - - ] - } - ], - "desc" : "SSL client configuration." - }, - { - "tags" : [ - - ], - "paths" : [ - "listeners.quic.$name.limiter.bytes_in", - "listeners.quic.$name.limiter.message_in", - "listeners.quic.$name.limiter.message_routing", - "listeners.ssl.$name.limiter.bytes_in", - "listeners.ssl.$name.limiter.message_in", - "listeners.ssl.$name.limiter.message_routing", - "listeners.tcp.$name.limiter.bytes_in", - "listeners.tcp.$name.limiter.message_in", - "listeners.tcp.$name.limiter.message_routing", - "listeners.ws.$name.limiter.bytes_in", - "listeners.ws.$name.limiter.message_in", - "listeners.ws.$name.limiter.message_routing", - "listeners.wss.$name.limiter.bytes_in", - "listeners.wss.$name.limiter.message_in", - "listeners.wss.$name.limiter.message_routing" - ], - "full_name" : "limiter:bucket_infinity", - "fields" : [ - { - "type" : { - "name" : "emqx_limiter_schema:rate()", - "kind" : "primitive" - }, - "raw_default" : "infinity", - "name" : "rate", - "desc" : "Rate for this bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"infinity\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_limiter_schema:capacity()", - "kind" : "primitive" - }, - "raw_default" : "infinity", - "name" : "capacity", - "desc" : "The capacity of this token bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"infinity\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_limiter_schema:initial()", - "kind" : "primitive" - }, - "raw_default" : "0", - "name" : "initial", - "desc" : "The initial number of tokens for this bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"0\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the bucket." - }, - { - "tags" : [ - - ], - "paths" : [ - "listeners.quic.$name.limiter.connection", - "listeners.ssl.$name.limiter.connection", - "listeners.tcp.$name.limiter.connection", - "listeners.ws.$name.limiter.connection", - "listeners.wss.$name.limiter.connection" - ], - "full_name" : "limiter:bucket_limit", - "fields" : [ - { - "type" : { - "name" : "emqx_limiter_schema:rate()", - "kind" : "primitive" - }, - "raw_default" : "1000/s", - "name" : "rate", - "desc" : "Rate for this bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"1000/s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_limiter_schema:capacity()", - "kind" : "primitive" - }, - "raw_default" : "1000", - "name" : "capacity", - "desc" : "The capacity of this token bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"1000\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_limiter_schema:initial()", - "kind" : "primitive" - }, - "raw_default" : "0", - "name" : "initial", - "desc" : "The initial number of tokens for this bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"0\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the bucket." - }, - { - "tags" : [ - - ], - "paths" : [ - "limiter.client" - ], - "full_name" : "limiter:client_fields", - "fields" : [ - { - "type" : { - "name" : "limiter:client_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "bytes_in", - "desc" : "The bytes_in limiter.\nThis is used to limit the inbound bytes rate for this EMQX node.\nOnce the limit is reached, the restricted client will be slow down even be hung for a while.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:client_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "message_in", - "desc" : "The message in limiter.\nThis is used to limit the inbound message numbers for this EMQX node\nOnce the limit is reached, the restricted client will be slow down even be hung for a while.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:client_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "connection", - "desc" : "The connection limiter.\nThis is used to limit the connection rate for this EMQX node.\nOnce the limit is reached, new connections will be refused", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:client_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "message_routing", - "desc" : "The message routing limiter.\nThis is used to limit the forwarding rate for this EMQX node.\nOnce the limit is reached, new publish will be refused", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:client_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "internal", - "desc" : "Limiter for EMQX internal app.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Fields of the client level." - }, - { - "tags" : [ - - ], - "paths" : [ - "limiter.client.bytes_in", - "limiter.client.connection", - "limiter.client.internal", - "limiter.client.message_in", - "limiter.client.message_routing", - "listeners.quic.$name.limiter.client.bytes_in", - "listeners.quic.$name.limiter.client.connection", - "listeners.quic.$name.limiter.client.message_in", - "listeners.quic.$name.limiter.client.message_routing", - "listeners.ssl.$name.limiter.client.bytes_in", - "listeners.ssl.$name.limiter.client.connection", - "listeners.ssl.$name.limiter.client.message_in", - "listeners.ssl.$name.limiter.client.message_routing", - "listeners.tcp.$name.limiter.client.bytes_in", - "listeners.tcp.$name.limiter.client.connection", - "listeners.tcp.$name.limiter.client.message_in", - "listeners.tcp.$name.limiter.client.message_routing", - "listeners.ws.$name.limiter.client.bytes_in", - "listeners.ws.$name.limiter.client.connection", - "listeners.ws.$name.limiter.client.message_in", - "listeners.ws.$name.limiter.client.message_routing", - "listeners.wss.$name.limiter.client.bytes_in", - "listeners.wss.$name.limiter.client.connection", - "listeners.wss.$name.limiter.client.message_in", - "listeners.wss.$name.limiter.client.message_routing", - "retainer.flow_control.batch_deliver_limiter.client" - ], - "full_name" : "limiter:client_opts", - "fields" : [ - { - "type" : { - "name" : "emqx_limiter_schema:rate()", - "kind" : "primitive" - }, - "raw_default" : "infinity", - "name" : "rate", - "desc" : "Rate for this bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"infinity\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_limiter_schema:initial()", - "kind" : "primitive" - }, - "raw_default" : "0", - "name" : "initial", - "desc" : "The initial number of tokens for this bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"0\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_limiter_schema:initial()", - "kind" : "primitive" - }, - "raw_default" : "0", - "name" : "low_watermark", - "desc" : "If the remaining tokens are lower than this value,\nthe check/consume will succeed, but it will be forced to wait for a short period of time.", - "default" : { - "oneliner" : true, - "hocon" : "\"0\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_limiter_schema:capacity()", - "kind" : "primitive" - }, - "raw_default" : "infinity", - "name" : "capacity", - "desc" : "The capacity of per user.", - "default" : { - "oneliner" : true, - "hocon" : "\"infinity\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "divisible", - "desc" : "Is it possible to split the number of requested tokens?", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "10s", - "name" : "max_retry_time", - "desc" : "The maximum retry time when acquire failed.", - "default" : { - "oneliner" : true, - "hocon" : "\"10s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_limiter_schema:failure_strategy()", - "kind" : "primitive" - }, - "raw_default" : "force", - "name" : "failure_strategy", - "desc" : "The strategy when all the retries failed.", - "default" : { - "oneliner" : true, - "hocon" : "force" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the client in bucket level." - }, - { - "tags" : [ - - ], - "paths" : [ - "retainer.flow_control.batch_deliver_limiter" - ], - "full_name" : "limiter:internal", - "fields" : [ - { - "type" : { - "name" : "emqx_limiter_schema:rate()", - "kind" : "primitive" - }, - "raw_default" : "infinity", - "name" : "rate", - "desc" : "Rate for this bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"infinity\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_limiter_schema:capacity()", - "kind" : "primitive" - }, - "raw_default" : "infinity", - "name" : "capacity", - "desc" : "The capacity of this token bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"infinity\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_limiter_schema:initial()", - "kind" : "primitive" - }, - "raw_default" : "0", - "name" : "initial", - "desc" : "The initial number of tokens for this bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"0\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:client_opts", - "kind" : "struct" - }, - "name" : "client", - "desc" : "The rate limit for each user of the bucket", - "aliases" : [ - - ] - } - ], - "desc" : "Internal limiter." - }, - { - "tags" : [ - - ], - "paths" : [ - "limiter" - ], - "full_name" : "limiter", - "fields" : [ - { - "type" : { - "name" : "limiter:node_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "bytes_in", - "desc" : "The bytes_in limiter.\nThis is used to limit the inbound bytes rate for this EMQX node.\nOnce the limit is reached, the restricted client will be slow down even be hung for a while.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:node_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "message_in", - "desc" : "The message in limiter.\nThis is used to limit the inbound message numbers for this EMQX node\nOnce the limit is reached, the restricted client will be slow down even be hung for a while.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:node_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "connection", - "desc" : "The connection limiter.\nThis is used to limit the connection rate for this EMQX node.\nOnce the limit is reached, new connections will be refused", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:node_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "message_routing", - "desc" : "The message routing limiter.\nThis is used to limit the forwarding rate for this EMQX node.\nOnce the limit is reached, new publish will be refused", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:node_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "internal", - "desc" : "Limiter for EMQX internal app.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:client_fields", - "kind" : "struct" - }, - "raw_default" : { - "message_routing" : { - - }, - "message_in" : { - - }, - "internal" : { - - }, - "connection" : { - - }, - "bytes_in" : { - - } - }, - "name" : "client", - "desc" : "The rate limit for each user of the bucket", - "default" : { - "oneliner" : false, - "hocon" : "{\n bytes_in {}\n connection {}\n internal {}\n message_in {}\n message_routing {}\n}\n" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the rate limiter." - }, - { - "tags" : [ - - ], - "paths" : [ - "listeners.quic.$name.limiter.client", - "listeners.ssl.$name.limiter.client", - "listeners.tcp.$name.limiter.client", - "listeners.ws.$name.limiter.client", - "listeners.wss.$name.limiter.client" - ], - "full_name" : "limiter:listener_client_fields", - "fields" : [ - { - "type" : { - "name" : "limiter:client_opts", - "kind" : "struct" - }, - "name" : "bytes_in", - "desc" : "The bytes_in limiter.\nThis is used to limit the inbound bytes rate for this EMQX node.\nOnce the limit is reached, the restricted client will be slow down even be hung for a while.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:client_opts", - "kind" : "struct" - }, - "name" : "message_in", - "desc" : "The message in limiter.\nThis is used to limit the inbound message numbers for this EMQX node\nOnce the limit is reached, the restricted client will be slow down even be hung for a while.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:client_opts", - "kind" : "struct" - }, - "name" : "connection", - "desc" : "The connection limiter.\nThis is used to limit the connection rate for this EMQX node.\nOnce the limit is reached, new connections will be refused", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:client_opts", - "kind" : "struct" - }, - "name" : "message_routing", - "desc" : "The message routing limiter.\nThis is used to limit the forwarding rate for this EMQX node.\nOnce the limit is reached, new publish will be refused", - "aliases" : [ - - ] - } - ], - "desc" : "Fields of the client level of the listener." - }, - { - "tags" : [ - - ], - "paths" : [ - "listeners.quic.$name.limiter", - "listeners.ssl.$name.limiter", - "listeners.tcp.$name.limiter", - "listeners.ws.$name.limiter", - "listeners.wss.$name.limiter" - ], - "full_name" : "limiter:listener_fields", - "fields" : [ - { - "type" : { - "name" : "limiter:bucket_infinity", - "kind" : "struct" - }, - "name" : "bytes_in", - "desc" : "The bytes_in limiter.\nThis is used to limit the inbound bytes rate for this EMQX node.\nOnce the limit is reached, the restricted client will be slow down even be hung for a while.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:bucket_infinity", - "kind" : "struct" - }, - "name" : "message_in", - "desc" : "The message in limiter.\nThis is used to limit the inbound message numbers for this EMQX node\nOnce the limit is reached, the restricted client will be slow down even be hung for a while.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:bucket_limit", - "kind" : "struct" - }, - "name" : "connection", - "desc" : "The connection limiter.\nThis is used to limit the connection rate for this EMQX node.\nOnce the limit is reached, new connections will be refused", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:bucket_infinity", - "kind" : "struct" - }, - "name" : "message_routing", - "desc" : "The message routing limiter.\nThis is used to limit the forwarding rate for this EMQX node.\nOnce the limit is reached, new publish will be refused", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:listener_client_fields", - "kind" : "struct" - }, - "name" : "client", - "desc" : "The rate limit for each user of the bucket", - "aliases" : [ - - ] - } - ], - "desc" : "Fields of the listener." - }, - { - "tags" : [ - - ], - "paths" : [ - "limiter.bytes_in", - "limiter.connection", - "limiter.internal", - "limiter.message_in", - "limiter.message_routing" - ], - "full_name" : "limiter:node_opts", - "fields" : [ - { - "type" : { - "name" : "emqx_limiter_schema:rate()", - "kind" : "primitive" - }, - "raw_default" : "infinity", - "name" : "rate", - "desc" : "Rate for this bucket.", - "default" : { - "oneliner" : true, - "hocon" : "\"infinity\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_limiter_schema:burst_rate()", - "kind" : "primitive" - }, - "raw_default" : "0", - "name" : "burst", - "desc" : "The burst, This value is based on rate.
\n This value + rate = the maximum limit that can be achieved when limiter burst.", - "default" : { - "oneliner" : true, - "hocon" : "\"0\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the limiter of the node level." - }, - { - "tags" : [ - - ], - "paths" : [ - "delayed" - ], - "full_name" : "modules:delayed", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable this feature", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "max_delayed_messages", - "desc" : "Maximum number of delayed messages (0 is no limit).", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the delayed module." - }, - { - "tags" : [ - - ], - "paths" : [ - "telemetry" - ], - "full_name" : "modules:telemetry", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable telemetry.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for the telemetry module." - }, - { - "tags" : [ - "Rule Engine" - ], - "paths" : [ - "rule_engine.rules.$id.actions.$INDEX" - ], - "full_name" : "rule_engine:builtin_action_console", - "fields" : [ - { - "type" : { - "name" : "console", - "kind" : "singleton" - }, - "name" : "function", - "desc" : "Print the actions to the console", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a built-in action." - }, - { - "tags" : [ - "Rule Engine" - ], - "paths" : [ - "rule_engine.rules.$id.actions.$INDEX" - ], - "full_name" : "rule_engine:builtin_action_republish", - "fields" : [ - { - "type" : { - "name" : "republish", - "kind" : "singleton" - }, - "name" : "function", - "desc" : "Republish the message as a new MQTT message", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "rule_engine:republish_args", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "args", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a built-in action." - }, - { - "tags" : [ - "Rule Engine" - ], - "paths" : [ - "rule_engine.rules.$id.actions.$INDEX.args" - ], - "full_name" : "rule_engine:republish_args", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "topic", - "examples" : [ - "a/1" - ], - "desc" : "The target topic of message to be re-published.\nTemplate with variables is allowed, see description of the 'republish_args'.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "qos()", - "kind" : "primitive" - }, - { - "name" : "binary()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "${qos}", - "name" : "qos", - "examples" : [ - "${qos}" - ], - "desc" : "The qos of the message to be re-published.\nTemplate with variables is allowed, see description of the 'republish_args'.\nDefaults to ${qos}. If variable ${qos} is not found from the selected result of the rule,\n0 is used.", - "default" : { - "oneliner" : true, - "hocon" : "\"${qos}\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "boolean()", - "kind" : "primitive" - }, - { - "name" : "binary()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "${retain}", - "name" : "retain", - "examples" : [ - "${retain}" - ], - "desc" : "The 'retain' flag of the message to be re-published.\nTemplate with variables is allowed, see description of the 'republish_args'.\nDefaults to ${retain}. If variable ${retain} is not found from the selected result\nof the rule, false is used.", - "default" : { - "oneliner" : true, - "hocon" : "\"${retain}\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "${payload}", - "name" : "payload", - "examples" : [ - "${payload}" - ], - "desc" : "The payload of the message to be re-published.\nTemplate with variables is allowed, see description of the 'republish_args'.\nDefaults to ${payload}. If variable ${payload} is not found from the selected result\nof the rule, then the string \"undefined\" is used.", - "default" : { - "oneliner" : true, - "hocon" : "\"${payload}\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "${user_properties}", - "name" : "user_properties", - "examples" : [ - "${pub_props.'User-Property'}" - ], - "desc" : "From which variable should the MQTT message's User-Property pairs be taken from.\nThe value must be a map.\nYou may configure it to ${pub_props.'User-Property'} or\nuse SELECT *,pub_props.'User-Property' as user_properties\nto forward the original user properties to the republished message.\nYou may also call map_put function like\nmap_put('my-prop-name', 'my-prop-value', user_properties) as user_properties\nto inject user properties.\nNOTE: MQTT spec allows duplicated user property names, but EMQX Rule-Engine does not.", - "default" : { - "oneliner" : true, - "hocon" : "\"${user_properties}\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "The arguments of the built-in 'republish' action.One can use variables in the args.\nThe variables are selected by the rule. For example, if the rule SQL is defined as following:\n\n SELECT clientid, qos, payload FROM \"t/1\"\n\nThen there are 3 variables available: clientid, qos and\npayload. And if we've set the args to:\n\n {\n topic = \"t/${clientid}\"\n qos = \"${qos}\"\n payload = \"msg: ${payload}\"\n }\n\nWhen the rule is triggered by an MQTT message with payload = `hello`, qos = 1,\nclientid = `Steve`, the rule will republish a new MQTT message to topic `t/Steve`,\npayload = `msg: hello`, and `qos = 1`." - }, - { - "tags" : [ - "Rule Engine" - ], - "paths" : [ - "rule_engine" - ], - "full_name" : "rule_engine", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "ignore_sys_message", - "desc" : "When set to 'true' (default), rule-engine will ignore messages published to $SYS topics.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "10s", - "name" : "jq_function_default_timeout", - "desc" : "Default timeout for the `jq` rule engine function", - "default" : { - "oneliner" : true, - "hocon" : "\"10s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "rule_engine:rules", - "kind" : "struct" - }, - "name" : "id", - "kind" : "map" - }, - "raw_default" : { - - }, - "name" : "rules", - "desc" : "The rules", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for the EMQX Rule Engine." - }, - { - "tags" : [ - "Rule Engine" - ], - "paths" : [ - "rule_engine.rules.$id" - ], - "full_name" : "rule_engine:rules", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "name", - "examples" : [ - [ - 102, - 111, - 111 - ] - ], - "desc" : "The name of the rule", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "sql", - "examples" : [ - [ - 83, - 69, - 76, - 69, - 67, - 84, - 32, - 42, - 32, - 70, - 82, - 79, - 77, - 32, - 34, - 116, - 101, - 115, - 116, - 47, - 116, - 111, - 112, - 105, - 99, - 34, - 32, - 87, - 72, - 69, - 82, - 69, - 32, - 112, - 97, - 121, - 108, - 111, - 97, - 100, - 46, - 120, - 32, - 61, - 32, - 49 - ] - ], - "desc" : "SQL query to transform the messages.\nExample: SELECT * FROM \"test/topic\" WHERE payload.x = 1", - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "members" : [ - { - "name" : "binary()", - "kind" : "primitive" - }, - { - "name" : "rule_engine:builtin_action_republish", - "kind" : "struct" - }, - { - "name" : "rule_engine:builtin_action_console", - "kind" : "struct" - }, - { - "name" : "rule_engine:user_provided_function", - "kind" : "struct" - } - ], - "kind" : "union" - } - }, - "raw_default" : [ - - ], - "name" : "actions", - "examples" : [ - [ - "webhook:my_webhook", - { - "function" : "republish", - "args" : { - "topic" : "t/1", - "payload" : "${payload}" - } - }, - { - "function" : "console" - } - ] - ], - "desc" : "A list of actions of the rule.\nAn action can be a string that refers to the channel ID of an EMQX bridge, or an object\nthat refers to a function.\nThere a some built-in functions like \"republish\" and \"console\", and we also support user\nprovided functions in the format: \"{module}:{function}\".\nThe actions in the list are executed sequentially.\nThis means that if one of the action is executing slowly, all the following actions will not\nbe executed until it returns.\nIf one of the action crashed, all other actions come after it will still be executed, in the\noriginal order.\nIf there's any error when running an action, there will be an error message, and the 'failure'\ncounter of the function action or the bridge channel will increase.", - "default" : { - "oneliner" : true, - "hocon" : "[]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable the rule", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "description", - "examples" : [ - [ - 83, - 111, - 109, - 101, - 32, - 100, - 101, - 115, - 99, - 114, - 105, - 112, - 116, - 105, - 111, - 110 - ] - ], - "desc" : "The description of the rule", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "name" : "metadata", - "desc" : "Rule metadata, do not change manually", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a rule." - }, - { - "tags" : [ - "Rule Engine" - ], - "paths" : [ - "rule_engine.rules.$id.actions.$INDEX" - ], - "full_name" : "rule_engine:user_provided_function", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "function", - "examples" : [ - [ - 109, - 111, - 100, - 117, - 108, - 101, - 58, - 102, - 117, - 110, - 99, - 116, - 105, - 111, - 110 - ] - ], - "desc" : "The user provided function. Should be in the format: '{module}:{function}'.\nWhere {module} is the Erlang callback module and {function} is the Erlang function.\n\nTo write your own function, checkout the function console and\nrepublish in the source file:\napps/emqx_rule_engine/src/emqx_rule_actions.erl as an example.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "args", - "desc" : "The args will be passed as the 3rd argument to module:function/3,\ncheckout the function console and republish in the source file:\napps/emqx_rule_engine/src/emqx_rule_actions.erl as an example.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a built-in action." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "cluster.dns" - ], - "full_name" : "cluster_dns", - "fields" : [ - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "localhost", - "name" : "name", - "desc" : "The domain name from which to discover peer EMQX nodes' IP addresses.\nApplicable when cluster.discovery_strategy = dns", - "default" : { - "oneliner" : true, - "hocon" : "\"localhost\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "a", - "srv" - ], - "kind" : "enum" - }, - "raw_default" : "a", - "name" : "record_type", - "desc" : "DNS record type.", - "default" : { - "oneliner" : true, - "hocon" : "a" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Service discovery via DNS SRV records." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "cluster.etcd" - ], - "full_name" : "cluster_etcd", - "fields" : [ - { - "type" : { - "name" : "emqx_schema:comma_separated_list()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "List of endpoint URLs of the etcd cluster", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "emqxcl", - "name" : "prefix", - "desc" : "Key prefix used for EMQX service discovery.", - "default" : { - "oneliner" : true, - "hocon" : "\"emqxcl\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "1m", - "name" : "node_ttl", - "desc" : "Expiration time of the etcd key associated with the node.\nIt is refreshed automatically, as long as the node is alive.", - "default" : { - "oneliner" : true, - "hocon" : "\"1m\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "name" : "ssl", - "desc" : "Options for the TLS connection to the etcd cluster.", - "aliases" : [ - - ] - } - ], - "desc" : "Service discovery using 'etcd' service." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "cluster.k8s" - ], - "full_name" : "cluster_k8s", - "fields" : [ - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "http://10.110.111.204:8080", - "name" : "apiserver", - "desc" : "Kubernetes API endpoint URL.", - "default" : { - "oneliner" : true, - "hocon" : "\"http://10.110.111.204:8080\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "emqx", - "name" : "service_name", - "desc" : "EMQX broker service name.", - "default" : { - "oneliner" : true, - "hocon" : "\"emqx\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "ip", - "dns", - "hostname" - ], - "kind" : "enum" - }, - "raw_default" : "ip", - "name" : "address_type", - "desc" : "Address type used for connecting to the discovered nodes.\nSetting cluster.k8s.address_type to ip will\nmake EMQX to discover IP addresses of peer nodes from Kubernetes API.", - "default" : { - "oneliner" : true, - "hocon" : "ip" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "default", - "name" : "namespace", - "desc" : "Kubernetes namespace.", - "default" : { - "oneliner" : true, - "hocon" : "\"default\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "pod.local", - "name" : "suffix", - "desc" : "Node name suffix.
\nNote: this parameter is only relevant when address_type is dns\nor hostname.", - "default" : { - "oneliner" : true, - "hocon" : "\"pod.local\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Service discovery via Kubernetes API server." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "cluster.mcast" - ], - "full_name" : "cluster_mcast", - "fields" : [ - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "239.192.0.1", - "name" : "addr", - "desc" : "Multicast IPv4 address.", - "default" : { - "oneliner" : true, - "hocon" : "\"239.192.0.1\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "integer()", - "kind" : "primitive" - } - }, - "raw_default" : [ - 4369, - 4370 - ], - "name" : "ports", - "desc" : "List of UDP ports used for service discovery.
\nNote: probe messages are broadcast to all the specified ports.", - "default" : { - "oneliner" : true, - "hocon" : "[4369,4370]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "0.0.0.0", - "name" : "iface", - "desc" : "Local IP address the node discovery service needs to bind to.", - "default" : { - "oneliner" : true, - "hocon" : "\"0.0.0.0\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "0..255", - "kind" : "primitive" - }, - "raw_default" : 255, - "name" : "ttl", - "desc" : "Time-to-live (TTL) for the outgoing UDP datagrams.", - "default" : { - "oneliner" : true, - "hocon" : "255" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "loop", - "desc" : "If true, loop UDP datagrams back to the local socket.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "16KB", - "name" : "sndbuf", - "desc" : "Size of the kernel-level buffer for outgoing datagrams.", - "default" : { - "oneliner" : true, - "hocon" : "\"16KB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "16KB", - "name" : "recbuf", - "desc" : "Size of the kernel-level buffer for incoming datagrams.", - "default" : { - "oneliner" : true, - "hocon" : "\"16KB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "32KB", - "name" : "buffer", - "desc" : "Size of the user-level buffer.", - "default" : { - "oneliner" : true, - "hocon" : "\"32KB\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Service discovery via UDP multicast." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "cluster.static" - ], - "full_name" : "cluster_static", - "fields" : [ - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "atom()", - "kind" : "primitive" - } - }, - "raw_default" : [ - - ], - "name" : "seeds", - "desc" : "List EMQX node names in the static cluster. See node.name.", - "default" : { - "oneliner" : true, - "hocon" : "[]" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Service discovery via static nodes.\nThe new node joins the cluster by connecting to one of the bootstrap nodes." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "authorization" - ], - "full_name" : "authorization", - "fields" : [ - { - "type" : { - "symbols" : [ - "allow", - "deny" - ], - "kind" : "enum" - }, - "raw_default" : "allow", - "name" : "no_match", - "desc" : "Default access control action if the user or client matches no ACL rules,\nor if no such user or client is found by the configurable authorization\nsources such as built_in_database, an HTTP API, or a query against PostgreSQL.\nFind more details in 'authorization.sources' config.", - "default" : { - "oneliner" : true, - "hocon" : "allow" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "ignore", - "disconnect" - ], - "kind" : "enum" - }, - "raw_default" : "ignore", - "name" : "deny_action", - "desc" : "The action when the authorization check rejects an operation.", - "default" : { - "oneliner" : true, - "hocon" : "ignore" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:authz_cache", - "kind" : "struct" - }, - "name" : "cache", - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "members" : [ - { - "name" : "authz:file", - "kind" : "struct" - }, - { - "name" : "authz:http_get", - "kind" : "struct" - }, - { - "name" : "authz:http_post", - "kind" : "struct" - }, - { - "name" : "authz:mnesia", - "kind" : "struct" - }, - { - "name" : "authz:mongo_single", - "kind" : "struct" - }, - { - "name" : "authz:mongo_rs", - "kind" : "struct" - }, - { - "name" : "authz:mongo_sharded", - "kind" : "struct" - }, - { - "name" : "authz:mysql", - "kind" : "struct" - }, - { - "name" : "authz:postgresql", - "kind" : "struct" - }, - { - "name" : "authz:redis_single", - "kind" : "struct" - }, - { - "name" : "authz:redis_sentinel", - "kind" : "struct" - }, - { - "name" : "authz:redis_cluster", - "kind" : "struct" - } - ], - "kind" : "union" - } - }, - "raw_default" : [ - { - "type" : "file", - "path" : "${EMQX_ETC_DIR}/acl.conf", - "enable" : true - } - ], - "name" : "sources", - "importance" : "low", - "extra" : { - "doc_lift" : true - }, - "desc" : "Authorization data sources.
\nAn array of authorization (ACL) data providers.\nIt is designed as an array, not a hash-map, so the sources can be\nordered to form a chain of access controls.
\n\nWhen authorizing a 'publish' or 'subscribe' action, the configured\nsources are checked in order. When checking an ACL source,\nin case the client (identified by username or client ID) is not found,\nit moves on to the next source. And it stops immediately\nonce an 'allow' or 'deny' decision is returned.
\n\nIf the client is not found in any of the sources,\nthe default action configured in 'authorization.no_match' is applied.
\n\nNOTE:\nThe source elements are identified by their 'type'.\nIt is NOT allowed to configure two or more sources of the same type.", - "default" : { - "oneliner" : false, - "hocon" : "[\n {\n enable = true\n path = \"${EMQX_ETC_DIR}/acl.conf\"\n type = \"file\"\n }\n]\n" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings that control client authorization." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "cluster" - ], - "full_name" : "cluster", - "fields" : [ - { - "type" : { - "name" : "atom()", - "kind" : "primitive" - }, - "raw_default" : "emqxcl", - "name" : "name", - "desc" : "Human-friendly name of the EMQX cluster.", - "default" : { - "oneliner" : true, - "hocon" : "emqxcl" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "manual", - "static", - "mcast", - "dns", - "etcd", - "k8s" - ], - "kind" : "enum" - }, - "raw_default" : "manual", - "name" : "discovery_strategy", - "desc" : "Service discovery method for the cluster nodes. Possible values are:\n- manual: Use emqx ctl cluster command to manage cluster.
\n- static: Configure static nodes list by setting seeds in config file.
\n- dns: Use DNS A record to discover peer nodes.
\n- etcd: Use etcd to discover peer nodes.
\n- k8s: Use Kubernetes API to discover peer pods.", - "default" : { - "oneliner" : true, - "hocon" : "manual" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:comma_separated_atoms()", - "kind" : "primitive" - }, - "raw_default" : [ - - ], - "name" : "core_nodes", - "desc" : "List of core nodes that the replicant will connect to.
\nNote: this parameter only takes effect when the backend is set\nto rlog and the role is set to replicant.
\nThis value needs to be defined for manual or static cluster discovery mechanisms.
\nIf an automatic cluster discovery mechanism is being used (such as etcd),\nthere is no need to set this value.", - "default" : { - "oneliner" : true, - "hocon" : "[]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5m", - "name" : "autoclean", - "desc" : "Remove disconnected nodes from the cluster after this interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"5m\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "autoheal", - "desc" : "If true, the node will try to heal network partitions automatically.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "inet_tcp", - "inet6_tcp", - "inet_tls" - ], - "kind" : "enum" - }, - "raw_default" : "inet_tcp", - "name" : "proto_dist", - "desc" : "The Erlang distribution protocol for the cluster.
\n- inet_tcp: IPv4 TCP
\n- inet_tls: IPv4 TLS, works together with etc/ssl_dist.conf", - "default" : { - "oneliner" : true, - "hocon" : "inet_tcp" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "cluster_static", - "kind" : "struct" - }, - "name" : "static", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "cluster_mcast", - "kind" : "struct" - }, - "name" : "mcast", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "cluster_dns", - "kind" : "struct" - }, - "name" : "dns", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "cluster_etcd", - "kind" : "struct" - }, - "name" : "etcd", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "cluster_k8s", - "kind" : "struct" - }, - "name" : "k8s", - "aliases" : [ - - ] - } - ], - "desc" : "EMQX nodes can form a cluster to scale up the total capacity.
\n Here holds the configs to instruct how individual nodes can discover each other." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "log.console_handler" - ], - "full_name" : "console_handler", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "enable", - "importance" : "low", - "desc" : "Enable this log handler.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_conf_schema:log_level()", - "kind" : "primitive" - }, - "raw_default" : "warning", - "name" : "level", - "importance" : "low", - "desc" : "The log level for the current log handler.\nDefaults to warning.", - "default" : { - "oneliner" : true, - "hocon" : "warning" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "system", - "name" : "time_offset", - "importance" : "low", - "desc" : "The time offset to be used when formatting the timestamp.\nCan be one of:\n - system: the time offset used by the local system\n - utc: the UTC time offset\n - +-[hh]:[mm]: user specified time offset, such as \"-02:00\" or \"+00:00\"\nDefaults to: system.", - "default" : { - "oneliner" : true, - "hocon" : "\"system\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "unlimited", - "kind" : "singleton" - }, - { - "name" : "100..inf", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "unlimited", - "name" : "chars_limit", - "importance" : "low", - "desc" : "Set the maximum length of a single log message. If this length is exceeded, the log message will be truncated.\nNOTE: Restrict char limiter if formatter is JSON , it will get a truncated incomplete JSON data, which is not recommended.", - "default" : { - "oneliner" : true, - "hocon" : "unlimited" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "text", - "json" - ], - "kind" : "enum" - }, - "raw_default" : "text", - "name" : "formatter", - "importance" : "medium", - "desc" : "Choose log formatter. text for free text, and json for structured logging.", - "default" : { - "oneliner" : true, - "hocon" : "text" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "single_line", - "importance" : "low", - "desc" : "Print logs in a single line if set to true. Otherwise, log messages may span multiple lines.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "sync_mode_qlen", - "importance" : "low", - "desc" : "As long as the number of buffered log events is lower than this value,\nall log events are handled asynchronously. This means that the client process sending the log event,\nby calling a log function in the Logger API, does not wait for a response from the handler\nbut continues executing immediately after the event is sent.\nIt is not affected by the time it takes the handler to print the event to the log device.\nIf the message queue grows larger than this value,\nthe handler starts handling log events synchronously instead,\nmeaning that the client process sending the event must wait for a response.\nWhen the handler reduces the message queue to a level below the sync_mode_qlen threshold,\nasynchronous operation is resumed.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 3000, - "name" : "drop_mode_qlen", - "importance" : "low", - "desc" : "When the number of buffered log events is larger than this value, the new log events are dropped.\nWhen drop mode is activated or deactivated, a message is printed in the logs.", - "default" : { - "oneliner" : true, - "hocon" : "3000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8000, - "name" : "flush_qlen", - "importance" : "low", - "desc" : "If the number of buffered log events grows larger than this threshold, a flush (delete) operation takes place.\nTo flush events, the handler discards the buffered log messages without logging.", - "default" : { - "oneliner" : true, - "hocon" : "8000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "log_overload_kill", - "kind" : "struct" - }, - "name" : "overload_kill", - "importance" : "low", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "log_burst_limit", - "kind" : "struct" - }, - "name" : "burst_limit", - "importance" : "low", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "error", - "progress" - ], - "kind" : "enum" - }, - "raw_default" : "error", - "name" : "supervisor_reports", - "importance" : "low", - "desc" : "Type of supervisor reports that are logged. Defaults to error
\n - error: only log errors in the Erlang processes
.\n - progress: log process startup.", - "default" : { - "oneliner" : true, - "hocon" : "error" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "unlimited", - "kind" : "singleton" - }, - { - "name" : "non_neg_integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 100, - "name" : "max_depth", - "importance" : "low", - "desc" : "Maximum depth for Erlang term log formatting and Erlang process message queue inspection.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Log handler that prints log events to the EMQX console." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "log" - ], - "full_name" : "log", - "fields" : [ - { - "type" : { - "name" : "console_handler", - "kind" : "struct" - }, - "name" : "console_handler", - "importance" : "high", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "log_file_handler", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "raw_default" : { - "default" : { - "level" : "warning" - } - }, - "name" : "file_handlers", - "importance" : "high", - "desc" : "File-based log handlers.", - "default" : { - "oneliner" : false, - "hocon" : "{\n default {level = \"warning\"}\n}\n" - }, - "aliases" : [ - - ] - } - ], - "desc" : "EMQX supports multiple log handlers, one console handler and multiple file handlers.\nEMQX by default logs to console when running in docker or in console/foreground mode,\notherwise it logs to file $EMQX_LOG_DIR/emqx.log.\nFor advanced configuration, you can find more parameters in this section." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "log.console_handler.burst_limit", - "log.file_handlers.$name.burst_limit" - ], - "full_name" : "log_burst_limit", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable log burst control feature.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 10000, - "name" : "max_count", - "desc" : "Maximum number of log events to handle within a `window_time` interval. After the limit is reached, successive events are dropped until the end of the `window_time`.", - "default" : { - "oneliner" : true, - "hocon" : "10000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "1s", - "name" : "window_time", - "desc" : "See max_count.", - "default" : { - "oneliner" : true, - "hocon" : "\"1s\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Large bursts of log events produced in a short time can potentially cause problems, such as:\n - Log files grow very large\n - Log files are rotated too quickly, and useful information gets overwritten\n - Overall performance impact on the system\n\nLog burst limit feature can temporarily disable logging to avoid these issues." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "log.file_handlers.$name" - ], - "full_name" : "log_file_handler", - "fields" : [ - { - "type" : { - "name" : "emqx_conf_schema:file()", - "kind" : "primitive" - }, - "raw_default" : "${EMQX_LOG_DIR}/emqx.log", - "name" : "file", - "desc" : "Name the log file.", - "default" : { - "oneliner" : true, - "hocon" : "\"${EMQX_LOG_DIR}/emqx.log\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "log_rotation", - "kind" : "struct" - }, - "name" : "rotation", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "50MB", - "name" : "max_size", - "importance" : "medium", - "desc" : "This parameter controls log file rotation. The value `infinity` means the log file will grow indefinitely, otherwise the log file will be rotated once it reaches `max_size` in bytes.", - "default" : { - "oneliner" : true, - "hocon" : "\"50MB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "importance" : "low", - "desc" : "Enable this log handler.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_conf_schema:log_level()", - "kind" : "primitive" - }, - "raw_default" : "warning", - "name" : "level", - "importance" : "low", - "desc" : "The log level for the current log handler.\nDefaults to warning.", - "default" : { - "oneliner" : true, - "hocon" : "warning" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "system", - "name" : "time_offset", - "importance" : "low", - "desc" : "The time offset to be used when formatting the timestamp.\nCan be one of:\n - system: the time offset used by the local system\n - utc: the UTC time offset\n - +-[hh]:[mm]: user specified time offset, such as \"-02:00\" or \"+00:00\"\nDefaults to: system.", - "default" : { - "oneliner" : true, - "hocon" : "\"system\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "unlimited", - "kind" : "singleton" - }, - { - "name" : "100..inf", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "unlimited", - "name" : "chars_limit", - "importance" : "low", - "desc" : "Set the maximum length of a single log message. If this length is exceeded, the log message will be truncated.\nNOTE: Restrict char limiter if formatter is JSON , it will get a truncated incomplete JSON data, which is not recommended.", - "default" : { - "oneliner" : true, - "hocon" : "unlimited" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "text", - "json" - ], - "kind" : "enum" - }, - "raw_default" : "text", - "name" : "formatter", - "importance" : "medium", - "desc" : "Choose log formatter. text for free text, and json for structured logging.", - "default" : { - "oneliner" : true, - "hocon" : "text" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "single_line", - "importance" : "low", - "desc" : "Print logs in a single line if set to true. Otherwise, log messages may span multiple lines.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "sync_mode_qlen", - "importance" : "low", - "desc" : "As long as the number of buffered log events is lower than this value,\nall log events are handled asynchronously. This means that the client process sending the log event,\nby calling a log function in the Logger API, does not wait for a response from the handler\nbut continues executing immediately after the event is sent.\nIt is not affected by the time it takes the handler to print the event to the log device.\nIf the message queue grows larger than this value,\nthe handler starts handling log events synchronously instead,\nmeaning that the client process sending the event must wait for a response.\nWhen the handler reduces the message queue to a level below the sync_mode_qlen threshold,\nasynchronous operation is resumed.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 3000, - "name" : "drop_mode_qlen", - "importance" : "low", - "desc" : "When the number of buffered log events is larger than this value, the new log events are dropped.\nWhen drop mode is activated or deactivated, a message is printed in the logs.", - "default" : { - "oneliner" : true, - "hocon" : "3000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8000, - "name" : "flush_qlen", - "importance" : "low", - "desc" : "If the number of buffered log events grows larger than this threshold, a flush (delete) operation takes place.\nTo flush events, the handler discards the buffered log messages without logging.", - "default" : { - "oneliner" : true, - "hocon" : "8000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "log_overload_kill", - "kind" : "struct" - }, - "name" : "overload_kill", - "importance" : "low", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "log_burst_limit", - "kind" : "struct" - }, - "name" : "burst_limit", - "importance" : "low", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "error", - "progress" - ], - "kind" : "enum" - }, - "raw_default" : "error", - "name" : "supervisor_reports", - "importance" : "low", - "desc" : "Type of supervisor reports that are logged. Defaults to error
\n - error: only log errors in the Erlang processes
.\n - progress: log process startup.", - "default" : { - "oneliner" : true, - "hocon" : "error" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "unlimited", - "kind" : "singleton" - }, - { - "name" : "non_neg_integer()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 100, - "name" : "max_depth", - "importance" : "low", - "desc" : "Maximum depth for Erlang term log formatting and Erlang process message queue inspection.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Log handler that prints log events to files." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "log.console_handler.overload_kill", - "log.file_handlers.$name.overload_kill" - ], - "full_name" : "log_overload_kill", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable log handler overload kill feature.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "30MB", - "name" : "mem_size", - "desc" : "Maximum memory size that the log handler process is allowed to use.", - "default" : { - "oneliner" : true, - "hocon" : "\"30MB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 20000, - "name" : "qlen", - "desc" : "Maximum allowed queue length.", - "default" : { - "oneliner" : true, - "hocon" : "20000" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - { - "name" : "infinity", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "raw_default" : "5s", - "name" : "restart_after", - "desc" : "The handler restarts automatically after a delay in the event of termination, unless the value `infinity` is set, which blocks any subsequent restarts.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Log overload kill features an overload protection that activates when the log handlers use too much memory or have too many buffered log messages.
\nWhen the overload is detected, the log handler is terminated and restarted after a cooldown period." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "log.file_handlers.$name.rotation" - ], - "full_name" : "log_rotation", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable log rotation feature.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1..2048", - "kind" : "primitive" - }, - "raw_default" : 10, - "name" : "count", - "desc" : "Maximum number of log files.", - "default" : { - "oneliner" : true, - "hocon" : "10" - }, - "aliases" : [ - - ] - } - ], - "desc" : "By default, the logs are stored in `./log` directory (for installation from zip file) or in `/var/log/emqx` (for binary installation).
\nThis section of the configuration controls the number of files kept for each log handler." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "node" - ], - "full_name" : "node", - "fields" : [ - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "emqx@127.0.0.1", - "name" : "name", - "importance" : "high", - "desc" : "Unique name of the EMQX node. It must follow %name%@FQDN or\n%name%@IPv4 format.", - "default" : { - "oneliner" : true, - "hocon" : "\"emqx@127.0.0.1\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "cookie", - "importance" : "high", - "desc" : "Secret cookie is a random string that should be the same on all nodes in\nthe given EMQX cluster, but unique per EMQX cluster. It is used to prevent EMQX nodes that\nbelong to different clusters from accidentally connecting to each other.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1024..134217727", - "kind" : "primitive" - }, - "raw_default" : 2097152, - "name" : "process_limit", - "importance" : "medium", - "desc" : "Maximum number of simultaneously existing processes for this Erlang system.\nThe actual maximum chosen may be much larger than the Number passed.\nFor more information, see: https://www.erlang.org/doc/man/erl.html", - "default" : { - "oneliner" : true, - "hocon" : "2097152" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1024..134217727", - "kind" : "primitive" - }, - "raw_default" : 1048576, - "name" : "max_ports", - "importance" : "high", - "desc" : "Maximum number of simultaneously existing ports for this Erlang system.\nThe actual maximum chosen may be much larger than the Number passed.\nFor more information, see: https://www.erlang.org/doc/man/erl.html", - "default" : { - "oneliner" : true, - "hocon" : "1048576" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1..2097151", - "kind" : "primitive" - }, - "raw_default" : 8192, - "name" : "dist_buffer_size", - "importance" : "low", - "desc" : "Erlang's distribution buffer busy limit in kilobytes.", - "default" : { - "oneliner" : true, - "hocon" : "8192" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "data_dir", - "importance" : "low", - "desc" : "Path to the persistent data directory.
\nPossible auto-created subdirectories are:
\n- `mnesia/`: EMQX's built-in database directory.
\nFor example, `mnesia/emqx@127.0.0.1`.
\nThere should be only one such subdirectory.
\nMeaning, in case the node is to be renamed (to e.g. `emqx@10.0.1.1`),
\nthe old dir should be deleted first.
\n- `configs`: Generated configs at boot time, and cluster/local override configs.
\n- `patches`: Hot-patch beam files are to be placed here.
\n- `trace`: Trace log files.
\n\n**NOTE**: One data dir cannot be shared by two or more EMQX nodes.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "disabled", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "15m", - "name" : "global_gc_interval", - "importance" : "low", - "desc" : "Periodic garbage collection interval. Set to disabled to have it disabled.", - "default" : { - "oneliner" : true, - "hocon" : "\"15m\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "core", - "replicant" - ], - "kind" : "enum" - }, - "raw_default" : "core", - "name" : "role", - "importance" : "high", - "desc" : "Select a node role.
\ncore nodes provide durability of the data, and take care of writes.\nIt is recommended to place core nodes in different racks or different availability zones.
\nreplicant nodes are ephemeral worker nodes. Removing them from the cluster\ndoesn't affect database redundancy
\nIt is recommended to have more replicant nodes than core nodes.
\nNote: this parameter only takes effect when the backend is set\nto rlog.", - "default" : { - "oneliner" : true, - "hocon" : "core" - }, - "aliases" : [ - "db_role" - ] - } - ], - "desc" : "Node name, cookie, config & data directories and the Erlang virtual machine (BEAM) boot parameters." - }, - { - "tags" : [ - "EMQX" - ], - "paths" : [ - "rpc" - ], - "full_name" : "rpc", - "fields" : [ - { - "type" : { - "symbols" : [ - "sync", - "async" - ], - "kind" : "enum" - }, - "raw_default" : "async", - "name" : "mode", - "desc" : "In sync mode the sending side waits for the ack from the receiving side.", - "default" : { - "oneliner" : true, - "hocon" : "async" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "tcp", - "ssl" - ], - "kind" : "enum" - }, - "raw_default" : "tcp", - "name" : "driver", - "desc" : "Transport protocol used for inter-broker communication", - "default" : { - "oneliner" : true, - "hocon" : "tcp" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 256, - "name" : "async_batch_size", - "desc" : "The maximum number of batch messages sent in asynchronous mode.\n Note that this configuration does not work in synchronous mode.", - "default" : { - "oneliner" : true, - "hocon" : "256" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "manual", - "stateless" - ], - "kind" : "enum" - }, - "raw_default" : "stateless", - "name" : "port_discovery", - "desc" : "manual: discover ports by tcp_server_port.
\nstateless: discover ports in a stateless manner, using the following algorithm.\nIf node name is emqxN@127.0.0.1, where the N is an integer,\nthen the listening port will be 5370 + N.", - "default" : { - "oneliner" : true, - "hocon" : "stateless" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 5369, - "name" : "tcp_server_port", - "desc" : "Listening port used by RPC local service.
\nNote that this config only takes effect when rpc.port_discovery is set to manual.", - "default" : { - "oneliner" : true, - "hocon" : "5369" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 5369, - "name" : "ssl_server_port", - "desc" : "Listening port used by RPC local service.
\nNote that this config only takes effect when rpc.port_discovery is set to manual\nand driver is set to ssl.", - "default" : { - "oneliner" : true, - "hocon" : "5369" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1..256", - "kind" : "primitive" - }, - "raw_default" : 10, - "name" : "tcp_client_num", - "desc" : "Set the maximum number of RPC communication channels initiated by this node to each remote node.", - "default" : { - "oneliner" : true, - "hocon" : "10" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "connect_timeout", - "desc" : "Timeout for establishing an RPC connection.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_conf_schema:file()", - "kind" : "primitive" - }, - "name" : "certfile", - "desc" : "Path to TLS certificate file used to validate identity of the cluster nodes.\nNote that this config only takes effect when rpc.driver is set to ssl.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_conf_schema:file()", - "kind" : "primitive" - }, - "name" : "keyfile", - "desc" : "Path to the private key file for the rpc.certfile.
\nNote: contents of this file are secret, so it's necessary to set permissions to 600.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_conf_schema:file()", - "kind" : "primitive" - }, - "name" : "cacertfile", - "desc" : "Path to certification authority TLS certificate file used to validate rpc.certfile.
\nNote: certificates of all nodes in the cluster must be signed by the same CA.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "send_timeout", - "desc" : "Timeout for sending the RPC request.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "authentication_timeout", - "desc" : "Timeout for the remote node authentication.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "call_receive_timeout", - "desc" : "Timeout for the reply to a synchronous RPC.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_s()", - "kind" : "primitive" - }, - "raw_default" : "15m", - "name" : "socket_keepalive_idle", - "desc" : "How long the connections between the brokers should remain open after the last message is sent.", - "default" : { - "oneliner" : true, - "hocon" : "\"15m\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_s()", - "kind" : "primitive" - }, - "raw_default" : "75s", - "name" : "socket_keepalive_interval", - "desc" : "The interval between keepalive messages.", - "default" : { - "oneliner" : true, - "hocon" : "\"75s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 9, - "name" : "socket_keepalive_count", - "desc" : "How many times the keepalive probe message can fail to receive a reply\nuntil the RPC connection is considered lost.", - "default" : { - "oneliner" : true, - "hocon" : "9" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "1MB", - "name" : "socket_sndbuf", - "desc" : "TCP tuning parameters. TCP sending buffer size.", - "default" : { - "oneliner" : true, - "hocon" : "\"1MB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "1MB", - "name" : "socket_recbuf", - "desc" : "TCP tuning parameters. TCP receiving buffer size.", - "default" : { - "oneliner" : true, - "hocon" : "\"1MB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "1MB", - "name" : "socket_buffer", - "desc" : "TCP tuning parameters. Socket buffer size in user mode.", - "default" : { - "oneliner" : true, - "hocon" : "\"1MB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "insecure_fallback", - "desc" : "Enable compatibility with old RPC authentication.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "EMQX uses a library called gen_rpc for inter-broker communication.
\nMost of the time the default config should work,\nbut in case you need to do performance fine-tuning or experiment a bit,\nthis is where to look." - }, - { - "tags" : [ - - ], - "paths" : [ - "authentication.$INDEX.topology", - "authorization.sources.$INDEX.topology", - "bridges.mongodb_rs.$name.topology", - "bridges.mongodb_sharded.$name.topology", - "bridges.mongodb_single.$name.topology" - ], - "full_name" : "topology", - "fields" : [ - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "max_overflow", - "desc" : "Max Overflow.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "name" : "overflow_ttl", - "desc" : "Period of time before workers that exceed the configured pool size (\"overflow\") to be terminated.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "name" : "overflow_check_period", - "desc" : "Period for checking if there are more workers than configured (\"overflow\").", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "name" : "local_threshold_ms", - "desc" : "The size of the latency window for selecting among multiple suitable MongoDB instances.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "name" : "connect_timeout_ms", - "desc" : "The duration to attempt a connection before timing out.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "name" : "socket_timeout_ms", - "desc" : "The duration to attempt to send or to receive on a socket before the attempt times out.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "name" : "server_selection_timeout_ms", - "desc" : "Specifies how long to block for server selection before throwing an exception.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "name" : "wait_queue_timeout_ms", - "desc" : "The maximum duration that a worker can wait for a connection to become available.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "200s", - "name" : "heartbeat_frequency_ms", - "desc" : "Controls when the driver checks the state of the MongoDB deployment. Specify the interval between checks, counted from the end of the previous check until the beginning of the next one. If the number of connections is increased (which will happen, for example, if you increase the pool size), you may need to increase this period as well to avoid creating too many log entries in the MongoDB log file.", - "default" : { - "oneliner" : true, - "hocon" : "\"200s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "name" : "min_heartbeat_frequency_ms", - "desc" : "Controls the minimum amount of time to wait between heartbeats.", - "aliases" : [ - - ] - } - ], - "desc" : "Topology of MongoDB." - }, - { - "tags" : [ - "Schema Registry" - ], - "paths" : [ - "schema_registry.schemas.$name" - ], - "full_name" : "avro", - "fields" : [ - { - "type" : { - "name" : "avro", - "kind" : "singleton" - }, - "name" : "type", - "desc" : "Schema type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:json_binary()", - "kind" : "primitive" - }, - "name" : "source", - "desc" : "Source text for the schema.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "description", - "desc" : "A description for this schema.", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "[Apache Avro](https://avro.apache.org/) serialization format." - }, - { - "tags" : [ - "Schema Registry" - ], - "paths" : [ - "schema_registry" - ], - "full_name" : "schema_registry", - "fields" : [ - { - "type" : { - "values" : { - "members" : [ - { - "name" : "avro", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "name" : "name", - "kind" : "map" - }, - "raw_default" : { - - }, - "name" : "schemas", - "desc" : "Registered schemas.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Schema registry configurations." - }, - { - "tags" : [ - "License" - ], - "paths" : [ - "license" - ], - "full_name" : "key_license", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "MjIwMTExCjAKMTAKRXZhbHVhdGlvbgpjb250YWN0QGVtcXguaW8KZGVmYXVsdAoyMDIzMDEwOQoxODI1CjEwMAo=.MEUCIG62t8W15g05f1cKx3tA3YgJoR0dmyHOPCdbUxBGxgKKAiEAhHKh8dUwhU+OxNEaOn8mgRDtiT3R8RZooqy6dEsOmDI=", - "name" : "key", - "desc" : "License string", - "default" : { - "oneliner" : true, - "hocon" : "\"MjIwMTExCjAKMTAKRXZhbHVhdGlvbgpjb250YWN0QGVtcXguaW8KZGVmYXVsdAoyMDIzMDEwOQoxODI1CjEwMAo=.MEUCIG62t8W15g05f1cKx3tA3YgJoR0dmyHOPCdbUxBGxgKKAiEAhHKh8dUwhU+OxNEaOn8mgRDtiT3R8RZooqy6dEsOmDI=\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:percent()", - "kind" : "primitive" - }, - "raw_default" : "75%", - "name" : "connection_low_watermark", - "desc" : "Low watermark limit below which license connection quota usage alarms are deactivated", - "default" : { - "oneliner" : true, - "hocon" : "\"75%\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:percent()", - "kind" : "primitive" - }, - "raw_default" : "80%", - "name" : "connection_high_watermark", - "desc" : "High watermark limit above which license connection quota usage alarms are activated", - "default" : { - "oneliner" : true, - "hocon" : "\"80%\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "License provisioned as a string." - }, - { - "tags" : [ - - ], - "paths" : [ - "zones.$name.force_gc" - ], - "full_name" : "zone:force_gc", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable", - "desc" : "Enable forced garbage collection.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "0..inf", - "kind" : "primitive" - }, - "name" : "count", - "desc" : "GC the process after this many received messages.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "name" : "bytes", - "desc" : "GC the process after specified number of bytes have passed through.", - "aliases" : [ - - ] - } - ], - "desc" : "Force garbage collection in MQTT connection process after\n they process certain number of messages or bytes of data." - }, - { - "tags" : [ - - ], - "paths" : [ - "zones.$name.force_shutdown" - ], - "full_name" : "zone:force_shutdown", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable", - "desc" : "Enable `force_shutdown` feature.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "0..inf", - "kind" : "primitive" - }, - "name" : "max_message_queue_len", - "desc" : "Maximum message queue length.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:wordsize()", - "kind" : "primitive" - }, - "name" : "max_heap_size", - "desc" : "Total heap size", - "aliases" : [ - - ] - } - ], - "desc" : "When the process message queue length, or the memory bytes\nreaches a certain value, the process is forced to close.\n\nNote: \"message queue\" here refers to the \"message mailbox\"\nof the Erlang process, not the `mqueue` of QoS 1 and QoS 2." - }, - { - "tags" : [ - - ], - "paths" : [ - "zones.$name.mqtt" - ], - "full_name" : "zone:mqtt", - "fields" : [ - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "name" : "idle_timeout", - "desc" : "Configure the duration of time that a connection can remain idle (i.e., without any data transfer) before being:\n - Automatically disconnected if no CONNECT package is received from the client yet.\n - Put into hibernation mode to save resources if some CONNECT packages are already received.\nNote: Please set the parameter with caution as long idle time will lead to resource waste.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "name" : "max_packet_size", - "desc" : "Maximum MQTT packet size allowed.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "23..65535", - "kind" : "primitive" - }, - "name" : "max_clientid_len", - "desc" : "Maximum allowed length of MQTT Client ID.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1..65535", - "kind" : "primitive" - }, - "name" : "max_topic_levels", - "desc" : "Maximum topic levels allowed.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "qos()", - "kind" : "primitive" - }, - "name" : "max_qos_allowed", - "desc" : "Maximum QoS allowed.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "0..65535", - "kind" : "primitive" - }, - "name" : "max_topic_alias", - "desc" : "Maximum topic alias, 0 means no topic alias supported.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "retain_available", - "desc" : "Whether to enable support for MQTT retained message.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "wildcard_subscription", - "desc" : "Whether to enable support for MQTT wildcard subscription.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "shared_subscription", - "desc" : "Whether to enable support for MQTT shared subscription.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "exclusive_subscription", - "desc" : "Whether to enable support for MQTT exclusive subscription.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "ignore_loop_deliver", - "desc" : "Whether the messages sent by the MQTT v3.1.1/v3.1.0 client will be looped back to the publisher itself, similar to No Local in MQTT 5.0.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "strict_mode", - "desc" : "Whether to parse MQTT messages in strict mode.\nIn strict mode, invalid utf8 strings in for example client ID, topic name, etc. will cause the client to be disconnected.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "response_information", - "desc" : "UTF-8 string, for creating the response topic, for example, if set to reqrsp/, the publisher/subscriber will communicate using the topic prefix reqrsp/.\nTo disable this feature, input \"\" in the text box below. Only applicable to MQTT 5.0 clients.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "integer()", - "kind" : "primitive" - }, - { - "name" : "disabled", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "name" : "server_keepalive", - "desc" : "The keep alive duration required by EMQX. To use the setting from the client side, choose disabled from the drop-down list. Only applicable to MQTT 5.0 clients.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "number()", - "kind" : "primitive" - }, - "name" : "keepalive_backoff", - "desc" : "The coefficient EMQX uses to confirm whether the keep alive duration of the client expires. Formula: Keep Alive * Backoff * 2", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "1..inf", - "kind" : "primitive" - }, - { - "name" : "infinity", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "name" : "max_subscriptions", - "desc" : "Maximum number of subscriptions allowed per client.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "upgrade_qos", - "desc" : "Force upgrade of QoS level according to subscription.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "1..65535", - "kind" : "primitive" - }, - "name" : "max_inflight", - "desc" : "Maximum number of QoS 1 and QoS 2 messages that are allowed to be delivered simultaneously before completing the acknowledgment.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "name" : "retry_interval", - "desc" : "Retry interval for QoS 1/2 message delivering.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "integer()", - "kind" : "primitive" - }, - { - "name" : "infinity", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "name" : "max_awaiting_rel", - "desc" : "For each publisher session, the maximum number of outstanding QoS 2 messages pending on the client to send PUBREL. After reaching this limit, new QoS 2 PUBLISH requests will be rejected with `147(0x93)` until either PUBREL is received or timed out.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "name" : "await_rel_timeout", - "desc" : "For client to broker QoS 2 message, the time limit for the broker to wait before the `PUBREL` message is received. The wait is aborted after timed out, meaning the packet ID is freed for new `PUBLISH` requests. Receiving a stale `PUBREL` causes a warning level log. Note, the message is delivered to subscribers before entering the wait for PUBREL.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "name" : "session_expiry_interval", - "desc" : "Specifies how long the session will expire after the connection is disconnected, only for non-MQTT 5.0 connections.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - { - "name" : "infinity", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "name" : "max_mqueue_len", - "desc" : "Maximum queue length. Enqueued messages when persistent client disconnected, or inflight window is full.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "map()", - "kind" : "primitive" - }, - { - "name" : "disabled", - "kind" : "singleton" - } - ], - "kind" : "union" - }, - "name" : "mqueue_priorities", - "desc" : "Topic priorities. Priority number [1-255]\nThere's no priority table by default, hence all messages are treated equal.\n\n**NOTE**: Comma and equal signs are not allowed for priority topic names.\n**NOTE**: Messages for topics not in the priority table are treated as either highest or lowest priority depending on the configured value for mqtt.mqueue_default_priority.\n\n**Examples**:\nTo configure \"topic/1\" > \"topic/2\":\nmqueue_priorities: {\"topic/1\": 10, \"topic/2\": 8}", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "highest", - "lowest" - ], - "kind" : "enum" - }, - "name" : "mqueue_default_priority", - "desc" : "Default topic priority, which will be used by topics not in Topic Priorities (mqueue_priorities).", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "mqueue_store_qos0", - "desc" : "Specifies whether to store QoS 0 messages in the message queue while the connection is down but the session remains.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "use_username_as_clientid", - "desc" : "Whether to use Username as Client ID.\nThis setting takes effect later than Use Peer Certificate as Username and Use peer certificate as Client ID.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "disabled", - "cn", - "dn", - "crt", - "pem", - "md5" - ], - "kind" : "enum" - }, - "name" : "peer_cert_as_username", - "desc" : "Use the CN, DN field in the peer certificate or the entire certificate content as Username. Only works for the TLS connection.\nSupported configurations are the following:\n- cn: CN field of the certificate\n- dn: DN field of the certificate\n- crt: Content of the DER or PEM certificate\n- pem: Convert DER certificate content to PEM format and use as Username\n- md5: MD5 value of the DER or PEM certificate", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "disabled", - "cn", - "dn", - "crt", - "pem", - "md5" - ], - "kind" : "enum" - }, - "name" : "peer_cert_as_clientid", - "desc" : "Use the CN, DN field in the peer certificate or the entire certificate content as Client ID. Only works for the TLS connection.\nSupported configurations are the following:\n- cn: CN field of the certificate\n- dn: DN field of the certificate\n- crt: DER or PEM certificate\n- pem: Convert DER certificate content to PEM format and use as Client ID\n- md5: MD5 value of the DER or PEM certificate", - "aliases" : [ - - ] - } - ], - "desc" : "Global MQTT configuration.
The configs here work as default values which can be overridden\nin zone configs" - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-builtin_db:authentication", - "fields" : [ - { - "type" : { - "name" : "password_based", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "built_in_database", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "clientid", - "username" - ], - "kind" : "enum" - }, - "raw_default" : "username", - "name" : "user_id_type", - "desc" : "Specify whether to use `clientid` or `username` for authentication.", - "default" : { - "oneliner" : true, - "hocon" : "\"username\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "authn-hash:bcrypt_rw", - "kind" : "struct" - }, - { - "name" : "authn-hash:pbkdf2", - "kind" : "struct" - }, - { - "name" : "authn-hash:simple", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "raw_default" : { - "salt_position" : "prefix", - "name" : "sha256" - }, - "name" : "password_hash_algorithm", - "desc" : "Options for password hash creation and verification.", - "default" : { - "oneliner" : true, - "hocon" : "{name = sha256, salt_position = prefix}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of authenticator using built-in database as data source." - }, - { - "tags" : [ - - ], - "paths" : [ - "authentication.$INDEX.password_hash_algorithm" - ], - "full_name" : "authn-hash:bcrypt", - "fields" : [ - { - "type" : { - "name" : "bcrypt", - "kind" : "singleton" - }, - "name" : "name", - "desc" : "BCRYPT password hashing.", - "aliases" : [ - - ] - } - ], - "desc" : "Settings for bcrypt password hashing algorithm." - }, - { - "tags" : [ - - ], - "paths" : [ - "authentication.$INDEX.password_hash_algorithm" - ], - "full_name" : "authn-hash:bcrypt_rw", - "fields" : [ - { - "type" : { - "name" : "bcrypt", - "kind" : "singleton" - }, - "name" : "name", - "desc" : "BCRYPT password hashing.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 10, - "name" : "salt_rounds", - "examples" : [ - 10 - ], - "desc" : "Salt rounds for BCRYPT password generation.", - "default" : { - "oneliner" : true, - "hocon" : "10" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for bcrypt password hashing algorithm (for DB backends with write capability)." - }, - { - "tags" : [ - - ], - "paths" : [ - "authentication.$INDEX.password_hash_algorithm" - ], - "full_name" : "authn-hash:pbkdf2", - "fields" : [ - { - "type" : { - "name" : "pbkdf2", - "kind" : "singleton" - }, - "name" : "name", - "desc" : "PBKDF2 password hashing.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "md4", - "md5", - "ripemd160", - "sha", - "sha224", - "sha256", - "sha384", - "sha512" - ], - "kind" : "enum" - }, - "name" : "mac_fun", - "desc" : "Specifies mac_fun for PBKDF2 hashing algorithm.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "name" : "iterations", - "desc" : "Iteration count for PBKDF2 hashing algorithm.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "name" : "dk_length", - "desc" : "Derived length for PBKDF2 hashing algorithm. If not specified, calculated automatically based on `mac_fun`.", - "aliases" : [ - - ] - } - ], - "desc" : "Settings for PBKDF2 password hashing algorithm." - }, - { - "tags" : [ - - ], - "paths" : [ - "authentication.$INDEX.password_hash_algorithm" - ], - "full_name" : "authn-hash:simple", - "fields" : [ - { - "type" : { - "symbols" : [ - "plain", - "md5", - "sha", - "sha256", - "sha512" - ], - "kind" : "enum" - }, - "name" : "name", - "desc" : "Simple password hashing algorithm.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "disable", - "prefix", - "suffix" - ], - "kind" : "enum" - }, - "raw_default" : "prefix", - "name" : "salt_position", - "desc" : "Salt position for PLAIN, MD5, SHA, SHA256 and SHA512 algorithms.", - "default" : { - "oneliner" : true, - "hocon" : "prefix" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for simple algorithms." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-http:get", - "fields" : [ - { - "type" : { - "name" : "get", - "kind" : "singleton" - }, - "name" : "method", - "desc" : "HTTP request method.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "raw_default" : { - "keep-alive" : "timeout=30, max=1000", - "connection" : "keep-alive", - "cache-control" : "no-cache", - "accept" : "application/json" - }, - "name" : "headers", - "desc" : "List of HTTP headers (without content-type).", - "default" : { - "oneliner" : false, - "hocon" : "{\n accept = \"application/json\"\n \"cache-control\" = \"no-cache\"\n connection = \"keep-alive\"\n \"keep-alive\" = \"timeout=30, max=1000\"\n}\n" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "password_based", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "http", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "url", - "desc" : "URL of the HTTP server.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "#{term() => binary()}", - "kind" : "primitive" - }, - "name" : "body", - "desc" : "HTTP request body.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "request_timeout", - "desc" : "HTTP request timeout.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "connect_timeout", - "desc" : "The timeout when connecting to the HTTP server.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "enable_pipelining", - "desc" : "A positive integer. Whether to send HTTP requests continuously, when set to 1, it means that after each HTTP request is sent, you need to wait for the server to return and then continue to send the next request.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "name" : "max_retries", - "desc" : "Deprecated since 5.0.4.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "The pool size.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "connector-http:request", - "kind" : "struct" - }, - "name" : "request", - "desc" : "Configure HTTP request parameters.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "name" : "retry_interval", - "desc" : "Deprecated since 5.0.4.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of authenticator using HTTP Server as authentication service (Using GET request)." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-http:post", - "fields" : [ - { - "type" : { - "name" : "post", - "kind" : "singleton" - }, - "name" : "method", - "desc" : "HTTP request method.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "raw_default" : { - "keep-alive" : "timeout=30, max=1000", - "content-type" : "application/json", - "connection" : "keep-alive", - "cache-control" : "no-cache", - "accept" : "application/json" - }, - "name" : "headers", - "desc" : "List of HTTP Headers.", - "default" : { - "oneliner" : false, - "hocon" : "{\n accept = \"application/json\"\n \"cache-control\" = \"no-cache\"\n connection = \"keep-alive\"\n \"content-type\" = \"application/json\"\n \"keep-alive\" = \"timeout=30, max=1000\"\n}\n" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "password_based", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "http", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "url", - "desc" : "URL of the HTTP server.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "#{term() => binary()}", - "kind" : "primitive" - }, - "name" : "body", - "desc" : "HTTP request body.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "request_timeout", - "desc" : "HTTP request timeout.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "connect_timeout", - "desc" : "The timeout when connecting to the HTTP server.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "enable_pipelining", - "desc" : "A positive integer. Whether to send HTTP requests continuously, when set to 1, it means that after each HTTP request is sent, you need to wait for the server to return and then continue to send the next request.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "name" : "max_retries", - "desc" : "Deprecated since 5.0.4.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "The pool size.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "connector-http:request", - "kind" : "struct" - }, - "name" : "request", - "desc" : "Configure HTTP request parameters.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "name" : "retry_interval", - "desc" : "Deprecated since 5.0.4.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of authenticator using HTTP Server as authentication service (Using POST request)." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-jwt:hmac-based", - "fields" : [ - { - "type" : { - "symbols" : [ - "false" - ], - "kind" : "enum" - }, - "name" : "use_jwks", - "desc" : "Whether to use JWKS.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "hmac-based" - ], - "kind" : "enum" - }, - "name" : "algorithm", - "desc" : "JWT signing algorithm, Supports HMAC (configured as hmac-based) and RSA, ECDSA (configured as public-key).", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "secret", - "desc" : "The key to verify the JWT using HMAC algorithm.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "secret_base64_encoded", - "desc" : "Whether secret is base64 encoded.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "jwt", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "acl", - "name" : "acl_claim_name", - "desc" : "JWT claim name to use for getting ACL rules.", - "default" : { - "oneliner" : true, - "hocon" : "\"acl\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "[term()]", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "verify_claims", - "desc" : "A list of custom claims to validate, which is a list of name/value pairs.\nValues can use the following placeholders:\n- ${username}: Will be replaced at runtime with Username used by the client when connecting\n- ${clientid}: Will be replaced at runtime with Client ID used by the client when connecting\nAuthentication will verify that the value of claims in the JWT (taken from the Password field) matches what is required in verify_claims.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "username", - "password" - ], - "kind" : "enum" - }, - "raw_default" : "password", - "name" : "from", - "desc" : "Field to take JWT from.", - "default" : { - "oneliner" : true, - "hocon" : "password" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration when the JWT for authentication is issued using the HMAC algorithm." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-jwt:jwks", - "fields" : [ - { - "type" : { - "symbols" : [ - "true" - ], - "kind" : "enum" - }, - "name" : "use_jwks", - "desc" : "Whether to use JWKS.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "endpoint", - "desc" : "JWKS endpoint, it's a read-only endpoint that returns the server's public key set in the JWKS format.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 300, - "name" : "refresh_interval", - "desc" : "JWKS refresh interval.", - "default" : { - "oneliner" : true, - "hocon" : "300" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL options.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "jwt", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "acl", - "name" : "acl_claim_name", - "desc" : "JWT claim name to use for getting ACL rules.", - "default" : { - "oneliner" : true, - "hocon" : "\"acl\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "[term()]", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "verify_claims", - "desc" : "A list of custom claims to validate, which is a list of name/value pairs.\nValues can use the following placeholders:\n- ${username}: Will be replaced at runtime with Username used by the client when connecting\n- ${clientid}: Will be replaced at runtime with Client ID used by the client when connecting\nAuthentication will verify that the value of claims in the JWT (taken from the Password field) matches what is required in verify_claims.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "username", - "password" - ], - "kind" : "enum" - }, - "raw_default" : "password", - "name" : "from", - "desc" : "Field to take JWT from.", - "default" : { - "oneliner" : true, - "hocon" : "password" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration when JWTs used for authentication need to be fetched from the JWKS endpoint." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-jwt:public-key", - "fields" : [ - { - "type" : { - "symbols" : [ - "false" - ], - "kind" : "enum" - }, - "name" : "use_jwks", - "desc" : "Whether to use JWKS.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "public-key" - ], - "kind" : "enum" - }, - "name" : "algorithm", - "desc" : "JWT signing algorithm, Supports HMAC (configured as hmac-based) and RSA, ECDSA (configured as public-key).", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "public_key", - "desc" : "The public key used to verify the JWT.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "jwt", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "acl", - "name" : "acl_claim_name", - "desc" : "JWT claim name to use for getting ACL rules.", - "default" : { - "oneliner" : true, - "hocon" : "\"acl\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "[term()]", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "verify_claims", - "desc" : "A list of custom claims to validate, which is a list of name/value pairs.\nValues can use the following placeholders:\n- ${username}: Will be replaced at runtime with Username used by the client when connecting\n- ${clientid}: Will be replaced at runtime with Client ID used by the client when connecting\nAuthentication will verify that the value of claims in the JWT (taken from the Password field) matches what is required in verify_claims.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "username", - "password" - ], - "kind" : "enum" - }, - "raw_default" : "password", - "name" : "from", - "desc" : "Field to take JWT from.", - "default" : { - "oneliner" : true, - "hocon" : "password" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration when the JWT for authentication is issued using RSA or ECDSA algorithm." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-mongodb:replica-set", - "fields" : [ - { - "type" : { - "name" : "password_based", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "mongodb", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "collection", - "desc" : "Collection used to store authentication data.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "filter", - "desc" : "Conditional expression that defines the filter condition in the query.\nFilter supports the following placeholders:\n- ${username}: Will be replaced at runtime with Username used by the client when connecting\n- ${clientid}: Will be replaced at runtime with Client ID used by the client when connecting", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "password_hash", - "name" : "password_hash_field", - "desc" : "Document field that contains password hash.", - "default" : { - "oneliner" : true, - "hocon" : "\"password_hash\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "salt", - "name" : "salt_field", - "desc" : "Document field that contains the password salt.", - "default" : { - "oneliner" : true, - "hocon" : "\"salt\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "is_superuser", - "name" : "is_superuser_field", - "desc" : "Document field that defines if the user has superuser privileges.", - "default" : { - "oneliner" : true, - "hocon" : "\"is_superuser\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "authn-hash:bcrypt", - "kind" : "struct" - }, - { - "name" : "authn-hash:pbkdf2", - "kind" : "struct" - }, - { - "name" : "authn-hash:simple", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "raw_default" : { - "salt_position" : "prefix", - "name" : "sha256" - }, - "name" : "password_hash_algorithm", - "desc" : "Options for password hash verification.", - "default" : { - "oneliner" : true, - "hocon" : "{name = sha256, salt_position = prefix}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "rs", - "kind" : "singleton" - }, - "raw_default" : "rs", - "name" : "mongo_type", - "desc" : "Replica set. Must be set to 'rs' when MongoDB server is running in 'replica set' mode.", - "default" : { - "oneliner" : true, - "hocon" : "rs" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe MongoDB default port 27017 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "unsafe", - "safe" - ], - "kind" : "enum" - }, - "raw_default" : "unsafe", - "name" : "w_mode", - "desc" : "Write mode.", - "default" : { - "oneliner" : true, - "hocon" : "unsafe" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "master", - "slave_ok" - ], - "kind" : "enum" - }, - "raw_default" : "master", - "name" : "r_mode", - "desc" : "Read mode.", - "default" : { - "oneliner" : true, - "hocon" : "master" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "replica_set_name", - "desc" : "Name of the replica set.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "srv_record", - "desc" : "Use DNS SRV record.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "auth_source", - "desc" : "Database name associated with the user's credentials.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "topology", - "kind" : "struct" - }, - "name" : "topology", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of authenticator using MongoDB (Replica Set) as authentication data source." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-mongodb:sharded-cluster", - "fields" : [ - { - "type" : { - "name" : "password_based", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "mongodb", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "collection", - "desc" : "Collection used to store authentication data.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "filter", - "desc" : "Conditional expression that defines the filter condition in the query.\nFilter supports the following placeholders:\n- ${username}: Will be replaced at runtime with Username used by the client when connecting\n- ${clientid}: Will be replaced at runtime with Client ID used by the client when connecting", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "password_hash", - "name" : "password_hash_field", - "desc" : "Document field that contains password hash.", - "default" : { - "oneliner" : true, - "hocon" : "\"password_hash\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "salt", - "name" : "salt_field", - "desc" : "Document field that contains the password salt.", - "default" : { - "oneliner" : true, - "hocon" : "\"salt\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "is_superuser", - "name" : "is_superuser_field", - "desc" : "Document field that defines if the user has superuser privileges.", - "default" : { - "oneliner" : true, - "hocon" : "\"is_superuser\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "authn-hash:bcrypt", - "kind" : "struct" - }, - { - "name" : "authn-hash:pbkdf2", - "kind" : "struct" - }, - { - "name" : "authn-hash:simple", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "raw_default" : { - "salt_position" : "prefix", - "name" : "sha256" - }, - "name" : "password_hash_algorithm", - "desc" : "Options for password hash verification.", - "default" : { - "oneliner" : true, - "hocon" : "{name = sha256, salt_position = prefix}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "sharded", - "kind" : "singleton" - }, - "raw_default" : "sharded", - "name" : "mongo_type", - "desc" : "Sharded cluster. Must be set to 'sharded' when MongoDB server is running in 'sharded' mode.", - "default" : { - "oneliner" : true, - "hocon" : "sharded" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe MongoDB default port 27017 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "unsafe", - "safe" - ], - "kind" : "enum" - }, - "raw_default" : "unsafe", - "name" : "w_mode", - "desc" : "Write mode.", - "default" : { - "oneliner" : true, - "hocon" : "unsafe" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "srv_record", - "desc" : "Use DNS SRV record.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "auth_source", - "desc" : "Database name associated with the user's credentials.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "topology", - "kind" : "struct" - }, - "name" : "topology", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of authenticator using MongoDB (Sharded Cluster) as authentication data source." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-mongodb:standalone", - "fields" : [ - { - "type" : { - "name" : "password_based", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "mongodb", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "collection", - "desc" : "Collection used to store authentication data.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "filter", - "desc" : "Conditional expression that defines the filter condition in the query.\nFilter supports the following placeholders:\n- ${username}: Will be replaced at runtime with Username used by the client when connecting\n- ${clientid}: Will be replaced at runtime with Client ID used by the client when connecting", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "password_hash", - "name" : "password_hash_field", - "desc" : "Document field that contains password hash.", - "default" : { - "oneliner" : true, - "hocon" : "\"password_hash\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "salt", - "name" : "salt_field", - "desc" : "Document field that contains the password salt.", - "default" : { - "oneliner" : true, - "hocon" : "\"salt\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "is_superuser", - "name" : "is_superuser_field", - "desc" : "Document field that defines if the user has superuser privileges.", - "default" : { - "oneliner" : true, - "hocon" : "\"is_superuser\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "authn-hash:bcrypt", - "kind" : "struct" - }, - { - "name" : "authn-hash:pbkdf2", - "kind" : "struct" - }, - { - "name" : "authn-hash:simple", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "raw_default" : { - "salt_position" : "prefix", - "name" : "sha256" - }, - "name" : "password_hash_algorithm", - "desc" : "Options for password hash verification.", - "default" : { - "oneliner" : true, - "hocon" : "{name = sha256, salt_position = prefix}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "single", - "kind" : "singleton" - }, - "raw_default" : "single", - "name" : "mongo_type", - "desc" : "Standalone instance. Must be set to 'single' when MongoDB server is running in standalone mode.", - "default" : { - "oneliner" : true, - "hocon" : "single" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe MongoDB default port 27017 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "unsafe", - "safe" - ], - "kind" : "enum" - }, - "raw_default" : "unsafe", - "name" : "w_mode", - "desc" : "Write mode.", - "default" : { - "oneliner" : true, - "hocon" : "unsafe" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "srv_record", - "desc" : "Use DNS SRV record.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "auth_source", - "desc" : "Database name associated with the user's credentials.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "topology", - "kind" : "struct" - }, - "name" : "topology", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of authenticator using MongoDB (Standalone) as authentication data source." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-mysql:authentication", - "fields" : [ - { - "type" : { - "name" : "password_based", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "mysql", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "authn-hash:bcrypt", - "kind" : "struct" - }, - { - "name" : "authn-hash:pbkdf2", - "kind" : "struct" - }, - { - "name" : "authn-hash:simple", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "raw_default" : { - "salt_position" : "prefix", - "name" : "sha256" - }, - "name" : "password_hash_algorithm", - "desc" : "Options for password hash verification.", - "default" : { - "oneliner" : true, - "hocon" : "{name = sha256, salt_position = prefix}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "query", - "desc" : "SQL used to query data for authentication, such as password hash.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "query_timeout", - "desc" : "Timeout for the SQL query.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe MySQL default port 3306 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "root", - "name" : "username", - "desc" : "EMQX's username in the external database.", - "default" : { - "oneliner" : true, - "hocon" : "\"root\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of authenticator using MySQL as authentication data source." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-postgresql:authentication", - "fields" : [ - { - "type" : { - "name" : "password_based", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "postgresql", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "authn-hash:bcrypt", - "kind" : "struct" - }, - { - "name" : "authn-hash:pbkdf2", - "kind" : "struct" - }, - { - "name" : "authn-hash:simple", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "raw_default" : { - "salt_position" : "prefix", - "name" : "sha256" - }, - "name" : "password_hash_algorithm", - "desc" : "Options for password hash verification.", - "default" : { - "oneliner" : true, - "hocon" : "{name = sha256, salt_position = prefix}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "query", - "desc" : "SQL used to query data for authentication, such as password hash.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe PostgreSQL default port 5432 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of authenticator using PostgreSQL as authentication data source." - }, - { - "tags" : [ - - ], - "paths" : [ - "psk_authentication" - ], - "full_name" : "authn-psk:psk_authentication", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "enable", - "desc" : "Whether to enable TLS PSK support", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "init_file", - "desc" : "If init_file is specified, EMQX will import PSKs from the file into the built-in database at startup for use by the runtime.\nThe file has to be structured line-by-line, each line must be in the format of PSKIdentity:SharedSecret.\nFor example: mydevice1:c2VjcmV0", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : ":", - "name" : "separator", - "desc" : "The separator between PSKIdentity and SharedSecret in the PSK file", - "default" : { - "oneliner" : true, - "hocon" : "\":\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 50, - "name" : "chunk_size", - "desc" : "The size of each chunk used to import to the built-in database from PSK file", - "default" : { - "oneliner" : true, - "hocon" : "50" - }, - "aliases" : [ - - ] - } - ], - "desc" : "PSK stands for 'Pre-Shared Keys'.\nThis config to enable TLS-PSK authentication.\n\nImportant! Make sure the SSL listener with only tlsv1.2 enabled, and also PSK cipher suites\nconfigured, such as RSA-PSK-AES256-GCM-SHA384.\n\nSee listener SSL options config for more details.\n\nThe IDs and secrets can be provided from a file which is configurable by the init_file field." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-redis:cluster", - "fields" : [ - { - "type" : { - "name" : "password_based", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "redis", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "cmd", - "desc" : "The Redis Command used to query data for authentication such as password hash, currently only supports HGET and HMGET.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "authn-hash:bcrypt", - "kind" : "struct" - }, - { - "name" : "authn-hash:pbkdf2", - "kind" : "struct" - }, - { - "name" : "authn-hash:simple", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "raw_default" : { - "salt_position" : "prefix", - "name" : "sha256" - }, - "name" : "password_hash_algorithm", - "desc" : "Options for password hash verification.", - "default" : { - "oneliner" : true, - "hocon" : "{name = sha256, salt_position = prefix}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe Redis default port 6379 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "cluster", - "kind" : "singleton" - }, - "raw_default" : "cluster", - "name" : "redis_type", - "desc" : "Cluster mode. Must be set to 'cluster' when Redis server is running in clustered mode.", - "default" : { - "oneliner" : true, - "hocon" : "cluster" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of authenticator using Redis (Cluster) as authentication data source." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-redis:sentinel", - "fields" : [ - { - "type" : { - "name" : "password_based", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "redis", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "cmd", - "desc" : "The Redis Command used to query data for authentication such as password hash, currently only supports HGET and HMGET.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "authn-hash:bcrypt", - "kind" : "struct" - }, - { - "name" : "authn-hash:pbkdf2", - "kind" : "struct" - }, - { - "name" : "authn-hash:simple", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "raw_default" : { - "salt_position" : "prefix", - "name" : "sha256" - }, - "name" : "password_hash_algorithm", - "desc" : "Options for password hash verification.", - "default" : { - "oneliner" : true, - "hocon" : "{name = sha256, salt_position = prefix}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe Redis default port 6379 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "sentinel", - "kind" : "singleton" - }, - "raw_default" : "sentinel", - "name" : "redis_type", - "desc" : "Sentinel mode. Must be set to 'sentinel' when Redis server is running in sentinel mode.", - "default" : { - "oneliner" : true, - "hocon" : "sentinel" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "sentinel", - "desc" : "The cluster name in Redis sentinel mode.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "database", - "desc" : "Redis database ID.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of authenticator using Redis (Sentinel) as authentication data source." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-redis:standalone", - "fields" : [ - { - "type" : { - "name" : "password_based", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "redis", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "cmd", - "desc" : "The Redis Command used to query data for authentication such as password hash, currently only supports HGET and HMGET.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "authn-hash:bcrypt", - "kind" : "struct" - }, - { - "name" : "authn-hash:pbkdf2", - "kind" : "struct" - }, - { - "name" : "authn-hash:simple", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "raw_default" : { - "salt_position" : "prefix", - "name" : "sha256" - }, - "name" : "password_hash_algorithm", - "desc" : "Options for password hash verification.", - "default" : { - "oneliner" : true, - "hocon" : "{name = sha256, salt_position = prefix}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe Redis default port 6379 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "single", - "kind" : "singleton" - }, - "raw_default" : "single", - "name" : "redis_type", - "desc" : "Single mode. Must be set to 'single' when Redis server is running in single mode.", - "default" : { - "oneliner" : true, - "hocon" : "single" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "database", - "desc" : "Redis database ID.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of authenticator using Redis (Standalone) as authentication data source." - }, - { - "tags" : [ - "Authentication" - ], - "paths" : [ - "authentication.$INDEX" - ], - "full_name" : "authn-scram-builtin_db:authentication", - "fields" : [ - { - "type" : { - "name" : "scram", - "kind" : "singleton" - }, - "name" : "mechanism", - "desc" : "Authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "built_in_database", - "kind" : "singleton" - }, - "name" : "backend", - "desc" : "Backend type.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "sha256", - "sha512" - ], - "kind" : "enum" - }, - "raw_default" : "sha256", - "name" : "algorithm", - "desc" : "Hashing algorithm.", - "default" : { - "oneliner" : true, - "hocon" : "sha256" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 4096, - "name" : "iteration_count", - "desc" : "Iteration count.", - "default" : { - "oneliner" : true, - "hocon" : "4096" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Set to true or false to disable this auth provider.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for Salted Challenge Response Authentication Mechanism\n(SCRAM) authentication." - }, - { - "tags" : [ - "Bridge" - ], - "paths" : [ - "bridges" - ], - "full_name" : "bridge:bridges", - "fields" : [ - { - "type" : { - "values" : { - "name" : "bridge_webhook:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "webhook", - "desc" : "WebHook to an HTTP server.", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_mqtt:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "mqtt", - "desc" : "MQTT bridges to/from another MQTT broker", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_hstreamdb:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "hstreamdb", - "desc" : "HStreamDB Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_gcp_pubsub:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "gcp_pubsub", - "desc" : "EMQX Enterprise Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_mysql:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "mysql", - "desc" : "MySQL Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_tdengine:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "tdengine", - "desc" : "TDengine Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_dynamo:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "dynamo", - "desc" : "Dynamo Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_rocketmq:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "rocketmq", - "desc" : "RocketMQ Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_cassa:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "cassandra", - "desc" : "Cassandra Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_kafka:kafka_producer", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "kafka", - "desc" : "Kafka Producer Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_kafka:kafka_consumer", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "kafka_consumer", - "desc" : "Kafka Consumer Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_mongodb:mongodb_rs", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "mongodb_rs", - "desc" : "MongoDB Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_mongodb:mongodb_sharded", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "mongodb_sharded", - "desc" : "MongoDB Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_mongodb:mongodb_single", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "mongodb_single", - "desc" : "MongoDB Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_influxdb:influxdb_api_v1", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "influxdb_api_v1", - "desc" : "InfluxDB Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_influxdb:influxdb_api_v2", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "influxdb_api_v2", - "desc" : "InfluxDB Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_redis:redis_single", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "redis_single", - "desc" : "Redis Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_redis:redis_sentinel", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "redis_sentinel", - "desc" : "Redis Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_redis:redis_cluster", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "redis_cluster", - "desc" : "Redis Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_pgsql:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "pgsql", - "desc" : "PostgreSQL Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_pgsql:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "timescale", - "desc" : "Timescale Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_pgsql:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "matrix", - "desc" : "Matrix Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_clickhouse:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "clickhouse", - "desc" : "Clickhouse Bridge Config", - "aliases" : [ - - ] - }, - { - "type" : { - "values" : { - "name" : "bridge_sqlserver:config", - "kind" : "struct" - }, - "name" : "name", - "kind" : "map" - }, - "name" : "sqlserver", - "desc" : "Microsoft SQL Server Bridge Config", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for MQTT bridges." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.cassandra.$name" - ], - "full_name" : "bridge_cassa:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "insert into mqtt_msg(topic, msgid, sender, qos, payload, arrived, retain) values (${topic}, ${id}, ${clientid}, ${qos}, ${payload}, ${timestamp}, ${flags.retain})", - "name" : "cql", - "desc" : "CQL Template", - "default" : { - "oneliner" : true, - "hocon" : "\"insert into mqtt_msg(topic, msgid, sender, qos, payload, arrived, retain) values (${topic}, ${id}, ${clientid}, ${qos}, ${payload}, ${timestamp}, ${flags.retain})\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to Cassandra. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "resource_schema:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port][,Host2:Port]`.
\nThe Cassandra default port 9042 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "keyspace", - "desc" : "Keyspace name to connect to.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a Cassandra bridge." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.clickhouse.$name" - ], - "full_name" : "bridge_clickhouse:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "INSERT INTO mqtt_test(payload, arrived) VALUES ('${payload}', ${timestamp})", - "name" : "sql", - "desc" : "The template string can contain ${field} placeholders for message metadata and payload field. Make sure that the inserted values are formatted and escaped correctly. [Prepared Statement](https://docs.emqx.com/en/enterprise/v5.0/data-integration/data-bridges.html#Prepared-Statement) is not supported.", - "default" : { - "oneliner" : true, - "hocon" : "\"INSERT INTO mqtt_test(payload, arrived) VALUES ('${payload}', ${timestamp})\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : ", ", - "name" : "batch_value_separator", - "desc" : "The default value ',' works for the VALUES format. You can also use other separator if other format is specified. See [INSERT INTO Statement](https://clickhouse.com/docs/en/sql-reference/statements/insert-into).", - "default" : { - "oneliner" : true, - "hocon" : "\", \"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to Clickhouse. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_clickhouse:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_ee_connector_clickhouse:url()", - "kind" : "primitive" - }, - "name" : "url", - "desc" : "The HTTP URL to the Clickhouse server that you want to connect to (for example http://myhostname:8123)", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "connect_timeout", - "desc" : "The timeout when connecting to the Clickhouse server.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a Clickhouse bridge." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.clickhouse.$name.resource_opts" - ], - "full_name" : "bridge_clickhouse:creation_opts", - "fields" : [ - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "worker_pool_size", - "desc" : "The number of buffer workers. Only applicable for egress type bridges.\nFor bridges only have ingress direction data flow, it can be set to 0 otherwise must be greater than 0.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "health_check_interval", - "desc" : "Health check interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : "true", - "name" : "start_after_created", - "desc" : "Whether start the resource right after created.", - "default" : { - "oneliner" : true, - "hocon" : "\"true\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "start_timeout", - "desc" : "Time interval to wait for an auto-started resource to become healthy before responding resource creation requests.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "60s", - "name" : "auto_restart_interval", - "desc" : "The auto restart interval after the resource is disconnected.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "sync", - "async" - ], - "kind" : "enum" - }, - "raw_default" : "async", - "name" : "query_mode", - "desc" : "Query mode. Optional 'sync/async', default 'async'.", - "default" : { - "oneliner" : true, - "hocon" : "async" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "15s", - "name" : "request_timeout", - "desc" : "Starting from the moment when the request enters the buffer, if the request remains in the buffer for the specified time or is sent but does not receive a response or acknowledgement in time, the request is considered expired.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "inflight_window", - "desc" : "Query inflight window. When query_mode is set to async, this config has to be set to 1 if messages from the same MQTT client have to be strictly ordered.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - "async_inflight_window" - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_batch", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 1, - "name" : "batch_size", - "desc" : "Maximum batch count. If equal to 1, there's effectively no batching.", - "default" : { - "oneliner" : true, - "hocon" : "1" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "0ms", - "name" : "batch_time", - "desc" : "Maximum waiting interval when accumulating a batch at a low message rates for more efficient resource usage.", - "default" : { - "oneliner" : true, - "hocon" : "\"0ms\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_queue", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "256MB", - "name" : "max_buffer_bytes", - "desc" : "Maximum number of bytes to buffer for each buffer worker.", - "default" : { - "oneliner" : true, - "hocon" : "\"256MB\"" - }, - "aliases" : [ - "max_queue_bytes" - ] - } - ], - "desc" : "Creation options." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.dynamo.$name" - ], - "full_name" : "bridge_dynamo:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "template", - "desc" : "Template, the default value is empty. When this value is empty the whole message will be stored in the database", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to DynamoDB. All MQTT `PUBLISH` messages with the topic\nmatching the `local_topic` will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also `local_topic` is\nconfigured, then both the data got from the rule and the MQTT messages that match `local_topic`\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_dynamo:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "url", - "desc" : "The url of DynamoDB endpoint.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "table", - "desc" : "DynamoDB Table.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "aws_access_key_id", - "desc" : "Access Key ID for connecting to DynamoDB.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "aws_secret_access_key", - "desc" : "AWS Secret Access Key for connecting to DynamoDB.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a DynamoDB bridge." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.dynamo.$name.resource_opts" - ], - "full_name" : "bridge_dynamo:creation_opts", - "fields" : [ - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "worker_pool_size", - "desc" : "The number of buffer workers. Only applicable for egress type bridges.\nFor bridges only have ingress direction data flow, it can be set to 0 otherwise must be greater than 0.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "health_check_interval", - "desc" : "Health check interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : "true", - "name" : "start_after_created", - "desc" : "Whether start the resource right after created.", - "default" : { - "oneliner" : true, - "hocon" : "\"true\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "start_timeout", - "desc" : "Time interval to wait for an auto-started resource to become healthy before responding resource creation requests.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "60s", - "name" : "auto_restart_interval", - "desc" : "The auto restart interval after the resource is disconnected.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "sync", - "async" - ], - "kind" : "enum" - }, - "raw_default" : "async", - "name" : "query_mode", - "desc" : "Query mode. Optional 'sync/async', default 'async'.", - "default" : { - "oneliner" : true, - "hocon" : "async" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "15s", - "name" : "request_timeout", - "desc" : "Starting from the moment when the request enters the buffer, if the request remains in the buffer for the specified time or is sent but does not receive a response or acknowledgement in time, the request is considered expired.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "inflight_window", - "desc" : "Query inflight window. When query_mode is set to async, this config has to be set to 1 if messages from the same MQTT client have to be strictly ordered.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - "async_inflight_window" - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_batch", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 1, - "name" : "batch_size", - "desc" : "Maximum batch count. If equal to 1, there's effectively no batching.", - "default" : { - "oneliner" : true, - "hocon" : "1" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "0ms", - "name" : "batch_time", - "desc" : "Maximum waiting interval when accumulating a batch at a low message rates for more efficient resource usage.", - "default" : { - "oneliner" : true, - "hocon" : "\"0ms\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_queue", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "256MB", - "name" : "max_buffer_bytes", - "desc" : "Maximum number of bytes to buffer for each buffer worker.", - "default" : { - "oneliner" : true, - "hocon" : "\"256MB\"" - }, - "aliases" : [ - "max_queue_bytes" - ] - } - ], - "desc" : "Creation options." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.gcp_pubsub.$name" - ], - "full_name" : "bridge_gcp_pubsub:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "resource_schema:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "connect_timeout", - "desc" : "The timeout when connecting to the HTTP server.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "The pool size.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "pipelining", - "desc" : "A positive integer. Whether to send HTTP requests continuously, when set to 1, it means that after each HTTP request is sent, you need to wait for the server to return and then continue to send the next request.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 2, - "name" : "max_retries", - "desc" : "Max retry times if an error occurs when sending a request.", - "default" : { - "oneliner" : true, - "hocon" : "2" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "name" : "request_timeout", - "desc" : "Deprecated since e5.0.1.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "payload_template", - "desc" : "The template for formatting the outgoing messages. If undefined, will send all the available context in JSON format.", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to GCP PubSub. All MQTT 'PUBLISH' messages with the topic\nmatching `local_topic` will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "pubsub_topic", - "desc" : "The GCP PubSub topic to publish messages to.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_ee_bridge_gcp_pubsub:service_account_json()", - "kind" : "primitive" - }, - "name" : "service_account_json", - "desc" : "JSON containing the GCP Service Account credentials to be used with PubSub.\nWhen a GCP Service Account is created (as described in https://developers.google.com/identity/protocols/oauth2/service-account#creatinganaccount), you have the option of downloading the credentials in JSON form. That's the file needed.", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a GCP PubSub bridge." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.hstreamdb.$name" - ], - "full_name" : "bridge_hstreamdb:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "egress", - "kind" : "singleton" - }, - "raw_default" : "egress", - "name" : "direction", - "desc" : "The direction of this bridge, MUST be 'egress'", - "default" : { - "oneliner" : true, - "hocon" : "egress" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to the HStreamDB. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "${payload}", - "name" : "payload", - "desc" : "The payload to be forwarded to the HStreamDB. Placeholders supported.", - "default" : { - "oneliner" : true, - "hocon" : "\"${payload}\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "binary()", - "kind" : "primitive" - }, - { - "name" : "connector_hstreamdb:config", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "name" : "connector", - "examples" : [ - "hstreamdb:demo" - ], - "desc" : "Generic configuration for the connector.", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for an HStreamDB bridge." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.influxdb_api_v1.$name" - ], - "full_name" : "bridge_influxdb:influxdb_api_v1", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to the InfluxDB. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_ee_bridge_influxdb:write_syntax()", - "kind" : "primitive" - }, - "name" : "write_syntax", - "desc" : "Conf of InfluxDB line protocol to write data points. It is a text-based format that provides the measurement, tag set, field set, and timestamp of a data point, and placeholder supported.\nSee also [InfluxDB 2.3 Line Protocol](https://docs.influxdata.com/influxdb/v2.3/reference/syntax/line-protocol/) and\n[InfluxDB 1.8 Line Protocol](https://docs.influxdata.com/influxdb/v1.8/write_protocols/line_protocol_tutorial/)
\nTLDR:
\n```\n[,=[,=]] =[,=] []\n```\nPlease note that a placeholder for an integer value must be annotated with a suffix `i`. For example `${payload.int_value}i`.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "resource_schema:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "127.0.0.1:8086", - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe InfluxDB default port 8086 is used if `[:Port]` is not specified.", - "default" : { - "oneliner" : true, - "hocon" : "\"127.0.0.1:8086\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "ns", - "us", - "ms", - "s" - ], - "kind" : "enum" - }, - "raw_default" : "ms", - "name" : "precision", - "desc" : "InfluxDB time precision.", - "default" : { - "oneliner" : true, - "hocon" : "ms" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "InfluxDB database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "InfluxDB username.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "InfluxDB password.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "InfluxDB's protocol. Support InfluxDB v1.8 and before." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.influxdb_api_v2.$name" - ], - "full_name" : "bridge_influxdb:influxdb_api_v2", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to the InfluxDB. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_ee_bridge_influxdb:write_syntax()", - "kind" : "primitive" - }, - "name" : "write_syntax", - "desc" : "Conf of InfluxDB line protocol to write data points. It is a text-based format that provides the measurement, tag set, field set, and timestamp of a data point, and placeholder supported.\nSee also [InfluxDB 2.3 Line Protocol](https://docs.influxdata.com/influxdb/v2.3/reference/syntax/line-protocol/) and\n[InfluxDB 1.8 Line Protocol](https://docs.influxdata.com/influxdb/v1.8/write_protocols/line_protocol_tutorial/)
\nTLDR:
\n```\n[,=[,=]] =[,=] []\n```\nPlease note that a placeholder for an integer value must be annotated with a suffix `i`. For example `${payload.int_value}i`.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "resource_schema:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "127.0.0.1:8086", - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe InfluxDB default port 8086 is used if `[:Port]` is not specified.", - "default" : { - "oneliner" : true, - "hocon" : "\"127.0.0.1:8086\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "ns", - "us", - "ms", - "s" - ], - "kind" : "enum" - }, - "raw_default" : "ms", - "name" : "precision", - "desc" : "InfluxDB time precision.", - "default" : { - "oneliner" : true, - "hocon" : "ms" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "bucket", - "desc" : "InfluxDB bucket name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "org", - "desc" : "Organization name of InfluxDB.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "token", - "desc" : "InfluxDB token.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "InfluxDB's protocol. Support InfluxDB v2.0 and after." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.kafka.$name.authentication", - "bridges.kafka_consumer.$name.authentication" - ], - "full_name" : "bridge_kafka:auth_gssapi_kerberos", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "kerberos_principal", - "desc" : "SASL GSSAPI authentication Kerberos principal. For example client_name@MY.KERBEROS.REALM.MYDOMAIN.COM, NOTE: The realm in use has to be configured in /etc/krb5.conf in EMQX nodes.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "kerberos_keytab_file", - "desc" : "SASL GSSAPI authentication Kerberos keytab file path. NOTE: This file has to be placed in EMQX nodes, and the EMQX service runner user requires read permission.", - "aliases" : [ - - ] - } - ], - "desc" : "Use GSSAPI/Kerberos authentication." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.kafka.$name.authentication", - "bridges.kafka_consumer.$name.authentication" - ], - "full_name" : "bridge_kafka:auth_username_password", - "fields" : [ - { - "type" : { - "symbols" : [ - "plain", - "scram_sha_256", - "scram_sha_512" - ], - "kind" : "enum" - }, - "name" : "mechanism", - "desc" : "SASL authentication mechanism.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "SASL authentication username.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "SASL authentication password.", - "aliases" : [ - - ] - } - ], - "desc" : "Username/password based authentication." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.kafka_consumer.$name.kafka" - ], - "full_name" : "bridge_kafka:consumer_kafka_opts", - "fields" : [ - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : [ - 56, - 57, - 54, - 75, - 66 - ], - "name" : "max_batch_bytes", - "desc" : "Set how many bytes to pull from Kafka in each fetch request. Please note that if the configured value is smaller than the message size in Kafka, it may negatively impact the fetch performance.", - "default" : { - "oneliner" : true, - "hocon" : "\"896KB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "latest", - "earliest" - ], - "kind" : "enum" - }, - "raw_default" : "latest", - "name" : "offset_reset_policy", - "desc" : "Defines from which offset a consumer should start fetching when there is no commit history or when the commit history becomes invalid.", - "default" : { - "oneliner" : true, - "hocon" : "latest" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 5, - "name" : "offset_commit_interval_seconds", - "desc" : "Defines the time interval between two offset commit requests sent for each consumer group.", - "default" : { - "oneliner" : true, - "hocon" : "5" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Kafka consumer configs." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.kafka_consumer.$name.topic_mapping.$INDEX" - ], - "full_name" : "bridge_kafka:consumer_topic_mapping", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "kafka_topic", - "desc" : "Kafka topic to consume from.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "mqtt_topic", - "desc" : "Local topic to which consumed Kafka messages should be published to.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "qos()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "qos", - "desc" : "MQTT QoS used to publish messages consumed from Kafka.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "${.}", - "name" : "payload_template", - "desc" : "The template for transforming the incoming Kafka message. By default, it will use JSON format to serialize inputs from the Kafka message. Such fields are:\nheaders: an object containing string key-value pairs.\nkey: Kafka message key (uses the chosen key encoding).\noffset: offset for the message.\ntopic: Kafka topic.\nts: message timestamp.\nts_type: message timestamp type, which is one of create, append or undefined.\nvalue: Kafka message value (uses the chosen value encoding).", - "default" : { - "oneliner" : true, - "hocon" : "\"${.}\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Defines the mapping between Kafka topics and MQTT topics. Must contain at least one item." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.kafka_consumer.$name" - ], - "full_name" : "bridge_kafka:kafka_consumer", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable (true) or disable (false) this Kafka bridge.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "bootstrap_hosts", - "desc" : "A comma separated list of Kafka host[:port] endpoints to bootstrap the client. Default port number is 9092.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "connect_timeout", - "desc" : "Maximum wait time for TCP connection establishment (including authentication time if enabled).", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "3s", - "name" : "min_metadata_refresh_interval", - "desc" : "Minimum time interval the client has to wait before refreshing Kafka broker and topic metadata. Setting too small value may add extra load on Kafka.", - "default" : { - "oneliner" : true, - "hocon" : "\"3s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "metadata_request_timeout", - "desc" : "Maximum wait time when fetching metadata from Kafka.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "none", - "kind" : "singleton" - }, - { - "name" : "bridge_kafka:auth_username_password", - "kind" : "struct" - }, - { - "name" : "bridge_kafka:auth_gssapi_kerberos", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "raw_default" : "none", - "name" : "authentication", - "desc" : "Authentication configs.", - "default" : { - "oneliner" : true, - "hocon" : "none" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_kafka:socket_opts", - "kind" : "struct" - }, - "name" : "socket_opts", - "desc" : "Extra socket options.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_kafka:consumer_kafka_opts", - "kind" : "struct" - }, - "name" : "kafka", - "desc" : "Kafka consumer configs.", - "aliases" : [ - - ] - }, - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "bridge_kafka:consumer_topic_mapping", - "kind" : "struct" - } - }, - "name" : "topic_mapping", - "desc" : "Defines the mapping between Kafka topics and MQTT topics. Must contain at least one item.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "none", - "base64" - ], - "kind" : "enum" - }, - "raw_default" : "none", - "name" : "key_encoding_mode", - "desc" : "Defines how the key from the Kafka message is encoded before being forwarded via MQTT.\nnone Uses the key from the Kafka message unchanged. Note: in this case, the key must be a valid UTF-8 string.\nbase64 Uses base-64 encoding on the received key.", - "default" : { - "oneliner" : true, - "hocon" : "none" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "none", - "base64" - ], - "kind" : "enum" - }, - "raw_default" : "none", - "name" : "value_encoding_mode", - "desc" : "Defines how the value from the Kafka message is encoded before being forwarded via MQTT.\nnone Uses the value from the Kafka message unchanged. Note: in this case, the value must be a valid UTF-8 string.\nbase64 Uses base-64 encoding on the received value.", - "default" : { - "oneliner" : true, - "hocon" : "none" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Kafka Consumer configuration." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.kafka.$name.kafka.message" - ], - "full_name" : "bridge_kafka:kafka_message", - "fields" : [ - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "${.clientid}", - "name" : "key", - "desc" : "Template to render Kafka message key. If the template is rendered into a NULL value (i.e. there is no such data field in Rule Engine context) then Kafka's NULL (but not empty string) is used.", - "default" : { - "oneliner" : true, - "hocon" : "\"${.clientid}\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "${.}", - "name" : "value", - "desc" : "Template to render Kafka message value. If the template is rendered into a NULL value (i.e. there is no such data field in Rule Engine context) then Kafka's NULL (but not empty string) is used.", - "default" : { - "oneliner" : true, - "hocon" : "\"${.}\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "${.timestamp}", - "name" : "timestamp", - "desc" : "Which timestamp to use. The timestamp is expected to be a millisecond precision Unix epoch which can be in string format, e.g. 1661326462115 or '1661326462115'. When the desired data field for this template is not found, or if the found data is not a valid integer, the current system timestamp will be used.", - "default" : { - "oneliner" : true, - "hocon" : "\"${.timestamp}\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Template to render a Kafka message." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.kafka.$name" - ], - "full_name" : "bridge_kafka:kafka_producer", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable (true) or disable (false) this Kafka bridge.", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "bootstrap_hosts", - "desc" : "A comma separated list of Kafka host[:port] endpoints to bootstrap the client. Default port number is 9092.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "connect_timeout", - "desc" : "Maximum wait time for TCP connection establishment (including authentication time if enabled).", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "3s", - "name" : "min_metadata_refresh_interval", - "desc" : "Minimum time interval the client has to wait before refreshing Kafka broker and topic metadata. Setting too small value may add extra load on Kafka.", - "default" : { - "oneliner" : true, - "hocon" : "\"3s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "metadata_request_timeout", - "desc" : "Maximum wait time when fetching metadata from Kafka.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "none", - "kind" : "singleton" - }, - { - "name" : "bridge_kafka:auth_username_password", - "kind" : "struct" - }, - { - "name" : "bridge_kafka:auth_gssapi_kerberos", - "kind" : "struct" - } - ], - "kind" : "union" - }, - "raw_default" : "none", - "name" : "authentication", - "desc" : "Authentication configs.", - "default" : { - "oneliner" : true, - "hocon" : "none" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_kafka:socket_opts", - "kind" : "struct" - }, - "name" : "socket_opts", - "desc" : "Extra socket options.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "MQTT topic or topic filter as data source (bridge input). If rule action is used as data source, this config should be left empty, otherwise messages will be duplicated in Kafka.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_kafka:producer_kafka_opts", - "kind" : "struct" - }, - "name" : "kafka", - "desc" : "Kafka producer configs.", - "aliases" : [ - - ] - } - ], - "desc" : "Kafka Producer configuration." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.kafka.$name.kafka.buffer" - ], - "full_name" : "bridge_kafka:producer_buffer", - "fields" : [ - { - "type" : { - "symbols" : [ - "memory", - "disk", - "hybrid" - ], - "kind" : "enum" - }, - "raw_default" : "memory", - "name" : "mode", - "desc" : "Message buffer mode.\n\nmemory: Buffer all messages in memory. The messages will be lost in case of EMQX node restart\ndisk: Buffer all messages on disk. The messages on disk are able to survive EMQX node restart.\nhybrid: Buffer message in memory first, when up to certain limit (see segment_bytes config for more information), then start offloading messages to disk, Like memory mode, the messages will be lost in case of EMQX node restart.", - "default" : { - "oneliner" : true, - "hocon" : "memory" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "2GB", - "name" : "per_partition_limit", - "desc" : "Number of bytes allowed to buffer for each Kafka partition. When this limit is exceeded, old messages will be dropped in a trade for credits for new messages to be buffered.", - "default" : { - "oneliner" : true, - "hocon" : "\"2GB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "100MB", - "name" : "segment_bytes", - "desc" : "Applicable when buffer mode is set to disk or hybrid.\nThis value is to specify the size of each on-disk buffer file.", - "default" : { - "oneliner" : true, - "hocon" : "\"100MB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "memory_overload_protection", - "desc" : "Applicable when buffer mode is set to memory\nEMQX will drop old buffered messages under high memory pressure. The high memory threshold is defined in config sysmon.os.sysmem_high_watermark. NOTE: This config only works on Linux.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configure producer message buffer.\n\nTell Kafka producer how to buffer messages when EMQX has more messages to send than Kafka can keep up, or when Kafka is down." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.kafka.$name.kafka" - ], - "full_name" : "bridge_kafka:producer_kafka_opts", - "fields" : [ - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "topic", - "desc" : "Kafka topic name", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_kafka:kafka_message", - "kind" : "struct" - }, - "name" : "message", - "desc" : "Template to render a Kafka message.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "896KB", - "name" : "max_batch_bytes", - "desc" : "Maximum bytes to collect in a Kafka message batch. Most of the Kafka brokers default to a limit of 1 MB batch size. EMQX's default value is less than 1 MB in order to compensate Kafka message encoding overheads (especially when each individual message is very small). When a single message is over the limit, it is still sent (as a single element batch).", - "default" : { - "oneliner" : true, - "hocon" : "\"896KB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "no_compression", - "snappy", - "gzip" - ], - "kind" : "enum" - }, - "raw_default" : "no_compression", - "name" : "compression", - "desc" : "Compression method.", - "default" : { - "oneliner" : true, - "hocon" : "no_compression" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "random", - "key_dispatch" - ], - "kind" : "enum" - }, - "raw_default" : "random", - "name" : "partition_strategy", - "desc" : "Partition strategy is to tell the producer how to dispatch messages to Kafka partitions.\n\nrandom: Randomly pick a partition for each message\nkey_dispatch: Hash Kafka message key to a partition number", - "default" : { - "oneliner" : true, - "hocon" : "random" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "all_isr", - "leader_only", - "none" - ], - "kind" : "enum" - }, - "raw_default" : "all_isr", - "name" : "required_acks", - "desc" : "Required acknowledgements for Kafka partition leader to wait for its followers before it sends back the acknowledgement to EMQX Kafka producer\n\nall_isr: Require all in-sync replicas to acknowledge.\nleader_only: Require only the partition-leader's acknowledgement.\nnone: No need for Kafka to acknowledge at all.", - "default" : { - "oneliner" : true, - "hocon" : "all_isr" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_s()", - "kind" : "primitive" - }, - "raw_default" : "60s", - "name" : "partition_count_refresh_interval", - "desc" : "The time interval for Kafka producer to discover increased number of partitions.\nAfter the number of partitions is increased in Kafka, EMQX will start taking the \ndiscovered partitions into account when dispatching messages per partition_strategy.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 10, - "name" : "max_inflight", - "desc" : "Maximum number of batches allowed for Kafka producer (per-partition) to send before receiving acknowledgement from Kafka. Greater value typically means better throughput. However, there can be a risk of message reordering when this value is greater than 1.", - "default" : { - "oneliner" : true, - "hocon" : "10" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_kafka:producer_buffer", - "kind" : "struct" - }, - "name" : "buffer", - "desc" : "Configure producer message buffer.\n\nTell Kafka producer how to buffer messages when EMQX has more messages to send than Kafka can keep up, or when Kafka is down.", - "aliases" : [ - - ] - } - ], - "desc" : "Kafka producer configs." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.kafka.$name.socket_opts", - "bridges.kafka_consumer.$name.socket_opts" - ], - "full_name" : "bridge_kafka:socket_opts", - "fields" : [ - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "1MB", - "name" : "sndbuf", - "desc" : "Fine tune the socket send buffer. The default value is tuned for high throughput.", - "default" : { - "oneliner" : true, - "hocon" : "\"1MB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "1MB", - "name" : "recbuf", - "desc" : "Fine tune the socket receive buffer. The default value is tuned for high throughput.", - "default" : { - "oneliner" : true, - "hocon" : "\"1MB\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Extra socket options." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mongodb_rs.$name" - ], - "full_name" : "bridge_mongodb:mongodb_rs", - "fields" : [ - { - "type" : { - "name" : "rs", - "kind" : "singleton" - }, - "raw_default" : "rs", - "name" : "mongo_type", - "desc" : "Replica set. Must be set to 'rs' when MongoDB server is running in 'replica set' mode.", - "default" : { - "oneliner" : true, - "hocon" : "rs" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe MongoDB default port 27017 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "unsafe", - "safe" - ], - "kind" : "enum" - }, - "raw_default" : "unsafe", - "name" : "w_mode", - "desc" : "Write mode.", - "default" : { - "oneliner" : true, - "hocon" : "unsafe" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "master", - "slave_ok" - ], - "kind" : "enum" - }, - "raw_default" : "master", - "name" : "r_mode", - "desc" : "Read mode.", - "default" : { - "oneliner" : true, - "hocon" : "master" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "replica_set_name", - "desc" : "Name of the replica set.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "srv_record", - "desc" : "Use DNS SRV record.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "auth_source", - "desc" : "Database name associated with the user's credentials.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "topology", - "kind" : "struct" - }, - "name" : "topology", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this MongoDB Bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "mqtt", - "name" : "collection", - "desc" : "The collection where data will be stored into", - "default" : { - "oneliner" : true, - "hocon" : "\"mqtt\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "payload_template", - "desc" : "The template for formatting the outgoing messages. If undefined, rule engine will use JSON format to serialize all visible inputs, such as clientid, topic, payload etc.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "resource_schema:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "MongoDB (Replica Set) configuration" - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mongodb_sharded.$name" - ], - "full_name" : "bridge_mongodb:mongodb_sharded", - "fields" : [ - { - "type" : { - "name" : "sharded", - "kind" : "singleton" - }, - "raw_default" : "sharded", - "name" : "mongo_type", - "desc" : "Sharded cluster. Must be set to 'sharded' when MongoDB server is running in 'sharded' mode.", - "default" : { - "oneliner" : true, - "hocon" : "sharded" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe MongoDB default port 27017 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "unsafe", - "safe" - ], - "kind" : "enum" - }, - "raw_default" : "unsafe", - "name" : "w_mode", - "desc" : "Write mode.", - "default" : { - "oneliner" : true, - "hocon" : "unsafe" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "srv_record", - "desc" : "Use DNS SRV record.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "auth_source", - "desc" : "Database name associated with the user's credentials.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "topology", - "kind" : "struct" - }, - "name" : "topology", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this MongoDB Bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "mqtt", - "name" : "collection", - "desc" : "The collection where data will be stored into", - "default" : { - "oneliner" : true, - "hocon" : "\"mqtt\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "payload_template", - "desc" : "The template for formatting the outgoing messages. If undefined, rule engine will use JSON format to serialize all visible inputs, such as clientid, topic, payload etc.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "resource_schema:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "MongoDB (Sharded) configuration" - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mongodb_single.$name" - ], - "full_name" : "bridge_mongodb:mongodb_single", - "fields" : [ - { - "type" : { - "name" : "single", - "kind" : "singleton" - }, - "raw_default" : "single", - "name" : "mongo_type", - "desc" : "Standalone instance. Must be set to 'single' when MongoDB server is running in standalone mode.", - "default" : { - "oneliner" : true, - "hocon" : "single" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe MongoDB default port 27017 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "unsafe", - "safe" - ], - "kind" : "enum" - }, - "raw_default" : "unsafe", - "name" : "w_mode", - "desc" : "Write mode.", - "default" : { - "oneliner" : true, - "hocon" : "unsafe" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "srv_record", - "desc" : "Use DNS SRV record.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "auth_source", - "desc" : "Database name associated with the user's credentials.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "topology", - "kind" : "struct" - }, - "name" : "topology", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this MongoDB Bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "mqtt", - "name" : "collection", - "desc" : "The collection where data will be stored into", - "default" : { - "oneliner" : true, - "hocon" : "\"mqtt\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "payload_template", - "desc" : "The template for formatting the outgoing messages. If undefined, rule engine will use JSON format to serialize all visible inputs, such as clientid, topic, payload etc.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "resource_schema:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "MongoDB (Standalone) configuration" - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mqtt.$name" - ], - "full_name" : "bridge_mqtt:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_mqtt:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "cluster_shareload" - ], - "kind" : "enum" - }, - "raw_default" : "cluster_shareload", - "name" : "mode", - "desc" : "The mode of the MQTT Bridge.
\n- cluster_shareload: create an MQTT connection on each node in the emqx cluster.
\nIn 'cluster_shareload' mode, the incoming load from the remote broker is shared by\nusing shared subscription.
\nNote that the 'clientid' is suffixed by the node name, this is to avoid\nclientid conflicts between different nodes. And we can only use shared subscription\ntopic filters for remote.topic of ingress connections.", - "default" : { - "oneliner" : true, - "hocon" : "cluster_shareload" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The host and port of the remote MQTT broker", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "clientid_prefix", - "desc" : "Optional prefix to prepend to the clientid used by egress bridges.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "reconnect_interval", - "desc" : "Deprecated since v5.0.16.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "v3", - "v4", - "v5" - ], - "kind" : "enum" - }, - "raw_default" : "v4", - "name" : "proto_ver", - "desc" : "The MQTT protocol version", - "default" : { - "oneliner" : true, - "hocon" : "v4" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "bridge_mode", - "desc" : "If enable bridge mode.\nNOTE: This setting is only for MQTT protocol version older than 5.0, and the remote MQTT\nbroker MUST support this feature.\nIf bridge_mode is set to true, the bridge will indicate to the remote broker that it is a bridge not an ordinary client.\nThis means that loop detection will be more effective and that retained messages will be propagated correctly.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "The username of the MQTT protocol", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "The password of the MQTT protocol", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "clean_start", - "desc" : "Whether to start a clean session when reconnecting a remote broker for ingress bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "300s", - "name" : "keepalive", - "desc" : "MQTT Keepalive. Time interval is a string that contains a number followed by time unit:
- `ms` for milliseconds,\n- `s` for seconds,\n- `m` for minutes,\n- `h` for hours;\n
or combination of whereof: `1h5m0s`", - "default" : { - "oneliner" : true, - "hocon" : "\"300s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "retry_interval", - "desc" : "Message retry interval. Delay for the MQTT bridge to retry sending the QoS1/QoS2 messages in case of ACK not received. Time interval is a string that contains a number followed by time unit:
- `ms` for milliseconds,\n- `s` for seconds,\n- `m` for minutes,\n- `h` for hours;\n
or combination of whereof: `1h5m0s`", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 32, - "name" : "max_inflight", - "desc" : "Max inflight (sent, but un-acked) messages of the MQTT protocol", - "default" : { - "oneliner" : true, - "hocon" : "32" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "connector-mqtt:ingress", - "kind" : "struct" - }, - "name" : "ingress", - "desc" : "The ingress config defines how this bridge receive messages from the remote MQTT broker, and then\n send them to the local broker.
\n Template with variables is allowed in 'remote.qos', 'local.topic', 'local.qos', 'local.retain', 'local.payload'.
\n NOTE: if this bridge is used as the input of a rule, and also 'local.topic' is\n configured, then messages got from the remote broker will be sent to both the 'local.topic' and\n the rule.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "connector-mqtt:egress", - "kind" : "struct" - }, - "name" : "egress", - "desc" : "The egress config defines how this bridge forwards messages from the local broker to the remote broker.
\nTemplate with variables is allowed in 'remote.topic', 'local.qos', 'local.retain', 'local.payload'.
\nNOTE: if this bridge is used as the action of a rule, and also 'local.topic'\nis configured, then both the data got from the rule and the MQTT messages that matches\n'local.topic' will be forwarded.", - "aliases" : [ - - ] - } - ], - "desc" : "The config for MQTT Bridges." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mqtt.$name.resource_opts" - ], - "full_name" : "bridge_mqtt:creation_opts", - "fields" : [ - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "worker_pool_size", - "desc" : "The number of buffer workers. Only applicable for egress type bridges.\nFor bridges only have ingress direction data flow, it can be set to 0 otherwise must be greater than 0.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "health_check_interval", - "desc" : "Health check interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : "true", - "name" : "start_after_created", - "desc" : "Whether start the resource right after created.", - "default" : { - "oneliner" : true, - "hocon" : "\"true\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "start_timeout", - "desc" : "Time interval to wait for an auto-started resource to become healthy before responding resource creation requests.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "60s", - "name" : "auto_restart_interval", - "desc" : "The auto restart interval after the resource is disconnected.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "sync", - "async" - ], - "kind" : "enum" - }, - "raw_default" : "async", - "name" : "query_mode", - "desc" : "Query mode. Optional 'sync/async', default 'async'.", - "default" : { - "oneliner" : true, - "hocon" : "async" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "15s", - "name" : "request_timeout", - "desc" : "Starting from the moment when the request enters the buffer, if the request remains in the buffer for the specified time or is sent but does not receive a response or acknowledgement in time, the request is considered expired.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "inflight_window", - "desc" : "Query inflight window. When query_mode is set to async, this config has to be set to 1 if messages from the same MQTT client have to be strictly ordered.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - "async_inflight_window" - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_queue", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "256MB", - "name" : "max_buffer_bytes", - "desc" : "Maximum number of bytes to buffer for each buffer worker.", - "default" : { - "oneliner" : true, - "hocon" : "\"256MB\"" - }, - "aliases" : [ - "max_queue_bytes" - ] - } - ], - "desc" : "Creation options." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mysql.$name" - ], - "full_name" : "bridge_mysql:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "insert into t_mqtt_msg(msgid, topic, qos, payload, arrived) values (${id}, ${topic}, ${qos}, ${payload}, FROM_UNIXTIME(${timestamp}/1000))", - "name" : "sql", - "desc" : "SQL Template", - "default" : { - "oneliner" : true, - "hocon" : "\"insert into t_mqtt_msg(msgid, topic, qos, payload, arrived) values (${id}, ${topic}, ${qos}, ${payload}, FROM_UNIXTIME(${timestamp}/1000))\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to MySQL. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "resource_schema:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe MySQL default port 3306 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "root", - "name" : "username", - "desc" : "EMQX's username in the external database.", - "default" : { - "oneliner" : true, - "hocon" : "\"root\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for an HStreamDB bridge." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.matrix.$name", - "bridges.pgsql.$name", - "bridges.timescale.$name" - ], - "full_name" : "bridge_pgsql:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "insert into t_mqtt_msg(msgid, topic, qos, payload, arrived) values (${id}, ${topic}, ${qos}, ${payload}, TO_TIMESTAMP((${timestamp} :: bigint)/1000))", - "name" : "sql", - "desc" : "SQL Template", - "default" : { - "oneliner" : true, - "hocon" : "\"insert into t_mqtt_msg(msgid, topic, qos, payload, arrived) values (${id}, ${topic}, ${qos}, ${payload}, TO_TIMESTAMP((${timestamp} :: bigint)/1000))\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to PostgreSQL. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "resource_schema:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe PostgreSQL default port 5432 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "username", - "desc" : "EMQX's username in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a PostgreSQL bridge." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.redis_cluster.$name" - ], - "full_name" : "bridge_redis:redis_cluster", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to Redis. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "[binary()]", - "kind" : "primitive" - }, - "name" : "command_template", - "desc" : "Redis command template used to export messages. Each list element stands for a command name or its argument.\nFor example, to push payloads in a Redis list by key `msgs`, the elements should be the following:\n`rpush`, `msgs`, `${payload}`.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_redis:creation_opts_redis_cluster", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe Redis default port 6379 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "cluster", - "kind" : "singleton" - }, - "raw_default" : "cluster", - "name" : "redis_type", - "desc" : "Cluster mode. Must be set to 'cluster' when Redis server is running in clustered mode.", - "default" : { - "oneliner" : true, - "hocon" : "cluster" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Cluster mode. Must be set to 'cluster' when Redis server is running in clustered mode." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.redis_sentinel.$name" - ], - "full_name" : "bridge_redis:redis_sentinel", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to Redis. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "[binary()]", - "kind" : "primitive" - }, - "name" : "command_template", - "desc" : "Redis command template used to export messages. Each list element stands for a command name or its argument.\nFor example, to push payloads in a Redis list by key `msgs`, the elements should be the following:\n`rpush`, `msgs`, `${payload}`.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_redis:creation_opts_redis_sentinel", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "A Node list for Cluster to connect to. The nodes should be separated with commas, such as: `Node[,Node].`\nFor each Node should be: The IPv4 or IPv6 address or the hostname to connect to.\nA host entry has the following form: `Host[:Port]`.\nThe Redis default port 6379 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "sentinel", - "kind" : "singleton" - }, - "raw_default" : "sentinel", - "name" : "redis_type", - "desc" : "Sentinel mode. Must be set to 'sentinel' when Redis server is running in sentinel mode.", - "default" : { - "oneliner" : true, - "hocon" : "sentinel" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "sentinel", - "desc" : "The cluster name in Redis sentinel mode.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "database", - "desc" : "Redis database ID.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Sentinel mode. Must be set to 'sentinel' when Redis server is running in sentinel mode." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.redis_single.$name" - ], - "full_name" : "bridge_redis:redis_single", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to Redis. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "[binary()]", - "kind" : "primitive" - }, - "name" : "command_template", - "desc" : "Redis command template used to export messages. Each list element stands for a command name or its argument.\nFor example, to push payloads in a Redis list by key `msgs`, the elements should be the following:\n`rpush`, `msgs`, `${payload}`.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_redis:creation_opts_redis_single", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe Redis default port 6379 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "single", - "kind" : "singleton" - }, - "raw_default" : "single", - "name" : "redis_type", - "desc" : "Single mode. Must be set to 'single' when Redis server is running in single mode.", - "default" : { - "oneliner" : true, - "hocon" : "single" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "integer()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "database", - "desc" : "Redis database ID.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Single mode. Must be set to 'single' when Redis server is running in single mode." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.redis_cluster.$name.resource_opts" - ], - "full_name" : "bridge_redis:creation_opts_redis_cluster", - "fields" : [ - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "worker_pool_size", - "desc" : "The number of buffer workers. Only applicable for egress type bridges.\nFor bridges only have ingress direction data flow, it can be set to 0 otherwise must be greater than 0.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "health_check_interval", - "desc" : "Health check interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : "true", - "name" : "start_after_created", - "desc" : "Whether start the resource right after created.", - "default" : { - "oneliner" : true, - "hocon" : "\"true\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "start_timeout", - "desc" : "Time interval to wait for an auto-started resource to become healthy before responding resource creation requests.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "60s", - "name" : "auto_restart_interval", - "desc" : "The auto restart interval after the resource is disconnected.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "sync", - "async" - ], - "kind" : "enum" - }, - "raw_default" : "async", - "name" : "query_mode", - "desc" : "Query mode. Optional 'sync/async', default 'async'.", - "default" : { - "oneliner" : true, - "hocon" : "async" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "15s", - "name" : "request_timeout", - "desc" : "Starting from the moment when the request enters the buffer, if the request remains in the buffer for the specified time or is sent but does not receive a response or acknowledgement in time, the request is considered expired.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "inflight_window", - "desc" : "Query inflight window. When query_mode is set to async, this config has to be set to 1 if messages from the same MQTT client have to be strictly ordered.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - "async_inflight_window" - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_queue", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "256MB", - "name" : "max_buffer_bytes", - "desc" : "Maximum number of bytes to buffer for each buffer worker.", - "default" : { - "oneliner" : true, - "hocon" : "\"256MB\"" - }, - "aliases" : [ - "max_queue_bytes" - ] - } - ], - "desc" : "Creation options." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.redis_sentinel.$name.resource_opts" - ], - "full_name" : "bridge_redis:creation_opts_redis_sentinel", - "fields" : [ - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "worker_pool_size", - "desc" : "The number of buffer workers. Only applicable for egress type bridges.\nFor bridges only have ingress direction data flow, it can be set to 0 otherwise must be greater than 0.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "health_check_interval", - "desc" : "Health check interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : "true", - "name" : "start_after_created", - "desc" : "Whether start the resource right after created.", - "default" : { - "oneliner" : true, - "hocon" : "\"true\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "start_timeout", - "desc" : "Time interval to wait for an auto-started resource to become healthy before responding resource creation requests.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "60s", - "name" : "auto_restart_interval", - "desc" : "The auto restart interval after the resource is disconnected.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "sync", - "async" - ], - "kind" : "enum" - }, - "raw_default" : "async", - "name" : "query_mode", - "desc" : "Query mode. Optional 'sync/async', default 'async'.", - "default" : { - "oneliner" : true, - "hocon" : "async" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "15s", - "name" : "request_timeout", - "desc" : "Starting from the moment when the request enters the buffer, if the request remains in the buffer for the specified time or is sent but does not receive a response or acknowledgement in time, the request is considered expired.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "inflight_window", - "desc" : "Query inflight window. When query_mode is set to async, this config has to be set to 1 if messages from the same MQTT client have to be strictly ordered.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - "async_inflight_window" - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_batch", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 1, - "name" : "batch_size", - "desc" : "Maximum batch count. If equal to 1, there's effectively no batching.", - "default" : { - "oneliner" : true, - "hocon" : "1" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "0ms", - "name" : "batch_time", - "desc" : "Maximum waiting interval when accumulating a batch at a low message rates for more efficient resource usage.", - "default" : { - "oneliner" : true, - "hocon" : "\"0ms\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_queue", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "256MB", - "name" : "max_buffer_bytes", - "desc" : "Maximum number of bytes to buffer for each buffer worker.", - "default" : { - "oneliner" : true, - "hocon" : "\"256MB\"" - }, - "aliases" : [ - "max_queue_bytes" - ] - } - ], - "desc" : "Creation options." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.redis_single.$name.resource_opts" - ], - "full_name" : "bridge_redis:creation_opts_redis_single", - "fields" : [ - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "worker_pool_size", - "desc" : "The number of buffer workers. Only applicable for egress type bridges.\nFor bridges only have ingress direction data flow, it can be set to 0 otherwise must be greater than 0.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "health_check_interval", - "desc" : "Health check interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : "true", - "name" : "start_after_created", - "desc" : "Whether start the resource right after created.", - "default" : { - "oneliner" : true, - "hocon" : "\"true\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "start_timeout", - "desc" : "Time interval to wait for an auto-started resource to become healthy before responding resource creation requests.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "60s", - "name" : "auto_restart_interval", - "desc" : "The auto restart interval after the resource is disconnected.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "sync", - "async" - ], - "kind" : "enum" - }, - "raw_default" : "async", - "name" : "query_mode", - "desc" : "Query mode. Optional 'sync/async', default 'async'.", - "default" : { - "oneliner" : true, - "hocon" : "async" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "15s", - "name" : "request_timeout", - "desc" : "Starting from the moment when the request enters the buffer, if the request remains in the buffer for the specified time or is sent but does not receive a response or acknowledgement in time, the request is considered expired.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "inflight_window", - "desc" : "Query inflight window. When query_mode is set to async, this config has to be set to 1 if messages from the same MQTT client have to be strictly ordered.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - "async_inflight_window" - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_batch", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 1, - "name" : "batch_size", - "desc" : "Maximum batch count. If equal to 1, there's effectively no batching.", - "default" : { - "oneliner" : true, - "hocon" : "1" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "0ms", - "name" : "batch_time", - "desc" : "Maximum waiting interval when accumulating a batch at a low message rates for more efficient resource usage.", - "default" : { - "oneliner" : true, - "hocon" : "\"0ms\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_queue", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "256MB", - "name" : "max_buffer_bytes", - "desc" : "Maximum number of bytes to buffer for each buffer worker.", - "default" : { - "oneliner" : true, - "hocon" : "\"256MB\"" - }, - "aliases" : [ - "max_queue_bytes" - ] - } - ], - "desc" : "Creation options." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.rocketmq.$name" - ], - "full_name" : "bridge_rocketmq:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "template", - "desc" : "Template, the default value is empty. When this value is empty the whole message will be stored in the RocketMQ.
\n The template can be any valid string with placeholders, example:
\n - ${id}, ${username}, ${clientid}, ${timestamp}
\n - {\"id\" : ${id}, \"username\" : ${username}}", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to RocketMQ. All MQTT `PUBLISH` messages with the topic\nmatching the `local_topic` will be forwarded.
\nNOTE: if the bridge is used as a rule action, `local_topic` should be left empty otherwise the messages will be duplicated.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "resource_schema:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "servers", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe RocketMQ default port 9876 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "TopicTest", - "name" : "topic", - "desc" : "RocketMQ Topic", - "default" : { - "oneliner" : true, - "hocon" : "\"TopicTest\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "access_key", - "desc" : "RocketMQ server `accessKey`.", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "secret_key", - "desc" : "RocketMQ server `secretKey`.", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "", - "name" : "security_token", - "desc" : "RocketMQ Server Security Token", - "default" : { - "oneliner" : true, - "hocon" : "\"\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "3s", - "name" : "sync_timeout", - "desc" : "Timeout of RocketMQ driver synchronous call.", - "default" : { - "oneliner" : true, - "hocon" : "\"3s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "3s", - "name" : "refresh_interval", - "desc" : "RocketMQ Topic Route Refresh Interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"3s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "1024KB", - "name" : "send_buffer", - "desc" : "The socket send buffer size of the RocketMQ driver client.", - "default" : { - "oneliner" : true, - "hocon" : "\"1024KB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a RocketMQ bridge." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.sqlserver.$name" - ], - "full_name" : "bridge_sqlserver:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "insert into t_mqtt_msg(msgid, topic, qos, payload) values ( ${id}, ${topic}, ${qos}, ${payload} )", - "name" : "sql", - "desc" : "SQL Template", - "default" : { - "oneliner" : true, - "hocon" : "\"insert into t_mqtt_msg(msgid, topic, qos, payload) values ( ${id}, ${topic}, ${qos}, ${payload} )\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "ms-sql", - "name" : "driver", - "desc" : "SQL Server Driver Name", - "default" : { - "oneliner" : true, - "hocon" : "\"ms-sql\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to Microsoft SQL Server. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_sqlserver:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe SQL Server default port 1433 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "sa", - "name" : "username", - "desc" : "EMQX's username in the external database.", - "default" : { - "oneliner" : true, - "hocon" : "\"sa\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a Microsoft SQL Server bridge." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.sqlserver.$name.resource_opts" - ], - "full_name" : "bridge_sqlserver:creation_opts", - "fields" : [ - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "worker_pool_size", - "desc" : "The number of buffer workers. Only applicable for egress type bridges.\nFor bridges only have ingress direction data flow, it can be set to 0 otherwise must be greater than 0.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "health_check_interval", - "desc" : "Health check interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : "true", - "name" : "start_after_created", - "desc" : "Whether start the resource right after created.", - "default" : { - "oneliner" : true, - "hocon" : "\"true\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "start_timeout", - "desc" : "Time interval to wait for an auto-started resource to become healthy before responding resource creation requests.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "60s", - "name" : "auto_restart_interval", - "desc" : "The auto restart interval after the resource is disconnected.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "sync", - "async" - ], - "kind" : "enum" - }, - "raw_default" : "async", - "name" : "query_mode", - "desc" : "Query mode. Optional 'sync/async', default 'async'.", - "default" : { - "oneliner" : true, - "hocon" : "async" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "15s", - "name" : "request_timeout", - "desc" : "Starting from the moment when the request enters the buffer, if the request remains in the buffer for the specified time or is sent but does not receive a response or acknowledgement in time, the request is considered expired.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "inflight_window", - "desc" : "Query inflight window. When query_mode is set to async, this config has to be set to 1 if messages from the same MQTT client have to be strictly ordered.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - "async_inflight_window" - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_batch", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 1, - "name" : "batch_size", - "desc" : "Maximum batch count. If equal to 1, there's effectively no batching.", - "default" : { - "oneliner" : true, - "hocon" : "1" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "0ms", - "name" : "batch_time", - "desc" : "Maximum waiting interval when accumulating a batch at a low message rates for more efficient resource usage.", - "default" : { - "oneliner" : true, - "hocon" : "\"0ms\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_queue", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "256MB", - "name" : "max_buffer_bytes", - "desc" : "Maximum number of bytes to buffer for each buffer worker.", - "default" : { - "oneliner" : true, - "hocon" : "\"256MB\"" - }, - "aliases" : [ - "max_queue_bytes" - ] - } - ], - "desc" : "Creation options." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.tdengine.$name" - ], - "full_name" : "bridge_tdengine:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "insert into t_mqtt_msg(ts, msgid, mqtt_topic, qos, payload, arrived) values (${ts}, ${id}, ${topic}, ${qos}, ${payload}, ${timestamp})", - "name" : "sql", - "desc" : "SQL Template", - "default" : { - "oneliner" : true, - "hocon" : "\"insert into t_mqtt_msg(ts, msgid, mqtt_topic, qos, payload, arrived) values (${ts}, ${id}, ${topic}, ${qos}, ${payload}, ${timestamp})\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to TDengine. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "resource_schema:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "server", - "desc" : "The IPv4 or IPv6 address or the hostname to connect to.
\nA host entry has the following form: `Host[:Port]`.
\nThe TDengine default port 6041 is used if `[:Port]` is not specified.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "database", - "desc" : "Database name.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "Size of the connection pool towards the bridge target service.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "root", - "name" : "username", - "desc" : "EMQX's username in the external database.", - "default" : { - "oneliner" : true, - "hocon" : "\"root\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "password", - "desc" : "EMQX's password in the external database.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "auto_reconnect", - "desc" : "Deprecated since v5.0.15.", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for a TDengine bridge." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.webhook.$name" - ], - "full_name" : "bridge_webhook:config", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable or disable this bridge", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "bridge_webhook:creation_opts", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "resource_opts", - "desc" : "Resource options.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "connect_timeout", - "desc" : "The timeout when connecting to the HTTP server.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "name" : "retry_interval", - "desc" : "Deprecated since 5.0.4.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_connector_http:pool_type()", - "kind" : "primitive" - }, - "raw_default" : "random", - "name" : "pool_type", - "desc" : "The type of the pool. Can be one of `random`, `hash`.", - "default" : { - "oneliner" : true, - "hocon" : "random" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 8, - "name" : "pool_size", - "desc" : "The pool size.", - "default" : { - "oneliner" : true, - "hocon" : "8" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "enable_pipelining", - "desc" : "A positive integer. Whether to send HTTP requests continuously, when set to 1, it means that after each HTTP request is sent, you need to wait for the server to return and then continue to send the next request.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "connector-http:request", - "kind" : "struct" - }, - "name" : "request", - "desc" : "Configure HTTP request parameters.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "broker:ssl_client_opts", - "kind" : "struct" - }, - "raw_default" : { - "enable" : false - }, - "name" : "ssl", - "desc" : "SSL connection settings.", - "default" : { - "oneliner" : true, - "hocon" : "{enable = false}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "url", - "desc" : "The URL of the HTTP Bridge.
\nTemplate with variables is allowed in the path, but variables cannot be used in the scheme, host,\nor port part.
\nFor example, http://localhost:9901/${topic} is allowed, but\n http://${host}:9901/message or http://localhost:${port}/message \nis not allowed.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "egress", - "kind" : "singleton" - }, - "name" : "direction", - "desc" : "Deprecated since 5.0.12.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "local_topic", - "desc" : "The MQTT topic filter to be forwarded to the HTTP server. All MQTT 'PUBLISH' messages with the topic\nmatching the local_topic will be forwarded.
\nNOTE: if this bridge is used as the action of a rule (EMQX rule engine), and also local_topic is\nconfigured, then both the data got from the rule and the MQTT messages that match local_topic\nwill be forwarded.", - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "post", - "put", - "get", - "delete" - ], - "kind" : "enum" - }, - "raw_default" : "post", - "name" : "method", - "desc" : "The method of the HTTP request. All the available methods are: post, put, get, delete.
\nTemplate with variables is allowed.", - "default" : { - "oneliner" : true, - "hocon" : "post" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "raw_default" : { - "keep-alive" : "timeout=5", - "content-type" : "application/json", - "connection" : "keep-alive", - "cache-control" : "no-cache", - "accept" : "application/json" - }, - "name" : "headers", - "desc" : "The headers of the HTTP request.
\nTemplate with variables is allowed.", - "default" : { - "oneliner" : false, - "hocon" : "{\n accept = \"application/json\"\n \"cache-control\" = \"no-cache\"\n connection = \"keep-alive\"\n \"content-type\" = \"application/json\"\n \"keep-alive\" = \"timeout=5\"\n}\n" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "body", - "desc" : "The body of the HTTP request.
\nIf not provided, the body will be a JSON object of all the available fields.
\nThere, 'all the available fields' means the context of a MQTT message when\nthis webhook is triggered by receiving a MQTT message (the `local_topic` is set),\nor the context of the event when this webhook is triggered by a rule (i.e. this\nwebhook is used as an action of a rule).
\nTemplate with variables is allowed.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 2, - "name" : "max_retries", - "desc" : "HTTP request max retry times if failed.", - "default" : { - "oneliner" : true, - "hocon" : "2" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "request_timeout", - "desc" : "HTTP request timeout.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration for an HTTP bridge." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.webhook.$name.resource_opts" - ], - "full_name" : "bridge_webhook:creation_opts", - "fields" : [ - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "worker_pool_size", - "desc" : "The number of buffer workers. Only applicable for egress type bridges.\nFor bridges only have ingress direction data flow, it can be set to 0 otherwise must be greater than 0.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "health_check_interval", - "desc" : "Health check interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : "true", - "name" : "start_after_created", - "desc" : "Whether start the resource right after created.", - "default" : { - "oneliner" : true, - "hocon" : "\"true\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "start_timeout", - "desc" : "Time interval to wait for an auto-started resource to become healthy before responding resource creation requests.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "60s", - "name" : "auto_restart_interval", - "desc" : "The auto restart interval after the resource is disconnected.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "sync", - "async" - ], - "kind" : "enum" - }, - "raw_default" : "async", - "name" : "query_mode", - "desc" : "Query mode. Optional 'sync/async', default 'async'.", - "default" : { - "oneliner" : true, - "hocon" : "async" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "15s", - "name" : "request_timeout", - "desc" : "Starting from the moment when the request enters the buffer, if the request remains in the buffer for the specified time or is sent but does not receive a response or acknowledgement in time, the request is considered expired.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "inflight_window", - "desc" : "Query inflight window. When query_mode is set to async, this config has to be set to 1 if messages from the same MQTT client have to be strictly ordered.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - "async_inflight_window" - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_queue", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "256MB", - "name" : "max_buffer_bytes", - "desc" : "Maximum number of bytes to buffer for each buffer worker.", - "default" : { - "oneliner" : true, - "hocon" : "\"256MB\"" - }, - "aliases" : [ - "max_queue_bytes" - ] - } - ], - "desc" : "Creation options." - }, - { - "tags" : [ - - ], - "paths" : [ - "authentication.$INDEX.request", - "authorization.sources.$INDEX.request", - "bridges.webhook.$name.request" - ], - "full_name" : "connector-http:request", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "method", - "desc" : "HTTP method.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "path", - "desc" : "URL path.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "body", - "desc" : "HTTP request body.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "name" : "headers", - "desc" : "List of HTTP headers.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "name" : "max_retries", - "desc" : "Max retry times if error on sending request.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "name" : "request_timeout", - "desc" : "HTTP request timeout.", - "aliases" : [ - - ] - } - ], - "desc" : "" - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mqtt.$name.egress" - ], - "full_name" : "connector-mqtt:egress", - "fields" : [ - { - "type" : { - "name" : "connector-mqtt:egress_local", - "kind" : "struct" - }, - "name" : "local", - "desc" : "The configs about receiving messages from local broker.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "connector-mqtt:egress_remote", - "kind" : "struct" - }, - "name" : "remote", - "desc" : "The configs about sending message to the remote broker.", - "aliases" : [ - - ] - } - ], - "desc" : "The egress config defines how this bridge forwards messages from the local broker to the remote broker.
\nTemplate with variables is allowed in 'remote.topic', 'local.qos', 'local.retain', 'local.payload'.
\nNOTE: if this bridge is used as the action of a rule, and also 'local.topic'\nis configured, then both the data got from the rule and the MQTT messages that matches\n'local.topic' will be forwarded." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mqtt.$name.egress.local" - ], - "full_name" : "connector-mqtt:egress_local", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "topic", - "desc" : "The local topic to be forwarded to the remote broker", - "aliases" : [ - - ] - } - ], - "desc" : "The configs about receiving messages from local broker." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mqtt.$name.egress.remote" - ], - "full_name" : "connector-mqtt:egress_remote", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "topic", - "desc" : "Forward to which topic of the remote broker.
\nTemplate with variables is allowed.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "qos()", - "kind" : "primitive" - }, - { - "name" : "binary()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : 1, - "name" : "qos", - "desc" : "The QoS of the MQTT message to be sent.
\nTemplate with variables is allowed.", - "default" : { - "oneliner" : true, - "hocon" : "1" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "boolean()", - "kind" : "primitive" - }, - { - "name" : "binary()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : false, - "name" : "retain", - "desc" : "The 'retain' flag of the MQTT message to be sent.
\nTemplate with variables is allowed.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "payload", - "desc" : "The payload of the MQTT message to be sent.
\nTemplate with variables is allowed.", - "aliases" : [ - - ] - } - ], - "desc" : "The configs about sending message to the remote broker." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mqtt.$name.ingress" - ], - "full_name" : "connector-mqtt:ingress", - "fields" : [ - { - "type" : { - "name" : "connector-mqtt:ingress_remote", - "kind" : "struct" - }, - "name" : "remote", - "desc" : "The configs about subscribing to the remote broker.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "connector-mqtt:ingress_local", - "kind" : "struct" - }, - "name" : "local", - "desc" : "The configs about sending message to the local broker.", - "aliases" : [ - - ] - } - ], - "desc" : "The ingress config defines how this bridge receive messages from the remote MQTT broker, and then\n send them to the local broker.
\n Template with variables is allowed in 'remote.qos', 'local.topic', 'local.qos', 'local.retain', 'local.payload'.
\n NOTE: if this bridge is used as the input of a rule, and also 'local.topic' is\n configured, then messages got from the remote broker will be sent to both the 'local.topic' and\n the rule." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mqtt.$name.ingress.local" - ], - "full_name" : "connector-mqtt:ingress_local", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "topic", - "desc" : "Send messages to which topic of the local broker.
\nTemplate with variables is allowed.", - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "qos()", - "kind" : "primitive" - }, - { - "name" : "binary()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "${qos}", - "name" : "qos", - "desc" : "The QoS of the MQTT message to be sent.
\nTemplate with variables is allowed.", - "default" : { - "oneliner" : true, - "hocon" : "\"${qos}\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "boolean()", - "kind" : "primitive" - }, - { - "name" : "binary()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "${retain}", - "name" : "retain", - "desc" : "The 'retain' flag of the MQTT message to be sent.
\nTemplate with variables is allowed.", - "default" : { - "oneliner" : true, - "hocon" : "\"${retain}\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "payload", - "desc" : "The payload of the MQTT message to be sent.
\nTemplate with variables is allowed.", - "aliases" : [ - - ] - } - ], - "desc" : "The configs about sending message to the local broker." - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.mqtt.$name.ingress.remote" - ], - "full_name" : "connector-mqtt:ingress_remote", - "fields" : [ - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "name" : "topic", - "desc" : "Receive messages from which topic of the remote broker", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "qos()", - "kind" : "primitive" - }, - "raw_default" : 1, - "name" : "qos", - "desc" : "The QoS level to be used when subscribing to the remote broker", - "default" : { - "oneliner" : true, - "hocon" : "1" - }, - "aliases" : [ - - ] - } - ], - "desc" : "The configs about subscribing to the remote broker." - }, - { - "tags" : [ - - ], - "paths" : [ - "plugins" - ], - "full_name" : "plugin:plugins", - "fields" : [ - { - "type" : { - "kind" : "array", - "elements" : { - "name" : "plugin:state", - "kind" : "struct" - } - }, - "raw_default" : [ - - ], - "name" : "states", - "desc" : "An array of plugins in the desired states.
\nThe plugins are started in the defined order", - "default" : { - "oneliner" : true, - "hocon" : "[]" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "plugins", - "name" : "install_dir", - "desc" : "The installation directory for the external plugins.\nThe plugin beam files and configuration files should reside in\nthe subdirectory named as emqx_foo_bar-0.1.0.\n
\nNOTE: For security reasons, this directory should **NOT** be writable\nby anyone except emqx (or any user which runs EMQX).", - "default" : { - "oneliner" : true, - "hocon" : "\"plugins\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "check_interval", - "desc" : "Check interval: check if the status of the plugins in the cluster is consistent,
\nif the results of 3 consecutive checks are not consistent, then alarm.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Manage EMQX plugins.
\nPlugins can be pre-built as a part of EMQX package,\nor installed as a standalone package in a location specified by\ninstall_dir config key
\nThe standalone-installed plugins are referred to as 'external' plugins." - }, - { - "tags" : [ - - ], - "paths" : [ - "plugins.states.$INDEX" - ], - "full_name" : "plugin:state", - "fields" : [ - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "name" : "name_vsn", - "desc" : "The {name}-{version} of the plugin.
\nIt should match the plugin application name-version as the for the plugin release package name
\nFor example: my_plugin-0.1.0.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable", - "desc" : "Set to 'true' to enable this plugin", - "aliases" : [ - - ] - } - ], - "desc" : "A per-plugin config to describe the desired state of the plugin." - }, - { - "tags" : [ - - ], - "paths" : [ - "prometheus" - ], - "full_name" : "prometheus", - "fields" : [ - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "http://127.0.0.1:9091", - "name" : "push_gateway_server", - "desc" : "URL of Prometheus server", - "default" : { - "oneliner" : true, - "hocon" : "\"http://127.0.0.1:9091\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "interval", - "desc" : "Data reporting interval", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "[{string(), string()}]", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "headers", - "desc" : "A list of HTTP Headers when pushing to Push Gateway.
\nFor example, { Authorization = \"some-authz-tokens\"}", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "binary()", - "kind" : "primitive" - }, - "raw_default" : "${name}/instance/${name}~${host}", - "name" : "job_name", - "desc" : "Job Name that is pushed to the Push Gateway. Available variables:
\n- ${name}: Name of EMQX node.
\n- ${host}: Host name of EMQX node.
\nFor example, when the EMQX node name is emqx@127.0.0.1 then the name variable takes value emqx and the host variable takes value 127.0.0.1.
\nDefault value is: ${name}/instance/${name}~${host}", - "default" : { - "oneliner" : true, - "hocon" : "\"${name}/instance/${name}~${host}\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "enable", - "desc" : "Turn Prometheus data pushing on or off", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Settings for reporting metrics to Prometheus" - }, - { - "tags" : [ - - ], - "paths" : [ - "bridges.cassandra.$name.resource_opts", - "bridges.gcp_pubsub.$name.resource_opts", - "bridges.influxdb_api_v1.$name.resource_opts", - "bridges.influxdb_api_v2.$name.resource_opts", - "bridges.matrix.$name.resource_opts", - "bridges.mongodb_rs.$name.resource_opts", - "bridges.mongodb_sharded.$name.resource_opts", - "bridges.mongodb_single.$name.resource_opts", - "bridges.mysql.$name.resource_opts", - "bridges.pgsql.$name.resource_opts", - "bridges.rocketmq.$name.resource_opts", - "bridges.tdengine.$name.resource_opts", - "bridges.timescale.$name.resource_opts" - ], - "full_name" : "resource_schema:creation_opts", - "fields" : [ - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 16, - "name" : "worker_pool_size", - "desc" : "The number of buffer workers. Only applicable for egress type bridges.\nFor bridges only have ingress direction data flow, it can be set to 0 otherwise must be greater than 0.", - "default" : { - "oneliner" : true, - "hocon" : "16" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "15s", - "name" : "health_check_interval", - "desc" : "Health check interval.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : "true", - "name" : "start_after_created", - "desc" : "Whether start the resource right after created.", - "default" : { - "oneliner" : true, - "hocon" : "\"true\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "5s", - "name" : "start_timeout", - "desc" : "Time interval to wait for an auto-started resource to become healthy before responding resource creation requests.", - "default" : { - "oneliner" : true, - "hocon" : "\"5s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "60s", - "name" : "auto_restart_interval", - "desc" : "The auto restart interval after the resource is disconnected.", - "default" : { - "oneliner" : true, - "hocon" : "\"60s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "sync", - "async" - ], - "kind" : "enum" - }, - "raw_default" : "async", - "name" : "query_mode", - "desc" : "Query mode. Optional 'sync/async', default 'async'.", - "default" : { - "oneliner" : true, - "hocon" : "async" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "members" : [ - { - "name" : "infinity", - "kind" : "singleton" - }, - { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - } - ], - "kind" : "union" - }, - "raw_default" : "15s", - "name" : "request_timeout", - "desc" : "Starting from the moment when the request enters the buffer, if the request remains in the buffer for the specified time or is sent but does not receive a response or acknowledgement in time, the request is considered expired.", - "default" : { - "oneliner" : true, - "hocon" : "\"15s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 100, - "name" : "inflight_window", - "desc" : "Query inflight window. When query_mode is set to async, this config has to be set to 1 if messages from the same MQTT client have to be strictly ordered.", - "default" : { - "oneliner" : true, - "hocon" : "100" - }, - "aliases" : [ - "async_inflight_window" - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_batch", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "pos_integer()", - "kind" : "primitive" - }, - "raw_default" : 1, - "name" : "batch_size", - "desc" : "Maximum batch count. If equal to 1, there's effectively no batching.", - "default" : { - "oneliner" : true, - "hocon" : "1" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "0ms", - "name" : "batch_time", - "desc" : "Maximum waiting interval when accumulating a batch at a low message rates for more efficient resource usage.", - "default" : { - "oneliner" : true, - "hocon" : "\"0ms\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "name" : "enable_queue", - "desc" : "Deprecated since v5.0.14.", - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "256MB", - "name" : "max_buffer_bytes", - "desc" : "Maximum number of bytes to buffer for each buffer worker.", - "default" : { - "oneliner" : true, - "hocon" : "\"256MB\"" - }, - "aliases" : [ - "max_queue_bytes" - ] - } - ], - "desc" : "Creation options." - }, - { - "tags" : [ - - ], - "paths" : [ - "retainer.flow_control" - ], - "full_name" : "retainer:flow_control", - "fields" : [ - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "batch_read_number", - "desc" : "Size of the batch when reading messages from storage. 0 means no limit.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "0..1000", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "batch_deliver_number", - "desc" : "The number of retained messages can be delivered per batch.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "limiter:internal", - "kind" : "struct" - }, - "name" : "batch_deliver_limiter", - "desc" : "The rate limiter name for retained messages' delivery.\nLimiter helps to avoid delivering too many messages to the client at once, which may cause the client to block or crash, or drop messages due to exceeding the size of the message queue.\nThe names of the available rate limiters are taken from the existing rate limiters under `limiter.batch`.\nIf this field is empty, limiter is not used.", - "aliases" : [ - - ] - } - ], - "desc" : "Retainer batching and rate limiting." - }, - { - "tags" : [ - - ], - "paths" : [ - "retainer.backend" - ], - "full_name" : "retainer:mnesia_config", - "fields" : [ - { - "type" : { - "name" : "built_in_database", - "kind" : "singleton" - }, - "raw_default" : "built_in_database", - "name" : "type", - "desc" : "Backend type.", - "default" : { - "oneliner" : true, - "hocon" : "built_in_database" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "symbols" : [ - "ram", - "disc" - ], - "kind" : "enum" - }, - "raw_default" : "ram", - "name" : "storage_type", - "desc" : "Specifies whether the messages are stored in RAM or persisted on disc.", - "default" : { - "oneliner" : true, - "hocon" : "ram" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "non_neg_integer()", - "kind" : "primitive" - }, - "raw_default" : 0, - "name" : "max_retained_messages", - "desc" : "Maximum number of retained messages. 0 means no limit.", - "default" : { - "oneliner" : true, - "hocon" : "0" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "[[integer()]]", - "kind" : "primitive" - }, - "raw_default" : [ - [ - 1, - 2, - 3 - ], - [ - 1, - 3 - ], - [ - 2, - 3 - ], - [ - 3 - ] - ], - "name" : "index_specs", - "examples" : [ - [ - [ - 2, - 4 - ], - [ - 1, - 3 - ] - ] - ], - "desc" : "Retainer index specifications: list of arrays of positive ascending integers. Each array specifies an index. Numbers in an index specification are 1-based word positions in topics. Words from specified positions will be used for indexing.
For example, it is good to have [2, 4] index to optimize +/X/+/Y/... topic wildcard subscriptions.", - "default" : { - "oneliner" : false, - "hocon" : "[\n [1, 2, 3],\n [1, 3],\n [2, 3],\n [3]\n]\n" - }, - "aliases" : [ - - ] - } - ], - "desc" : "Configuration of the internal database storing retained messages." - }, - { - "tags" : [ - - ], - "paths" : [ - "retainer" - ], - "full_name" : "retainer", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : true, - "name" : "enable", - "desc" : "Enable retainer feature", - "default" : { - "oneliner" : true, - "hocon" : "true" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "0s", - "name" : "msg_expiry_interval", - "desc" : "Message retention time. 0 means message will never be expired.", - "default" : { - "oneliner" : true, - "hocon" : "\"0s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "0s", - "name" : "msg_clear_interval", - "desc" : "Periodic interval for cleaning up expired messages.\nNever clear if the value is 0.", - "default" : { - "oneliner" : true, - "hocon" : "\"0s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "retainer:flow_control", - "kind" : "struct" - }, - "raw_default" : { - - }, - "name" : "flow_control", - "desc" : "Flow control.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:bytesize()", - "kind" : "primitive" - }, - "raw_default" : "1MB", - "name" : "max_payload_size", - "desc" : "Maximum retained message size.", - "default" : { - "oneliner" : true, - "hocon" : "\"1MB\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "stop_publish_clear_msg", - "desc" : "When the retained flag of the `PUBLISH` message is set and Payload is empty,\nwhether to continue to publish the message.\nSee:\nhttp://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718038", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "retainer:mnesia_config", - "kind" : "struct" - }, - "name" : "backend", - "desc" : "Settings for the database storing the retained messages.", - "aliases" : [ - - ] - } - ], - "desc" : "Configuration related to handling `PUBLISH` packets with a `retain` flag set to 1." - }, - { - "tags" : [ - - ], - "paths" : [ - "statsd" - ], - "full_name" : "statsd", - "fields" : [ - { - "type" : { - "name" : "boolean()", - "kind" : "primitive" - }, - "raw_default" : false, - "name" : "enable", - "desc" : "Enable or disable StatsD metrics collection and push service.", - "default" : { - "oneliner" : true, - "hocon" : "false" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "string()", - "kind" : "primitive" - }, - "raw_default" : "127.0.0.1:8125", - "name" : "server", - "desc" : "StatsD server address.", - "default" : { - "oneliner" : true, - "hocon" : "\"127.0.0.1:8125\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "30s", - "name" : "sample_time_interval", - "desc" : "The sampling interval for metrics.", - "default" : { - "oneliner" : true, - "hocon" : "\"30s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "emqx_schema:duration_ms()", - "kind" : "primitive" - }, - "raw_default" : "30s", - "name" : "flush_time_interval", - "desc" : "The push interval for metrics.", - "default" : { - "oneliner" : true, - "hocon" : "\"30s\"" - }, - "aliases" : [ - - ] - }, - { - "type" : { - "name" : "map()", - "kind" : "primitive" - }, - "raw_default" : { - - }, - "name" : "tags", - "desc" : "The tags for metrics.", - "default" : { - "oneliner" : true, - "hocon" : "{}" - }, - "aliases" : [ - - ] - } - ], - "desc" : "StatsD metrics collection and push configuration." - } -] \ No newline at end of file From 9feec7d21346c4c751dec946229d06e3f274bd84 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 09:43:45 +0800 Subject: [PATCH 11/49] update(S3 Tables): Support for Parquet file format Import changes in https://github.com/emqx/emqx-docs/pull/3071/files to v5.10.1 --- en_US/data-integration/s3-tables.md | 3 +++ zh_CN/data-integration/s3-tables.md | 5 ++++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/en_US/data-integration/s3-tables.md b/en_US/data-integration/s3-tables.md index 0980b2b8b..c4120df79 100644 --- a/en_US/data-integration/s3-tables.md +++ b/en_US/data-integration/s3-tables.md @@ -171,6 +171,9 @@ This section demonstrates how to create a rule in EMQX to process messages from - **Table**: The name of the Iceberg table to append data to (e.g., `testtable`). - **Max Records**: The maximum number of records to batch before writing to S3. Once reached, the batch is flushed and uploaded immediately. - **Time Interval**: The maximum time (in milliseconds) to wait before flushing data, even if the record count has not reached the Max Records threshold. + - **Data File Format**: Defines the format of the data file used to store batched MQTT messages in S3. Supported values: + - `avro`: (Default) Stores records in Avro format, which is row-based and ideal for streaming data and evolving schemas. + - `parquet`: Stores records in Apache Parquet format, which is column-based and optimized for analytical queries over large datasets. 8. **Fallback Actions (Optional)**: If you want to improve reliability in case of message delivery failure, you can define one or more fallback actions. These actions will be triggered if the primary Sink fails to process a message. See [Fallback Actions](./data-bridges.md#fallback-actions) for more details. diff --git a/zh_CN/data-integration/s3-tables.md b/zh_CN/data-integration/s3-tables.md index f42b0c4a4..122e332b4 100644 --- a/zh_CN/data-integration/s3-tables.md +++ b/zh_CN/data-integration/s3-tables.md @@ -107,7 +107,7 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 ::: tip 在执行查询 SQL 之前，请确保在 Athena 中已选择正确的**目录**和**数据库**，以确保数据表被创建在正确的 S3 表存储桶中。 - ::: + ::: ## 创建连接器 @@ -168,6 +168,9 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 - **表**：要写入的 Iceberg 表名称。 - **最大记录数**：当达到该数量时，当前数据会被聚合成一个文件上传，并重置时间间隔。 - **时间间隔**：达到设定时间后，无论记录数是否达到上限，当前批次也将被上传并重置计数器。 + - **数据文件格式**：用于定义存储批量 MQTT 消息的数据文件格式。支持的格式包括： + - `avro`：默认值，以 Avro 格式存储记录，采用行式存储，适用于流式数据和可演化的数据结构。 + - `parquet`：以 Apache Parquet 格式存储记录，采用列式存储，适合对大规模数据集进行高效的分析查询。 8. **备选动作（可选）**：如果您希望在消息投递失败时提升系统的可靠性，可以为 Sink 配置一个或多个备选动作。当 Sink 无法成功处理消息时，这些备选动作将被触发。更多信息请参见：[备选动作](./data-bridges.md#备选动作)。 From ddbdaa16e5de2c4ca036a6ff0fa815c9f6defd10 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Tue, 19 Aug 2025 10:32:51 +0800 Subject: [PATCH 12/49] update(S3): Update config method for access keys https://emqx.atlassian.net/browse/EMQX-14526 --- en_US/data-integration/s3-tables.md | 53 +++++++++++++++----- en_US/data-integration/s3.md | 25 +++++++-- zh_CN/data-integration/s3-tables.md | 78 ++++++++++++++++++++--------- zh_CN/data-integration/s3.md | 32 ++++++++++-- 4 files changed, 143 insertions(+), 45 deletions(-) diff --git a/en_US/data-integration/s3-tables.md b/en_US/data-integration/s3-tables.md index 0980b2b8b..df3a57b2a 100644 --- a/en_US/data-integration/s3-tables.md +++ b/en_US/data-integration/s3-tables.md @@ -8,7 +8,7 @@ This page provides a detailed introduction to the data integration between EMQX ## How It Works -EMQX integrates with Amazon S3 Tables to enable real-time, structured ingestion of MQTT data into Amazon S3 for long-term storage and analytics. This integration leverages EMQX’s rule engine and S3 Tables Sink to transform and stream MQTT messages directly into Apache Iceberg-formatted tables stored in S3 Table Buckets. +EMQX’s Amazon S3 Tables integration is an out-of-the-box feature. This integration leverages EMQX’s rule engine and S3 Tables Sink to transform and stream MQTT messages directly into Apache Iceberg-formatted tables stored in S3 Table Buckets for long-term storage and downstream analysis. In a typical IoT scenario: @@ -52,12 +52,35 @@ Before proceeding, make sure you are familiar with the following: If you're new to AWS S3 Tables, review the following key terms: +- **EC2**: AWS’s virtual machine service (compute instances). +- **IAM**: AWS Identity and Access Management; an instance role can issue temporary credentials to programs running on that instance. +- **IMDSv2**: EC2’s Instance Metadata Service v2 for retrieving metadata/temporary credentials; token-based and more secure. - **Table Bucket**: A specialized S3 bucket used for storing Iceberg-based table data and metadata in S3 Tables. - **Amazon Athena**: A serverless query engine that lets you run SQL queries directly on data stored in Amazon S3. Athena supports standard SQL syntax, including Data Definition Language (DDL) statements such as `CREATE TABLE` to define schema and structure for querying. - **Catalog**: A metadata container in Athena that organizes databases (namespaces) and tables. - **Database (Namespace)**: A logical group of tables under a catalog. - **Iceberg Table**: A high-performance, transactional table format for data lakes. It supports schema evolution, partition pruning, and time travel queries. +### Deployment Prerequisites and Credential Sources + +The S3 Tables connector supports two ways to obtain credentials. Choose based on your EMQX deployment environment: + +- **Option 1: Manually configure access keys** + When [creating a connector](#create-a-connector), you need to provide the **Access Key ID** and **Secret Access Key**. These credentials must have the required permissions for the target S3 Tables and Athena. Suitable for local/container/Kubernetes/non-AWS clouds, or EC2 without an attached instance role. + + To create and manage access keys for an IAM user, see the [AWS documentation on managing access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). + +- **Option 2: Automatically obtain temporary credentials (EC2 only, since EMQX 5.10.1)** + If EMQX runs on an AWS EC2 instance and the instance has an attached IAM role with the necessary permissions, you can leave **Access Key ID** and **Secret Access Key** blank in the connector. EMQX will use IMDSv2 to fetch temporary credentials associated with that role. + + To learn how to assign an IAM role to an EC2 instance, see the [AWS documentation on IAM roles for Amazon EC2](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html). + +::: tip Note + +- Ensure the instance role has sufficient permissions to the target S3 Tables (bucket/table) and Athena; otherwise, **Test Connectivity** may fail. +- It’s recommended to enforce IMDSv2-only on the instance; if you are not on EC2 or no role is attached, use **Option 1** and enter access keys manually. + ::: + ### Prepare an S3 Tables Bucket Before creating a Sink in EMQX, you need to prepare the destination of MQTT data in AWS S3 Tables, including: @@ -66,7 +89,7 @@ Before creating a Sink in EMQX, you need to prepare the destination of MQTT data - A Namespace to logically group related tables. - An Iceberg-based Table to receive structured MQTT data. -1. Log into the AWS Management Console. +1. Log in to the AWS Management Console. 1. Go to the S3 service. In the left navigation pane, click **Table buckets**. @@ -117,10 +140,12 @@ Before adding the S3 Tables Sink, you need to create the corresponding connector 1. Go to the Dashboard **Integration** -> **Connector** page. 2. Click the **Create** button in the top right corner. 3. Select **S3 Tables** as the connector type and click next. -4. Enter the connector name, a combination of upper and lowercase letters and numbers. Here, enter `my-s3-tables`. +4. Enter a name for the connector. The name must start with a letter or number and can contain letters, numbers, hyphens, or underscores. In this example, enter `my-s3-tables`. 5. Provide the required connection details: - **S3Tables ARN**: Enter the Amazon Resource Name (ARN) of your S3 Table Bucket. You can find this in the Table buckets section in the AWS Console. - - **Access Key ID** and **Secret Access Key**: Enter the AWS access credentials associated with an IAM user or role that has permission to access S3 Tables and Athena. + - **Access Key ID and Secret Access Key** (optional): + - **Manual configuration:** Enter AWS credentials associated with an IAM user or role that has permission to access S3 Tables and Athena. + - **Automatic retrieval (since EMQX 5.10.1):** If EMQX is deployed on an AWS EC2 instance and the instance is associated with an IAM role that has the required permissions, you can leave this field blank. EMQX will automatically obtain temporary credentials through IMDSv2. See [Deployment Prerequisites and Credential Sources](#deployment-prerequisites-and-credential-sources) for details. - **Enable TLS**: TLS is enabled by default when connecting to S3 Tables. For detailed TLS connection options, see [TLS for External Resource Access](../network/overview.md#enable-tls-encryption-for-accessing-external-resources). - **Health Check Timeout**: Specify the timeout duration for the connector to perform automatic health checks on its connection with S3 Tables. 7. Use the default values for the remaining settings. @@ -216,11 +241,15 @@ This section shows how to test the rule configured with the S3 Tables Sink. This section delves into the advanced configuration options available for the S3 Tables Sink. In the Dashboard, when configuring the Sink, you can expand **Advanced Settings** to adjust the following parameters based on your specific needs. -| Field Name | Description | Default Value | -| ------------------------- | ------------------------------------------------------------ | -------------- | -| **Buffer Pool Size** | Specifies the number of buffer worker processes, which are allocated to manage the data flow between EMQX and S3 Tables. These workers temporarily store and process data before sending it to the target service, crucial for optimizing performance and ensuring smooth data transmission. | `16` | -| **Request TTL** | The "Request TTL" (Time To Live) configuration setting specifies the maximum duration, in seconds, that a request is considered valid once it enters the buffer. This timer starts ticking from the moment the request is buffered. If the request stays in the buffer for a period exceeding this TTL setting or if it is sent but does not receive a timely response or acknowledgment from S3 Tables, the request is deemed to have expired. | | -| **Health Check Interval** | Specifies the time interval (in seconds) for the Sink to perform automatic health checks on its connection with S3 Tables. | `15` | -| **Max Buffer Queue Size** | Specifies the maximum number of bytes that can be buffered by each buffer worker process in the S3 Tables Sink. The buffer workers temporarily store data before sending it to S3 Tables, acting as intermediaries to handle the data stream more efficiently. Adjust this value based on system performance and data transmission requirements. | `256` | -| **Query Mode** | Allows you to choose between `synchronous` or `asynchronous` request modes to optimize message transmission according to different requirements. In asynchronous mode, writing to S3 Tables does not block the MQTT message publishing process. However, this may lead to clients receiving messages before they arrive at S3 Tables. | `Asynchronous` | -| **In-flight Window** | "In-flight queue requests" refer to requests that have been initiated but have not yet received a response or acknowledgment. This setting controls the maximum number of in-flight queue requests that can exist simultaneously during Sink communication with S3 Tables.
When **Request Mode** is set to `asynchronous`, the "Request In-flight Queue Window" parameter becomes particularly important. If strict sequential processing of messages from the same MQTT client is crucial, then this value should be set to `1`. | `100` | +| Field Name | Description | Default Value | +| -------------------------------- | ------------------------------------------------------------ | -------------- | +| **Min Part Size** | The minimum part size for multipart uploads.
Uploaded data will be accumulated in memory until this size is reached. | `5` MB | +| **Max Part Size** | The maximum part size for multipart uploads.
S3 uploader won't try to upload parts larger than this size. | `5` GB | +| **Buffer Pool Size** | Specifies the number of buffer worker processes, which are allocated to manage the data flow between EMQX and S3 Tables. These workers temporarily store and process data before sending it to the target service, crucial for optimizing performance and ensuring smooth data transmission. | `16` | +| **Request TTL** | The "Request TTL" (Time To Live) configuration setting specifies the maximum duration, in seconds, that a request is considered valid once it enters the buffer. This timer starts ticking from the moment the request is buffered. If the request stays in the buffer for a period exceeding this TTL setting or if it is sent but does not receive a timely response or acknowledgment from S3 Tables, the request is deemed to have expired. | `45` second | +| **Health Check Interval** | Specifies the time interval (in seconds) for the Sink to perform automatic health checks on its connection with S3 Tables. | `15` second | +| **Health Check Interval Jitter** | | | +| **Health Check Timeout** | | | +| **Max Buffer Queue Size** | Specifies the maximum number of bytes that can be buffered by each buffer worker process in the S3 Tables Sink. The buffer workers temporarily store data before sending it to S3 Tables, acting as intermediaries to handle the data stream more efficiently. Adjust this value based on system performance and data transmission requirements. | `256` | +| **Query Mode** | Allows you to choose between `synchronous` or `asynchronous` request modes to optimize message transmission according to different requirements. In asynchronous mode, writing to S3 Tables does not block the MQTT message publishing process. However, this may lead to clients receiving messages before they arrive at S3 Tables. | `Asynchronous` | +| **In-flight Window** | "In-flight queue requests" refer to requests that have been initiated but have not yet received a response or acknowledgment. This setting controls the maximum number of in-flight queue requests that can exist simultaneously during Sink communication with S3 Tables.
When **Request Mode** is set to `asynchronous`, the "Request In-flight Queue Window" parameter becomes particularly important. If strict sequential processing of messages from the same MQTT client is crucial, then this value should be set to `1`. | `100` | diff --git a/en_US/data-integration/s3.md b/en_US/data-integration/s3.md index 085cf8ab4..2dceda062 100644 --- a/en_US/data-integration/s3.md +++ b/en_US/data-integration/s3.md @@ -62,7 +62,9 @@ EMQX supports Amazon S3 and other S3-compatible storage services. You can use AW 1. In the [AWS S3 Console](https://console.amazonaws.cn/s3/home), click the **Create bucket** button. Follow the instructions to enter the relevant information, such as bucket name and region, to create an S3 bucket. For detailed operations, refer to the [AWS Documentation](https://docs.amazonaws.cn/AmazonS3/latest/userguide/creating-bucket.html). 2. Set bucket permissions. After the bucket is created successfully, select the bucket and click the **Permissions** tab. Based on your needs, you can set the bucket to public read/write, private, or other permissions. -3. Obtain access keys. In the AWS Console, search for and select the **IAM** service. Create a new user for S3 and obtain the Access Key and Secret Key. +3. Obtain access keys. + - **Manual Configuration**: In the AWS Console, search for and select the **IAM** service. Create a new user for S3 and obtain the Access Key ID and Secret Access Key. See [AWS guide: Managing access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). + - **Automatic Retrieval (EC2 only, since EMQX 6.0.0)**: If EMQX is running on **AWS EC2**, attach an **IAM role** with sufficient permissions. EMQX can automatically fetch temporary credentials from **IMDSv2**. See [AWS guide: IAM roles for Amazon EC2](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html). With the Amazon S3 bucket created and configured, you are now ready to create an Amazon S3 Sink in EMQX. @@ -102,20 +104,35 @@ With MinIO installed and configured, you are now ready to create the Amazon S3 S Before adding the S3 Sink, you need to create the corresponding connector. 1. Go to the Dashboard **Integration** -> **Connector** page. + 2. Click the **Create** button in the top right corner. -3. Select **Amazon S3** as the connector type and click next. -4. Enter the connector name, a combination of upper and lowercase letters and numbers. Here, enter `my-s3`. + +3. Select **Amazon S3** as the connector type and click **Next**. + +4. Enter a name for the connector. The name must start with a letter or number and can contain letters, numbers, hyphens, or underscores. In this example, enter `my-s3`. + 5. Enter the connection information. - If you are using the Amazon S3 bucket, enter the following information: - **Host**: The host varies by region and is formatted as `s3.{region}.amazonaws.com`. + - **Port**: Enter `443`. - - **Access Key ID** and **Secret Access Key**: Enter the access keys created in AWS. + + - **Access Key ID** and **Secret Access Key**: + + - Enter the access keys created in AWS, or + - Leave blank if running EMQX on EC2 with an attached IAM role. + + See the "Amazon S3" tab in [Prepare S3 Bucket](prepare-s3-bucket) for details. + - If you are using MinIO, enter the following information: - **Host**: Enter `127.0.0.1`. If you are running MinIO remotely, enter the actual host address. - **Port**: Enter `9000`. - **Access Key ID** and **Secret Access Key**: Enter the access keys created in MinIO. + 6. Use the default values for the remaining settings. + 7. Before clicking **Create**, you can click **Test Connectivity** to test if the connector can connect to the S3 service. + 8. Click the **Create** button at the bottom to complete the connector creation. You have now completed the connector creation and will proceed to create a rule and Sink for specifying the data to be written into the S3 service. diff --git a/zh_CN/data-integration/s3-tables.md b/zh_CN/data-integration/s3-tables.md index f42b0c4a4..bb7952b86 100644 --- a/zh_CN/data-integration/s3-tables.md +++ b/zh_CN/data-integration/s3-tables.md @@ -8,7 +8,7 @@ EMQX 现已支持与 Amazon S3 表类数据存储服务的无缝集成，可高 ## 工作原理 -EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数据结构化写入 Amazon S3 表数据存储服务中，实现长期存储与数据分析。此集成通过 EMQX 的规则引擎和 S3 Tables Sink，将 MQTT 消息直接流式写入采用 Apache Iceberg 格式的表，并存储于 S3 表存储桶中。 +EMQX 的 Amazaon S3 Tables 数据集成是一个开箱即用的功能。EMQX 通过规则引擎和 S3 Tables Sink 将实时 MQTT 数据结构化写入 Iceberg 表（存放于 S3 表存储桶），用于长期存储与后续分析。 在典型的 IoT 应用场景中： @@ -28,11 +28,11 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 ## 特性与优势 -在 EMQX 中集成 Amazon S3 表数据存储服务，可以为你的业务带来以下功能和优势： +在 EMQX 中集成 Amazon S3 表数据存储服务，可以为您的业务带来以下功能和优势： - **实时流处理**：EMQX 的规则引擎支持在消息写入 S3 表数据存储服务之前，实时提取、转换和按条件路由 MQTT 消息。 - **基于 Iceberg 的 S3 存储**：消息被写入 Apache Iceberg 表，无需使用传统数据库，同时支持类 SQL 的访问模式。 -- **轻松集成分析工具**：数据写入 S3 表后，可通过 Amazon Athena（SQL）、Amazon EMR、Redshift Spectrum，或第三方引擎（如 Presto、Trino、Snowflake）进行查询和分析。 +- **轻松集成分析工具**：数据写入 S3 表后，可通过 Amazon Athena（SQL）、Amazon EMR、Redshift Spectrum，或第三方引擎（如 Presto、Trino、S3 Tables）进行查询和分析。 - **灵活且具成本效益的存储**：Amazon S3 提供高度耐久、低成本的对象存储，适用于设备生成数据的归档、合规存储及时序数据分析等场景。 ## 准备工作 @@ -41,7 +41,7 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 ### 前置准备 -在开始之前，请确保你已了解以下内容： +在开始之前，请确保您已了解以下内容： #### EMQX 相关概念： @@ -50,17 +50,40 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 #### AWS 相关概念： -如果你是第一次使用 AWS S3 表数据存储服务，请了解以下关键术语： +如果您是第一次使用 AWS S3 表数据存储服务，请了解以下关键术语： +- **EC2**：AWS 的虚拟机服务（计算实例）。 +- **IAM**：AWS 身份与权限管理，实例角色可给在该实例上运行的程序签发临时凭证。 +- **IMDSv2**：EC2 的实例元数据/临时凭证获取接口，采用令牌机制，更安全。 - **表存储桶**：一种专用的 S3 存储桶，用于在 S3 表中存储基于 Iceberg 的表格数据及其元数据。 - **Amazon Athena**：一款无服务器的查询引擎，可直接对存储在 Amazon S3 中的数据执行 SQL 查询。Athena 支持标准 SQL 语法，包括如 `CREATE TABLE` 等数据定义语言（DDL）语句，用于定义查询所需的表结构和模式。 - **目录**：Athena 中的元数据容器，用于组织数据库（命名空间）和数据表。 - **数据库（命名空间）**：目录下用于逻辑分组数据表的结构。 - **Iceberg 表**：一种用于数据湖的高性能事务型表格式，支持模式演进、分区裁剪和时间旅行查询等特性。 +### 部署前提与凭证获取方式 + +S3 Tables 连接器支持两种凭证获取方式，请根据 EMQX 的部署环境选择其一： + +- **方式一：手动配置访问密钥（通用）** + 需在[创建连接器](#创建连接器)时填写**访问密钥 ID** 与**访问密钥**。该凭证应具备访问目标 S3 表与 Athena 的必要权限。适用于本地/容器/Kubernetes/非 AWS 云，或未绑定实例角色的 EC2 环境。 + + 关于如何为 IAM 用户创建和管理访问密钥，请参阅 [AWS 官方文档：管理访问密钥](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html)。 + +- **方式二：自动获取临时凭证（EC2 专用，自 EMQX 5.10.1 起）** + 若 EMQX 运行在 AWS EC2 实例上，且实例已绑定具备所需权限的 IAM 角色，可在连接器中将**访问密钥 ID**与**访问密钥**留空。EMQX 将通过 IMDSv2 自动获取与该角色对应的临时凭证。 + + 关于如何为 EC2 实例分配 IAM 角色，请参阅 [AWS 官方文档：Amazon EC2 的 IAM 角色](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html)。 + +::: tip 提示 + +- 请确保实例角色对目标 S3 表（桶/表）与 Athena 拥有足够权限，否则“测试连接”可能失败。 +- 建议在实例上仅允许 IMDSv2；若非 EC2 或未绑定角色，请使用“方式一”手动填写访问密钥。 + ::: + ### 准备 S3 表存储桶 -在创建 EMQX Sink 之前，你需要在 Amazon S3 表数据存储服务中准备 MQTT 数据的目标存储，包括以下内容： +在创建 EMQX Sink 之前，您需要在 Amazon S3 表数据存储服务中准备 MQTT 数据的目标存储，包括以下内容： - 一个用于存储实际数据文件的**表存储桶** - 一个用于逻辑管理相关表的**命名空间** @@ -70,21 +93,21 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 2. 打开 S3 服务。在左侧导航栏中点击**表存储桶**。 -3. 点击**创建表存储桶**。输入你的表存储桶名称（例如：`mybucket`），然后点击**创建表存储桶**。 +3. 点击**创建表存储桶**。输入您的表存储桶名称（例如：`mybucket`），然后点击**创建表存储桶**。 4. 表存储桶创建完成后，点击该表存储桶，进入其**表**列表页面。 -5. 点击**使用 Athena 创建表**。此时会弹出一个窗口，提示你选择命名空间。 +5. 点击**使用 Athena 创建表**。此时会弹出一个窗口，提示您选择命名空间。 6. 选择**创建命名空间**，输入一个命名空间名称，并点击**创建命名空间**进行确认。 7. 命名空间创建完成后，继续点击**使用 Athena 创建表**。 -8. 定义你的 Iceberg 表结构： +8. 定义您的 Iceberg 表结构： - 点击**使用 Athena 查询表**，进入**查询编辑器**： - - 在**目录**选择器中，选择你的目录（例如，如果你的表存储桶名为 `mybucket`，则目录可能为 `s3tablescatalog/mybucket`）。 + - 在**目录**选择器中，选择您的目录（例如，如果您的表存储桶名为 `mybucket`，则目录可能为 `s3tablescatalog/mybucket`）。 - 在**数据库**选择器中，选择刚才创建的命名空间。 - 执行以下数据定义语言（DDL）来创建数据表，并确保表类型设置为 `ICEBERG`。例如： @@ -116,10 +139,12 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 1. 转到 Dashboard **集成** -> **连接器**页面。 2. 点击页面右上角的**创建**。 3. 在连接器类型中选择 **S3 Tables**，点击下一步。 -4. 输入连接器名称，支持大写字母、小写字母和数字的组合。例如：`my-s3-tables`。 +4. 输入连接器名称。名称必须以字母或数字开头，可以包含字母、数字、连字符或下划线。例如：`my-s3-tables`。 5. 输入连接信息： - - **表资源名称（ARN）**：输入你在 AWS 控制台中 S3 表存储桶列表中找到的 Amazon Resource Name (ARN)。 - - **访问密钥 ID 和访问密钥**：输入与具有访问 S3 表和 Athena 权限的 IAM 用户或角色关联的 AWS 访问凭证。 + - **表资源名称（ARN）**：输入您在 AWS 控制台中 S3 表存储桶列表中找到的 Amazon Resource Name (ARN)。 + - **访问密钥 ID 和访问密钥** （可选）： + - 手动配置：输入与具有访问 S3 表和 Athena 权限的 IAM 用户或角色关联的 AWS 访问凭证。 + - 自动获取（自 EMQX 5.10.1 起）：若 EMQX 部署在 AWS EC2 实例上，且实例已关联具备所需权限的 IAM 角色，可将本项留空。EMQX 将通过 IMDSv2 自动获取临时凭证。详情与前置条件见[部署前提与凭证获取方式](#部署前提与凭证获取方式)。 - **启用 TLS**：连接到 S3 表数据存储服务时默认启用 TLS。有关 TLS 连接选项的详细信息，请参阅[外部资源访问的 TLS](../network/overview.md#启用-tls-加密访问外部资源)。 - **健康检查超时**：指定对与 S3 表数据存储服务的连接执行自动健康检查的超时时间。 6. 其余设置保持默认值。 @@ -164,7 +189,7 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 7. 配置 Sink 设置： - - **命名空间**：你的 Iceberg 表所在的命名空间。如果命名空间为多级结构，使用点号分隔（例如 `my.name.space`）。 + - **命名空间**：您的 Iceberg 表所在的命名空间。如果命名空间为多级结构，使用点号分隔（例如 `my.name.space`）。 - **表**：要写入的 Iceberg 表名称。 - **最大记录数**：当达到该数量时，当前数据会被聚合成一个文件上传，并重置时间间隔。 - **时间间隔**：达到设定时间后，无论记录数是否达到上限，当前批次也将被上传并重置计数器。 @@ -177,7 +202,7 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 11. 回到规则创建页面，点击**创建**按钮完成整个规则创建。 -现在您已成功创建了规则，你可以在**规则**页面上看到新建的规则，同时在**动作(Sink)** 标签页看到新建的 S3 Tables Sink。 +现在您已成功创建了规则，您可以在**规则**页面上看到新建的规则，同时在**动作(Sink)** 标签页看到新建的 S3 Tables Sink。 您也可以点击 **集成** -> **Flow 设计器**查看拓扑，通过拓扑可以直观的看到，主题 `t/#` 下的消息在经过规则 `my_rule` 解析后被写入到 S3 Tables 中。 @@ -193,7 +218,7 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 这条消息包含 `payload.str` 和 `payload.int` 字段，与之前定义的规则 SQL 和数据表结构相匹配。 -2. 在**规则**页面中监控规则指标和 Sink 状态。你应该会看到一条新的入站消息和一条新的出站消息。 +2. 在**规则**页面中监控规则指标和 Sink 状态。您应该会看到一条新的入站消息和一条新的出站消息。 3. 打开 Athena 查询编辑器。确保已选择正确的**目录**（例如：`s3tablescatalog/mybucket`）和**数据库（命名空间）**。 @@ -203,7 +228,7 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 SELECT * FROM testtable ``` - 你应该会看到类似如下的一行记录： + 您应该会看到类似如下的一行记录： | c_str | c_long | | --------------- | ------ | @@ -215,11 +240,14 @@ EMQX 与 Amazon S3 表类数据存储服务集成，支持将实时的 MQTT 数 | 字段名称 | 描述 | 默认值 | | -------------------- | ------------------------------------------------------------ | ------ | -| **缓存池大小** | 指定缓冲区工作进程数量，这些工作进程将被分配用于管理 EMQX 与 S3 Tables 的数据流，它们负责在将数据发送到目标服务之前临时存储和处理数据。此设置对于优化性能并确保 Sink 数据传输顺利进行尤为重要。 | `16` | -| **请求超期** | “请求 TTL”（生存时间）设置指定了请求在进入缓冲区后被视为有效的最长持续时间（以秒为单位）。此计时器从请求进入缓冲区时开始计时。如果请求在缓冲区内停留的时间超过了此 TTL 设置或者如果请求已发送但未能在 S3 Tables 中及时收到响应或确认，则将视为请求已过期。 | `45` | -| **健康检查间隔** | 指定 Sink 对与 S3 Tables 的连接执行自动健康检查的时间间隔（以秒为单位）。 | `15` | -| **缓存队列最大长度** | 指定可以由 S3 Tables Sink 中的每个缓冲器工作进程缓冲的最大字节数。缓冲器工作进程在将数据发送到 S3 Tables 之前会临时存储数据，充当处理数据流的中介以更高效地处理数据流。根据系统性能和数据传输要求调整该值。 | `256` | -| **请求模式** | 允许您选择`同步`或`异步`请求模式，以根据不同要求优化消息传输。在异步模式下，写入到 S3 Tables 不会阻塞 MQTT 消息发布过程。但是，这可能导致客户在它们到达 S3 Tables 之前就收到了消息。 | `异步` | -| **请求飞行队列窗口** | “飞行队列请求”是指已启动但尚未收到响应或确认的请求。此设置控制 Sink 与 S3 Tables 通信时可以同时存在的最大飞行队列请求数。
当 **请求模式** 设置为 `异步` 时，“请求飞行队列窗口”参数变得特别重要。如果对于来自同一 MQTT 客户端的消息严格按顺序处理很重要，则应将此值设置为 `1`。 | `100` | -| **最小分片大小** | 聚合完成后的分片上传的最小块大小，上传的数据将在内存中累积，直到达到此大小。 | `5MB` | -| **最大分片大小** | 分块上传的最大分块大小。S3 Tables Sink 不会尝试上传超过此大小的分片。 | `5GB` | \ No newline at end of file +| **最小分块大小** | 聚合完成后的分块上传的最小块大小，上传的数据将在内存中累积，直到达到此大小。 | `5` MB | +| **最大分块大小** | 分块上传的最大分块大小。
S3 上传程序不会尝试上传超过此大小的部分。 | `5` GB | +| **缓存池大小** | 指定缓冲区工作进程数量，这些工作进程将被分配用于管理 EMQX 与 S3 Tables 的数据流，它们负责在将数据发送到目标服务之前临时存储和处理数据。此设置对于优化性能并确保 Sink 数据传输顺利进行尤为重要。 | `16` | +| **请求超期** | “请求 TTL”（生存时间）设置指定了请求在进入缓冲区后被视为有效的最长持续时间（以秒为单位）。此计时器从请求进入缓冲区时开始计时。如果请求在缓冲区内停留的时间超过了此 TTL 设置或者如果请求已发送但未能在 S3 Tables 中及时收到响应或确认，则将视为请求已过期。 | `45` 秒 | +| **健康检查间隔** | 指定 Sink 对与 S3 Tables 的连接执行自动健康检查的时间间隔（以秒为单位）。 | `15` | +| **健康检查间隔抖动** | 在健康检查间隔中添加一个均匀的随机延迟（抖动），用于避免多个节点在相同时间触发健康检查请求。当多个 Sink 或 Source 使用同一连接器时，启用抖动可确保它们在不同时间启动健康检查，提升系统稳定性。 | `0` 毫秒 | +| **健康检查超时** | 指定对与 S3 Tables 服务的连接执行自动健康检查的超时时间。 | `60` 秒 | +| **缓存队列最大长度** | 指定可以由 S3 Tables Sink 中的每个缓冲器工作进程缓冲的最大字节数。缓冲器工作进程在将数据发送到 S3 Tables 之前会临时存储数据，充当处理数据流的中介以更高效地处理数据流。根据系统性能和数据传输要求调整该值。 | `256` MB | +| **最大批量请求大小** | 指定从 EMQX 向 S3 Tables 传输数据时的单次传输最大数据批大小。通过调整批处理大小，您可以微调 EMQX 和 S3 Tables 之间数据传输的效率和性能。
如果将“批处理大小”设置为 "1"，则数据记录将单独发送，而不会被分组为批处理。 | `1000` | +| **请求模式** | 允许您选择`同步`或`异步`请求模式，以根据不同要求优化消息传输。在异步模式下，写入到 S3 Tables 不会阻塞 MQTT 消息发布过程。但是，这可能导致客户在它们到达 S3 Tables 之前就收到了消息。 | `异步` | +| **请求飞行队列窗口** | “飞行队列请求”是指已启动但尚未收到响应或确认的请求。此设置控制 Sink 与 S3 Tables 通信时可以同时存在的最大飞行队列请求数。
当**请求模式**设置为 `异步` 时，“请求飞行队列窗口”参数变得特别重要。如果对于来自同一 MQTT 客户端的消息严格按顺序处理很重要，则应将此值设置为 `1`。 | `10` | diff --git a/zh_CN/data-integration/s3.md b/zh_CN/data-integration/s3.md index 2bd832de2..fc777dee1 100644 --- a/zh_CN/data-integration/s3.md +++ b/zh_CN/data-integration/s3.md @@ -61,7 +61,9 @@ EMQX 支持 Amazon S3 以及兼容 S3 的存储服务，您可以使用 AWS 云 1. 在 [AWS S3 控制台](https://console.amazonaws.cn/s3/home)中，点击**创建存储桶**按钮。然后按照向导的指示填写相关信息，如存储桶名称（例如 `iot-data`）、区域等，创建一个 S3 存储桶。详细操作可参考 [AWS 文档](https://docs.amazonaws.cn/AmazonS3/latest/userguide/creating-bucket.html)。 2. 设置存储桶权限：在存储桶创建成功后，选择该存储桶，并点击**权限**选项卡，根据需求可以为存储桶选择公共读写、私有等权限。 -3. 获取访问密钥：在 AWS 控制台中，搜索并选择 **IAM** 服务，为 S3 创建一个新的用户，获取 Access Key 和 Secret Key。 +3. 获取访问密钥： + - **手动配置**：在 AWS 控制台中搜索并选择 **IAM** 服务，为 S3 创建新的用户，并获取**访问密钥 ID** 和**访问密钥**。参见 [AWS 指南：管理访问密钥](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html)。 + - **自动获取（自 EMQX 6.0.0 起，仅适用于 EC2）**：如果 EMQX 运行在 **AWS EC2** 上，请为实例绑定具备足够权限的 **IAM 角色**。EMQX 将通过 **IMDSv2** 自动获取临时凭证。参见 [AWS 指南：对 Amazon EC2 上的应用程序使用角色](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html)。 至此，您已经完成了 S3 存储桶的创建与配置，接下来我们将在 EMQX 中创建 Amazon S3 Sink。 @@ -100,13 +102,35 @@ EMQX 支持 Amazon S3 以及兼容 S3 的存储服务，您可以使用 AWS 云 在添加 S3 Sink 前，您需要创建对应的连接器，创建步骤如下： 1. 转到 Dashboard **集成** -> **连接器**页面。 + 2. 点击页面右上角的**创建**。 + 3. 在连接器类型中选择 **Amazon S3**，点击下一步。 -4. 输入连接器名称，要求是大小写英文字母和数字的组合。这里我们输入 `my-s3`。 + +4. 输入连接器名称。名称必须以字母或数字开头，可以包含字母、数字、连字符或下划线。例如： `my-s3`。 + 5. 输入连接信息： - - 对于 AWS S3：**主机**根据区域不同，格式为 `s3.{region}.amazonaws.com`，**端口**填写 `443`，**访问密钥 ID** 和**私有访问密钥**填写 AWS 中创建的访问密钥。 - - 对于 MinIO：**主机**填写 `127.0.0.1` （如果 MinIO 在远程运行，填写实际地址），**端口**填写 `9000`，**访问密钥 ID** 和**私有访问密钥**填写 MinIO 中创建的访问密钥。 + - 如果使用 AWS S3 存储桶，请输入以下信息： + + - **主机**：根据区域不同，格式为 `s3.{region}.amazonaws.com`。 + + - **端口**：填写 `443`。 + + - **访问密钥 ID** 和**私有访问密钥**： + + - 填写在 AWS 中创建的访问密钥，或者 + - 如果 EMQX 运行在已绑定 IAM 角色的 EC2 上，可留空。 + + 详细说明请参见[准备 S3 存储桶](准备-s3-存储桶)中的 Amazon S3 标签页。 + + - 如果使用 MinIO： + + - **主机**：填写 `127.0.0.1` （如果 MinIO 在远程运行，填写实际地址）。 + - **端口**：填写 `9000`。 + - **访问密钥 ID** 和**私有访问密钥**：填写 MinIO 中创建的访问密钥。 + 6. 点击**创建**之前，您可以先点击**测试连接**来测试连接器是否可以连接到 S3 服务。 + 7. 点击最下方**创建**按钮完成连接器创建。 至此您已经完成连接器创建，接下来将继续创建一条规则和 Sink 来指定需要写入的数据。 From 0a3437312b9a8716228e649bce1d46822d674243 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 10:40:05 +0800 Subject: [PATCH 13/49] Update s3-tables.md --- en_US/data-integration/s3-tables.md | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/en_US/data-integration/s3-tables.md b/en_US/data-integration/s3-tables.md index df3a57b2a..8df85bb49 100644 --- a/en_US/data-integration/s3-tables.md +++ b/en_US/data-integration/s3-tables.md @@ -241,15 +241,16 @@ This section shows how to test the rule configured with the S3 Tables Sink. This section delves into the advanced configuration options available for the S3 Tables Sink. In the Dashboard, when configuring the Sink, you can expand **Advanced Settings** to adjust the following parameters based on your specific needs. -| Field Name | Description | Default Value | -| -------------------------------- | ------------------------------------------------------------ | -------------- | -| **Min Part Size** | The minimum part size for multipart uploads.
Uploaded data will be accumulated in memory until this size is reached. | `5` MB | -| **Max Part Size** | The maximum part size for multipart uploads.
S3 uploader won't try to upload parts larger than this size. | `5` GB | -| **Buffer Pool Size** | Specifies the number of buffer worker processes, which are allocated to manage the data flow between EMQX and S3 Tables. These workers temporarily store and process data before sending it to the target service, crucial for optimizing performance and ensuring smooth data transmission. | `16` | -| **Request TTL** | The "Request TTL" (Time To Live) configuration setting specifies the maximum duration, in seconds, that a request is considered valid once it enters the buffer. This timer starts ticking from the moment the request is buffered. If the request stays in the buffer for a period exceeding this TTL setting or if it is sent but does not receive a timely response or acknowledgment from S3 Tables, the request is deemed to have expired. | `45` second | -| **Health Check Interval** | Specifies the time interval (in seconds) for the Sink to perform automatic health checks on its connection with S3 Tables. | `15` second | -| **Health Check Interval Jitter** | | | -| **Health Check Timeout** | | | -| **Max Buffer Queue Size** | Specifies the maximum number of bytes that can be buffered by each buffer worker process in the S3 Tables Sink. The buffer workers temporarily store data before sending it to S3 Tables, acting as intermediaries to handle the data stream more efficiently. Adjust this value based on system performance and data transmission requirements. | `256` | -| **Query Mode** | Allows you to choose between `synchronous` or `asynchronous` request modes to optimize message transmission according to different requirements. In asynchronous mode, writing to S3 Tables does not block the MQTT message publishing process. However, this may lead to clients receiving messages before they arrive at S3 Tables. | `Asynchronous` | -| **In-flight Window** | "In-flight queue requests" refer to requests that have been initiated but have not yet received a response or acknowledgment. This setting controls the maximum number of in-flight queue requests that can exist simultaneously during Sink communication with S3 Tables.
When **Request Mode** is set to `asynchronous`, the "Request In-flight Queue Window" parameter becomes particularly important. If strict sequential processing of messages from the same MQTT client is crucial, then this value should be set to `1`. | `100` | +| Field Name | Description | Default Value | +| -------------------------------- | ------------------------------------------------------------ | --------------- | +| **Min Part Size** | The minimum part size for multipart uploads.
Uploaded data will be accumulated in memory until this size is reached. | `5` MB | +| **Max Part Size** | The maximum part size for multipart uploads.
S3 uploader won't try to upload parts larger than this size. | `5` GB | +| **Buffer Pool Size** | Specifies the number of buffer worker processes, which are allocated to manage the data flow between EMQX and S3 Tables. These workers temporarily store and process data before sending it to the target service, crucial for optimizing performance and ensuring smooth data transmission. | `16` | +| **Request TTL** | The "Request TTL" (Time To Live) configuration setting specifies the maximum duration, in seconds, that a request is considered valid once it enters the buffer. This timer starts ticking from the moment the request is buffered. If the request stays in the buffer for a period exceeding this TTL setting or if it is sent but does not receive a timely response or acknowledgment from S3 Tables, the request is deemed to have expired. | `45` second | +| **Health Check Interval** | Specifies the time interval (in seconds) for the Sink to perform automatic health checks on its connection with S3 Tables. | `15` seconds | +| **Health Check Interval Jitter** | A uniform random delay added on top of the base health check interval to reduce the chance that multiple nodes initiate health checks at the same time. When multiple Actions or Sources share the same Connector, enabling jitter ensures their health checks are initiated at slightly different times. | `0` millisecond | +| **Health Check Timeout** | Specify the timeout duration for the connector to perform automatic health checks on its connection with S3 Tables. | `60` seconds | +| **Max Buffer Queue Size** | Specifies the maximum number of bytes that can be buffered by each buffer worker process in the S3 Tables Sink. The buffer workers temporarily store data before sending it to S3 Tables, acting as intermediaries to handle the data stream more efficiently. Adjust this value based on system performance and data transmission requirements. | `256` MB | +| **Batch Size** | Specifies the maximum size of data batches transmitted from EMQX to BigQuery in a single transfer operation. By adjusting the size, you can fine-tune the efficiency and performance of data transfer between EMQX and BigQuery. If the "Batch Size" is set to "1," data records are sent individually, without being grouped into batches. | 1000 | +| **Query Mode** | Allows you to choose between `synchronous` or `asynchronous` request modes to optimize message transmission according to different requirements. In asynchronous mode, writing to S3 Tables does not block the MQTT message publishing process. However, this may lead to clients receiving messages before they arrive at S3 Tables. | `Asynchronous` | +| **In-flight Window** | "In-flight queue requests" refer to requests that have been initiated but have not yet received a response or acknowledgment. This setting controls the maximum number of in-flight queue requests that can exist simultaneously during Sink communication with S3 Tables.
When **Request Mode** is set to `asynchronous`, the "Request In-flight Queue Window" parameter becomes particularly important. If strict sequential processing of messages from the same MQTT client is crucial, then this value should be set to `1`. | `100` | From eaf40ad52f598c0f978c7a5d74381c87310f9e94 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 10:52:25 +0800 Subject: [PATCH 14/49] Update S3.md --- en_US/data-integration/s3.md | 24 ++++++++++++------------ zh_CN/data-integration/s3.md | 14 +++++++------- 2 files changed, 19 insertions(+), 19 deletions(-) diff --git a/en_US/data-integration/s3.md b/en_US/data-integration/s3.md index 2dceda062..0ca94b96f 100644 --- a/en_US/data-integration/s3.md +++ b/en_US/data-integration/s3.md @@ -64,7 +64,7 @@ EMQX supports Amazon S3 and other S3-compatible storage services. You can use AW 2. Set bucket permissions. After the bucket is created successfully, select the bucket and click the **Permissions** tab. Based on your needs, you can set the bucket to public read/write, private, or other permissions. 3. Obtain access keys. - **Manual Configuration**: In the AWS Console, search for and select the **IAM** service. Create a new user for S3 and obtain the Access Key ID and Secret Access Key. See [AWS guide: Managing access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). - - **Automatic Retrieval (EC2 only, since EMQX 6.0.0)**: If EMQX is running on **AWS EC2**, attach an **IAM role** with sufficient permissions. EMQX can automatically fetch temporary credentials from **IMDSv2**. See [AWS guide: IAM roles for Amazon EC2](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html). + - **Automatic Retrieval (EC2 only, since EMQX 5.10.1)**: If EMQX is running on **AWS EC2**, attach an **IAM role** with sufficient permissions. EMQX can automatically fetch temporary credentials from **IMDSv2**. See [AWS guide: IAM roles for Amazon EC2](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html). With the Amazon S3 bucket created and configured, you are now ready to create an Amazon S3 Sink in EMQX. @@ -122,7 +122,7 @@ Before adding the S3 Sink, you need to create the corresponding connector. - Enter the access keys created in AWS, or - Leave blank if running EMQX on EC2 with an attached IAM role. - See the "Amazon S3" tab in [Prepare S3 Bucket](prepare-s3-bucket) for details. + See the "Amazon S3" tab in [Prepare S3 Bucket](#prepare-s3-bucket) for details. - If you are using MinIO, enter the following information: - **Host**: Enter `127.0.0.1`. If you are running MinIO remotely, enter the actual host address. @@ -269,13 +269,13 @@ Open the `iot-data` bucket. You should see the messages you published successful This section delves into the advanced configuration options available for the S3 Sink. In the Dashboard, when configuring the Sink, you can expand **Advanced Settings** to adjust the following parameters based on your specific needs. -| Field Name | Description | Default Value | -| ------------------------- | ------------------------------------------------------------ | -------------- | -| **Buffer Pool Size** | Specifies the number of buffer worker processes, which are allocated to manage the data flow between EMQX and S3. These workers temporarily store and process data before sending it to the target service, crucial for optimizing performance and ensuring smooth data transmission. | `16` | -| **Request TTL** | The "Request TTL" (Time To Live) configuration setting specifies the maximum duration, in seconds, that a request is considered valid once it enters the buffer. This timer starts ticking from the moment the request is buffered. If the request stays in the buffer for a period exceeding this TTL setting or if it is sent but does not receive a timely response or acknowledgment from S3, the request is deemed to have expired. | | -| **Health Check Interval** | Specifies the time interval (in seconds) for the Sink to perform automatic health checks on its connection with S3. | `15` | -| **Max Buffer Queue Size** | Specifies the maximum number of bytes that can be buffered by each buffer worker process in the S3 Sink. The buffer workers temporarily store data before sending it to S3, acting as intermediaries to handle the data stream more efficiently. Adjust this value based on system performance and data transmission requirements. | `256` | -| **Query Mode** | Allows you to choose between `synchronous` or `asynchronous` request modes to optimize message transmission according to different requirements. In asynchronous mode, writing to S3 does not block the MQTT message publishing process. However, this may lead to clients receiving messages before they arrive at S3. | `Asynchronous` | -| **In-flight Window** | "In-flight queue requests" refer to requests that have been initiated but have not yet received a response or acknowledgment. This setting controls the maximum number of in-flight queue requests that can exist simultaneously during Sink communication with S3.
When **Request Mode** is set to `asynchronous`, the "Request In-flight Queue Window" parameter becomes particularly important. If strict sequential processing of messages from the same MQTT client is crucial, then this value should be set to `1`. | `100` | -| **Min Part Size** | The minimum chunk size for part uploads after aggregation is complete. The data to be uploaded will accumulate in memory until it reaches this size. | `5MB` | -| **Max Part Size** | The maximum chunk size for part uploads. The S3 Sink will not attempt to upload parts exceeding this size. | `5GB` | +| Field Name | Description | Default Value | +| -------------------------------- | ------------------------------------------------------------ | --------------- | +| **Buffer Pool Size** | Specifies the number of buffer worker processes, which are allocated to manage the data flow between EMQX and S3. These workers temporarily store and process data before sending it to the target service, crucial for optimizing performance and ensuring smooth data transmission. | `16` | +| **Request TTL** | The "Request TTL" (Time To Live) configuration setting specifies the maximum duration, in seconds, that a request is considered valid once it enters the buffer. This timer starts ticking from the moment the request is buffered. If the request stays in the buffer for a period exceeding this TTL setting or if it is sent but does not receive a timely response or acknowledgment from S3, the request is deemed to have expired. | `45` | +| **Health Check Interval** | Specifies the time interval (in seconds) for the Sink to perform automatic health checks on its connection with S3. | `15` seconds | +| **Health Check Interval Jitter** | A uniform random delay added on top of the base health check interval to reduce the chance that multiple nodes initiate health checks at the same time. When multiple Actions or Sources share the same Connector, enabling jitter ensures their health checks are initiated at slightly different times. | `0` millisecond | +| **Health Check Timeout** | Specify the timeout duration for the connector to perform automatic health checks on its connection with S3 Tables. | `60` seconds | +| **Max Buffer Queue Size** | Specifies the maximum number of bytes that can be buffered by each buffer worker process in the S3 Sink. The buffer workers temporarily store data before sending it to S3, acting as intermediaries to handle the data stream more efficiently. Adjust this value based on system performance and data transmission requirements. | `256` MB | +| **Query Mode** | Allows you to choose between `synchronous` or `asynchronous` request modes to optimize message transmission according to different requirements. In asynchronous mode, writing to S3 does not block the MQTT message publishing process. However, this may lead to clients receiving messages before they arrive at S3. | `Asynchronous` | +| **In-flight Window** | "In-flight queue requests" refer to requests that have been initiated but have not yet received a response or acknowledgment. This setting controls the maximum number of in-flight queue requests that can exist simultaneously during Sink communication with S3.
When **Request Mode** is set to `asynchronous`, the "Request In-flight Queue Window" parameter becomes particularly important. If strict sequential processing of messages from the same MQTT client is crucial, then this value should be set to `1`. | `100` | diff --git a/zh_CN/data-integration/s3.md b/zh_CN/data-integration/s3.md index fc777dee1..55c123e86 100644 --- a/zh_CN/data-integration/s3.md +++ b/zh_CN/data-integration/s3.md @@ -63,7 +63,7 @@ EMQX 支持 Amazon S3 以及兼容 S3 的存储服务，您可以使用 AWS 云 2. 设置存储桶权限：在存储桶创建成功后，选择该存储桶，并点击**权限**选项卡，根据需求可以为存储桶选择公共读写、私有等权限。 3. 获取访问密钥： - **手动配置**：在 AWS 控制台中搜索并选择 **IAM** 服务，为 S3 创建新的用户，并获取**访问密钥 ID** 和**访问密钥**。参见 [AWS 指南：管理访问密钥](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html)。 - - **自动获取（自 EMQX 6.0.0 起，仅适用于 EC2）**：如果 EMQX 运行在 **AWS EC2** 上，请为实例绑定具备足够权限的 **IAM 角色**。EMQX 将通过 **IMDSv2** 自动获取临时凭证。参见 [AWS 指南：对 Amazon EC2 上的应用程序使用角色](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html)。 + - **自动获取（自 EMQX 5.10.1 起，仅适用于 EC2）**：如果 EMQX 运行在 **AWS EC2** 上，请为实例绑定具备足够权限的 **IAM 角色**。EMQX 将通过 **IMDSv2** 自动获取临时凭证。参见 [AWS 指南：对 Amazon EC2 上的应用程序使用角色](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html)。 至此，您已经完成了 S3 存储桶的创建与配置，接下来我们将在 EMQX 中创建 Amazon S3 Sink。 @@ -121,7 +121,7 @@ EMQX 支持 Amazon S3 以及兼容 S3 的存储服务，您可以使用 AWS 云 - 填写在 AWS 中创建的访问密钥，或者 - 如果 EMQX 运行在已绑定 IAM 角色的 EC2 上，可留空。 - 详细说明请参见[准备 S3 存储桶](准备-s3-存储桶)中的 Amazon S3 标签页。 + 详细说明请参见[准备 S3 存储桶](#准备-s3-存储桶)中的 Amazon S3 标签页。 - 如果使用 MinIO： @@ -269,10 +269,10 @@ mqttx pub -i emqx_c -t t/1 -m '{ "msg": "hello S3" }' | 字段名称 | 描述 | 默认值 | | -------------------- | ------------------------------------------------------------ | ------ | | **缓存池大小** | 指定缓冲区工作进程数量，这些工作进程将被分配用于管理 EMQX 与 S3 的数据流，它们负责在将数据发送到目标服务之前临时存储和处理数据。此设置对于优化性能并确保 Sink 数据传输顺利进行尤为重要。 | `16` | -| **请求超期** | “请求 TTL”（生存时间）设置指定了请求在进入缓冲区后被视为有效的最长持续时间（以秒为单位）。此计时器从请求进入缓冲区时开始计时。如果请求在缓冲区内停留的时间超过了此 TTL 设置或者如果请求已发送但未能在 S3 中及时收到响应或确认，则将视为请求已过期。 | `45` | -| **健康检查间隔** | 指定 Sink 对与 S3 的连接执行自动健康检查的时间间隔（以秒为单位）。 | `15` | -| **缓存队列最大长度** | 指定可以由 S3 Sink 中的每个缓冲器工作进程缓冲的最大字节数。缓冲器工作进程在将数据发送到 S3 之前会临时存储数据，充当处理数据流的中介以更高效地处理数据流。根据系统性能和数据传输要求调整该值。 | `256` | +| **请求超期** | “请求 TTL”（生存时间）设置指定了请求在进入缓冲区后被视为有效的最长持续时间（以秒为单位）。此计时器从请求进入缓冲区时开始计时。如果请求在缓冲区内停留的时间超过了此 TTL 设置或者如果请求已发送但未能在 S3 中及时收到响应或确认，则将视为请求已过期。 | `45` 秒 | +| **健康检查间隔** | 指定 Sink 对与 S3 的连接执行自动健康检查的时间间隔（以秒为单位）。 | `15` 秒 | +| **健康检查间隔抖动** | 在健康检查间隔中添加一个均匀的随机延迟（抖动），用于避免多个节点在相同时间触发健康检查请求。当多个 Sink 或 Source 使用同一连接器时，启用抖动可确保它们在不同时间启动健康检查，提升系统稳定性。 | `0` 毫秒 | +| **健康检查超时** | 指定对与 S3 Tables 服务的连接执行自动健康检查的超时时间。 | `60` 秒 | +| **缓存队列最大长度** | 指定可以由 S3 Sink 中的每个缓冲器工作进程缓冲的最大字节数。缓冲器工作进程在将数据发送到 S3 之前会临时存储数据，充当处理数据流的中介以更高效地处理数据流。根据系统性能和数据传输要求调整该值。 | `256` MB | | **请求模式** | 允许您选择`同步`或`异步`请求模式，以根据不同要求优化消息传输。在异步模式下，写入到 S3 不会阻塞 MQTT 消息发布过程。但是，这可能导致客户在它们到达 S3 之前就收到了消息。 | `异步` | | **请求飞行队列窗口** | “飞行队列请求”是指已启动但尚未收到响应或确认的请求。此设置控制 Sink 与 S3 通信时可以同时存在的最大飞行队列请求数。
当 **请求模式** 设置为 `异步` 时，“请求飞行队列窗口”参数变得特别重要。如果对于来自同一 MQTT 客户端的消息严格按顺序处理很重要，则应将此值设置为 `1`。 | `100` | -| **最小分片大小** | 聚合完成后的分片上传的最小块大小，上传的数据将在内存中累积，直到达到此大小。 | `5MB` | -| **最大分片大小** | 分块上传的最大分块大小。S3 Sink 不会尝试上传超过此大小的分片。 | `5GB` | From 0edd1ceea236ec4c510fac46913876a99f05b008 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 11:04:53 +0800 Subject: [PATCH 15/49] Remove version statements --- en_US/data-integration/s3-tables.md | 4 ++-- en_US/data-integration/s3.md | 2 +- zh_CN/data-integration/s3-tables.md | 6 +++--- zh_CN/data-integration/s3.md | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/en_US/data-integration/s3-tables.md b/en_US/data-integration/s3-tables.md index 8df85bb49..65bb1c305 100644 --- a/en_US/data-integration/s3-tables.md +++ b/en_US/data-integration/s3-tables.md @@ -70,7 +70,7 @@ The S3 Tables connector supports two ways to obtain credentials. Choose based on To create and manage access keys for an IAM user, see the [AWS documentation on managing access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). -- **Option 2: Automatically obtain temporary credentials (EC2 only, since EMQX 5.10.1)** +- **Option 2: Automatically obtain temporary credentials (EC2 only)** If EMQX runs on an AWS EC2 instance and the instance has an attached IAM role with the necessary permissions, you can leave **Access Key ID** and **Secret Access Key** blank in the connector. EMQX will use IMDSv2 to fetch temporary credentials associated with that role. To learn how to assign an IAM role to an EC2 instance, see the [AWS documentation on IAM roles for Amazon EC2](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html). @@ -145,7 +145,7 @@ Before adding the S3 Tables Sink, you need to create the corresponding connector - **S3Tables ARN**: Enter the Amazon Resource Name (ARN) of your S3 Table Bucket. You can find this in the Table buckets section in the AWS Console. - **Access Key ID and Secret Access Key** (optional): - **Manual configuration:** Enter AWS credentials associated with an IAM user or role that has permission to access S3 Tables and Athena. - - **Automatic retrieval (since EMQX 5.10.1):** If EMQX is deployed on an AWS EC2 instance and the instance is associated with an IAM role that has the required permissions, you can leave this field blank. EMQX will automatically obtain temporary credentials through IMDSv2. See [Deployment Prerequisites and Credential Sources](#deployment-prerequisites-and-credential-sources) for details. + - **Automatic retrieval:** If EMQX is deployed on an AWS EC2 instance and the instance is associated with an IAM role that has the required permissions, you can leave this field blank. EMQX will automatically obtain temporary credentials through IMDSv2. See [Deployment Prerequisites and Credential Sources](#deployment-prerequisites-and-credential-sources) for details. - **Enable TLS**: TLS is enabled by default when connecting to S3 Tables. For detailed TLS connection options, see [TLS for External Resource Access](../network/overview.md#enable-tls-encryption-for-accessing-external-resources). - **Health Check Timeout**: Specify the timeout duration for the connector to perform automatic health checks on its connection with S3 Tables. 7. Use the default values for the remaining settings. diff --git a/en_US/data-integration/s3.md b/en_US/data-integration/s3.md index 0ca94b96f..cddb5e191 100644 --- a/en_US/data-integration/s3.md +++ b/en_US/data-integration/s3.md @@ -64,7 +64,7 @@ EMQX supports Amazon S3 and other S3-compatible storage services. You can use AW 2. Set bucket permissions. After the bucket is created successfully, select the bucket and click the **Permissions** tab. Based on your needs, you can set the bucket to public read/write, private, or other permissions. 3. Obtain access keys. - **Manual Configuration**: In the AWS Console, search for and select the **IAM** service. Create a new user for S3 and obtain the Access Key ID and Secret Access Key. See [AWS guide: Managing access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). - - **Automatic Retrieval (EC2 only, since EMQX 5.10.1)**: If EMQX is running on **AWS EC2**, attach an **IAM role** with sufficient permissions. EMQX can automatically fetch temporary credentials from **IMDSv2**. See [AWS guide: IAM roles for Amazon EC2](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html). + - **Automatic Retrieval (EC2 only)**: If EMQX is running on **AWS EC2**, attach an **IAM role** with sufficient permissions. EMQX can automatically fetch temporary credentials from **IMDSv2**. See [AWS guide: IAM roles for Amazon EC2](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html). With the Amazon S3 bucket created and configured, you are now ready to create an Amazon S3 Sink in EMQX. diff --git a/zh_CN/data-integration/s3-tables.md b/zh_CN/data-integration/s3-tables.md index bb7952b86..6d66bed98 100644 --- a/zh_CN/data-integration/s3-tables.md +++ b/zh_CN/data-integration/s3-tables.md @@ -70,7 +70,7 @@ S3 Tables 连接器支持两种凭证获取方式，请根据 EMQX 的部署环 关于如何为 IAM 用户创建和管理访问密钥，请参阅 [AWS 官方文档：管理访问密钥](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html)。 -- **方式二：自动获取临时凭证（EC2 专用，自 EMQX 5.10.1 起）** +- **方式二：自动获取临时凭证（EC2 专用）** 若 EMQX 运行在 AWS EC2 实例上，且实例已绑定具备所需权限的 IAM 角色，可在连接器中将**访问密钥 ID**与**访问密钥**留空。EMQX 将通过 IMDSv2 自动获取与该角色对应的临时凭证。 关于如何为 EC2 实例分配 IAM 角色，请参阅 [AWS 官方文档：Amazon EC2 的 IAM 角色](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html)。 @@ -143,8 +143,8 @@ S3 Tables 连接器支持两种凭证获取方式，请根据 EMQX 的部署环 5. 输入连接信息： - **表资源名称（ARN）**：输入您在 AWS 控制台中 S3 表存储桶列表中找到的 Amazon Resource Name (ARN)。 - **访问密钥 ID 和访问密钥** （可选）： - - 手动配置：输入与具有访问 S3 表和 Athena 权限的 IAM 用户或角色关联的 AWS 访问凭证。 - - 自动获取（自 EMQX 5.10.1 起）：若 EMQX 部署在 AWS EC2 实例上，且实例已关联具备所需权限的 IAM 角色，可将本项留空。EMQX 将通过 IMDSv2 自动获取临时凭证。详情与前置条件见[部署前提与凭证获取方式](#部署前提与凭证获取方式)。 + - **手动配置**：输入与具有访问 S3 表和 Athena 权限的 IAM 用户或角色关联的 AWS 访问凭证。 + - **自动获取**：若 EMQX 部署在 AWS EC2 实例上，且实例已关联具备所需权限的 IAM 角色，可将本项留空。EMQX 将通过 IMDSv2 自动获取临时凭证。详情与前置条件见[部署前提与凭证获取方式](#部署前提与凭证获取方式)。 - **启用 TLS**：连接到 S3 表数据存储服务时默认启用 TLS。有关 TLS 连接选项的详细信息，请参阅[外部资源访问的 TLS](../network/overview.md#启用-tls-加密访问外部资源)。 - **健康检查超时**：指定对与 S3 表数据存储服务的连接执行自动健康检查的超时时间。 6. 其余设置保持默认值。 diff --git a/zh_CN/data-integration/s3.md b/zh_CN/data-integration/s3.md index 55c123e86..953eedda2 100644 --- a/zh_CN/data-integration/s3.md +++ b/zh_CN/data-integration/s3.md @@ -63,7 +63,7 @@ EMQX 支持 Amazon S3 以及兼容 S3 的存储服务，您可以使用 AWS 云 2. 设置存储桶权限：在存储桶创建成功后，选择该存储桶，并点击**权限**选项卡，根据需求可以为存储桶选择公共读写、私有等权限。 3. 获取访问密钥： - **手动配置**：在 AWS 控制台中搜索并选择 **IAM** 服务，为 S3 创建新的用户，并获取**访问密钥 ID** 和**访问密钥**。参见 [AWS 指南：管理访问密钥](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html)。 - - **自动获取（自 EMQX 5.10.1 起，仅适用于 EC2）**：如果 EMQX 运行在 **AWS EC2** 上，请为实例绑定具备足够权限的 **IAM 角色**。EMQX 将通过 **IMDSv2** 自动获取临时凭证。参见 [AWS 指南：对 Amazon EC2 上的应用程序使用角色](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html)。 + - **自动获取（仅适用于 EC2）**：如果 EMQX 运行在 **AWS EC2** 上，请为实例绑定具备足够权限的 **IAM 角色**。EMQX 将通过 **IMDSv2** 自动获取临时凭证。参见 [AWS 指南：对 Amazon EC2 上的应用程序使用角色](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html)。 至此，您已经完成了 S3 存储桶的创建与配置，接下来我们将在 EMQX 中创建 Amazon S3 Sink。 From 5f4ccba707892a0243145da34f6df6216f03af43 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 11:07:24 +0800 Subject: [PATCH 16/49] Update zh_CN/data-integration/s3-tables.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- zh_CN/data-integration/s3-tables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/zh_CN/data-integration/s3-tables.md b/zh_CN/data-integration/s3-tables.md index 6d66bed98..3581f5876 100644 --- a/zh_CN/data-integration/s3-tables.md +++ b/zh_CN/data-integration/s3-tables.md @@ -8,7 +8,7 @@ EMQX 现已支持与 Amazon S3 表类数据存储服务的无缝集成，可高 ## 工作原理 -EMQX 的 Amazaon S3 Tables 数据集成是一个开箱即用的功能。EMQX 通过规则引擎和 S3 Tables Sink 将实时 MQTT 数据结构化写入 Iceberg 表（存放于 S3 表存储桶），用于长期存储与后续分析。 +EMQX 的 Amazon S3 Tables 数据集成是一个开箱即用的功能。EMQX 通过规则引擎和 S3 Tables Sink 将实时 MQTT 数据结构化写入 Iceberg 表（存放于 S3 表存储桶），用于长期存储与后续分析。 在典型的 IoT 应用场景中： From d99501543c891e702394077e060b4e692e968090 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 11:08:08 +0800 Subject: [PATCH 17/49] Update zh_CN/data-integration/s3-tables.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- zh_CN/data-integration/s3-tables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/zh_CN/data-integration/s3-tables.md b/zh_CN/data-integration/s3-tables.md index 3581f5876..935617145 100644 --- a/zh_CN/data-integration/s3-tables.md +++ b/zh_CN/data-integration/s3-tables.md @@ -246,7 +246,7 @@ S3 Tables 连接器支持两种凭证获取方式，请根据 EMQX 的部署环 | **请求超期** | “请求 TTL”（生存时间）设置指定了请求在进入缓冲区后被视为有效的最长持续时间（以秒为单位）。此计时器从请求进入缓冲区时开始计时。如果请求在缓冲区内停留的时间超过了此 TTL 设置或者如果请求已发送但未能在 S3 Tables 中及时收到响应或确认，则将视为请求已过期。 | `45` 秒 | | **健康检查间隔** | 指定 Sink 对与 S3 Tables 的连接执行自动健康检查的时间间隔（以秒为单位）。 | `15` | | **健康检查间隔抖动** | 在健康检查间隔中添加一个均匀的随机延迟（抖动），用于避免多个节点在相同时间触发健康检查请求。当多个 Sink 或 Source 使用同一连接器时，启用抖动可确保它们在不同时间启动健康检查，提升系统稳定性。 | `0` 毫秒 | -| **健康检查超时** | 指定对与 S3 Tables 服务的连接执行自动健康检查的超时时间。 | `60` 秒 | +| **健康检查超时** | 指定对与 S3 Tables 的连接执行自动健康检查的超时时间。 | `60` 秒 | | **缓存队列最大长度** | 指定可以由 S3 Tables Sink 中的每个缓冲器工作进程缓冲的最大字节数。缓冲器工作进程在将数据发送到 S3 Tables 之前会临时存储数据，充当处理数据流的中介以更高效地处理数据流。根据系统性能和数据传输要求调整该值。 | `256` MB | | **最大批量请求大小** | 指定从 EMQX 向 S3 Tables 传输数据时的单次传输最大数据批大小。通过调整批处理大小，您可以微调 EMQX 和 S3 Tables 之间数据传输的效率和性能。
如果将“批处理大小”设置为 "1"，则数据记录将单独发送，而不会被分组为批处理。 | `1000` | | **请求模式** | 允许您选择`同步`或`异步`请求模式，以根据不同要求优化消息传输。在异步模式下，写入到 S3 Tables 不会阻塞 MQTT 消息发布过程。但是，这可能导致客户在它们到达 S3 Tables 之前就收到了消息。 | `异步` | From 1e1a52dc81dbef9d1356222475a09c24362ea00c Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 11:08:19 +0800 Subject: [PATCH 18/49] Update en_US/data-integration/s3-tables.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- en_US/data-integration/s3-tables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/en_US/data-integration/s3-tables.md b/en_US/data-integration/s3-tables.md index 65bb1c305..f7ac196f3 100644 --- a/en_US/data-integration/s3-tables.md +++ b/en_US/data-integration/s3-tables.md @@ -251,6 +251,6 @@ This section delves into the advanced configuration options available for the S3 | **Health Check Interval Jitter** | A uniform random delay added on top of the base health check interval to reduce the chance that multiple nodes initiate health checks at the same time. When multiple Actions or Sources share the same Connector, enabling jitter ensures their health checks are initiated at slightly different times. | `0` millisecond | | **Health Check Timeout** | Specify the timeout duration for the connector to perform automatic health checks on its connection with S3 Tables. | `60` seconds | | **Max Buffer Queue Size** | Specifies the maximum number of bytes that can be buffered by each buffer worker process in the S3 Tables Sink. The buffer workers temporarily store data before sending it to S3 Tables, acting as intermediaries to handle the data stream more efficiently. Adjust this value based on system performance and data transmission requirements. | `256` MB | -| **Batch Size** | Specifies the maximum size of data batches transmitted from EMQX to BigQuery in a single transfer operation. By adjusting the size, you can fine-tune the efficiency and performance of data transfer between EMQX and BigQuery. If the "Batch Size" is set to "1," data records are sent individually, without being grouped into batches. | 1000 | +| **Batch Size** | Specifies the maximum size of data batches transmitted from EMQX to S3 Tables in a single transfer operation. By adjusting the size, you can fine-tune the efficiency and performance of data transfer between EMQX and S3 Tables. If the "Batch Size" is set to "1," data records are sent individually, without being grouped into batches. | 1000 | | **Query Mode** | Allows you to choose between `synchronous` or `asynchronous` request modes to optimize message transmission according to different requirements. In asynchronous mode, writing to S3 Tables does not block the MQTT message publishing process. However, this may lead to clients receiving messages before they arrive at S3 Tables. | `Asynchronous` | | **In-flight Window** | "In-flight queue requests" refer to requests that have been initiated but have not yet received a response or acknowledgment. This setting controls the maximum number of in-flight queue requests that can exist simultaneously during Sink communication with S3 Tables.
When **Request Mode** is set to `asynchronous`, the "Request In-flight Queue Window" parameter becomes particularly important. If strict sequential processing of messages from the same MQTT client is crucial, then this value should be set to `1`. | `100` | From 91f55f175cdae23d6e86e868bff0e73c84f74a4a Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 11:23:59 +0800 Subject: [PATCH 19/49] Update zh_CN/data-integration/s3.md --- zh_CN/data-integration/s3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/zh_CN/data-integration/s3.md b/zh_CN/data-integration/s3.md index 953eedda2..8311bd4c5 100644 --- a/zh_CN/data-integration/s3.md +++ b/zh_CN/data-integration/s3.md @@ -105,7 +105,7 @@ EMQX 支持 Amazon S3 以及兼容 S3 的存储服务，您可以使用 AWS 云 2. 点击页面右上角的**创建**。 -3. 在连接器类型中选择 **Amazon S3**，点击下一步。 +3. 在连接器类型中选择 **Amazon S3**，点击**下一步**。 4. 输入连接器名称。名称必须以字母或数字开头，可以包含字母、数字、连字符或下划线。例如： `my-s3`。 From c30b9a2bf6b76d24ee715b66ea0beffb28b0d65e Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 13:14:27 +0800 Subject: [PATCH 20/49] update(install): Support Debian 13 https://emqx.atlassian.net/browse/EMQX-14675 --- en_US/deploy/install-debian-ce.md | 1 + en_US/deploy/install-debian.md | 1 + en_US/deploy/install.md | 2 +- zh_CN/deploy/install-debian-ce.md | 1 + zh_CN/deploy/install-debian.md | 1 + zh_CN/deploy/install.md | 2 +- 6 files changed, 6 insertions(+), 2 deletions(-) diff --git a/en_US/deploy/install-debian-ce.md b/en_US/deploy/install-debian-ce.md index 275469857..423a3f5dc 100644 --- a/en_US/deploy/install-debian-ce.md +++ b/en_US/deploy/install-debian-ce.md @@ -5,6 +5,7 @@ This page guides you on installing and starting the latest EMQX on the Debian sy Supported versions: +- Debian 13 - Debian 12 - Debian 11 - Debian 10 diff --git a/en_US/deploy/install-debian.md b/en_US/deploy/install-debian.md index d6f849489..64b6881b0 100644 --- a/en_US/deploy/install-debian.md +++ b/en_US/deploy/install-debian.md @@ -4,6 +4,7 @@ This page guides you on installing and starting the latest EMQX on the Debian sy Supported versions: +- Debian 13 - Debian 12 - Debian 11 - Debian 10 diff --git a/en_US/deploy/install.md b/en_US/deploy/install.md index a18733aaf..0b845eabd 100644 --- a/en_US/deploy/install.md +++ b/en_US/deploy/install.md @@ -9,7 +9,7 @@ The table below lists the operating systems and versions that EMQX supports. | Operating system | Versions supported | x86_64/amd64 | arm64 | | :---------------------------------| :----------------------------------------------- | :-- | :-- | | [Ubuntu](./install-ubuntu.md) | Ubuntu 18.04
Ubuntu 20.04
Ubuntu 22.04
Ubuntu 24.04 | Yes | Yes | -| [Debian](./install-debian.md) | Debian 10
Debian 11
Debian 12 | Yes | Yes | +| [Debian](./install-debian.md) | Debian 10
Debian 11
Debian 12
Debian 13 | Yes | Yes | | [CentOS/RHEL](./install-rhel.md) | CentOS 7
Rocky Linux 8
Rocky Linux 9 | Yes | Yes | | [Amazon Linux](./install-rhel.md) | Amazon Linux 2
Amazon Linux 2023 | Yes | Yes | | [macOS](./install-macOS.md) | macOS 13
macOS 14 | Yes | Yes | diff --git a/zh_CN/deploy/install-debian-ce.md b/zh_CN/deploy/install-debian-ce.md index 7a8cd6a69..47ce135ee 100644 --- a/zh_CN/deploy/install-debian-ce.md +++ b/zh_CN/deploy/install-debian-ce.md @@ -4,6 +4,7 @@ 支持的 Debian 版本： +- Debian 13 - Debian 12 - Debian 11 - Debian 10 diff --git a/zh_CN/deploy/install-debian.md b/zh_CN/deploy/install-debian.md index 70fd9284e..4ef876f24 100644 --- a/zh_CN/deploy/install-debian.md +++ b/zh_CN/deploy/install-debian.md @@ -4,6 +4,7 @@ 支持的 Debian 版本： +- Debian 13 - Debian 12 - Debian 11 - Debian 10 diff --git a/zh_CN/deploy/install.md b/zh_CN/deploy/install.md index df82468a1..f513027a0 100644 --- a/zh_CN/deploy/install.md +++ b/zh_CN/deploy/install.md @@ -23,7 +23,7 @@ EMQX 可以跨平台的在多种操作系统和硬件平台上运行，以下是 | 操作系统 | 支持版本 | x86_64/amd64 | arm64 | | :-------------------------------- | :----------------------- | :----------- | :---- | | Ubuntu | Ubuntu 18.04
Ubuntu 20.04
Ubuntu 22.04
Ubuntu 24.04 | 是 | 是 | -| Debian | Debian 10
Debian 11
Debian 12 | 是 | 是 | +| Debian | Debian 10
Debian 11
Debian 12
Debian 13 | 是 | 是 | | CentOS/RHEL | CentOS 7
Rocky Linux 8
Rocky Linux 9 | 是 | 是 | | Amazon Linux | Amazon Linux 2
Amazon Linux 2023 | 是 | 是 | | macOS | macOS 13
macOS 14 | 是 | 是 | From 3cc2cf57d69d2bb3962090a82194e44130ce176d Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 14:47:30 +0800 Subject: [PATCH 21/49] Remove Debian 10 --- en_US/deploy/install-debian-ce.md | 1 - en_US/deploy/install-debian.md | 1 - en_US/deploy/install.md | 2 +- zh_CN/deploy/install-debian-ce.md | 1 - zh_CN/deploy/install-debian.md | 1 - zh_CN/deploy/install.md | 2 +- 6 files changed, 2 insertions(+), 6 deletions(-) diff --git a/en_US/deploy/install-debian-ce.md b/en_US/deploy/install-debian-ce.md index 423a3f5dc..05485af1d 100644 --- a/en_US/deploy/install-debian-ce.md +++ b/en_US/deploy/install-debian-ce.md @@ -8,7 +8,6 @@ Supported versions: - Debian 13 - Debian 12 - Debian 11 -- Debian 10 ## Install with Apt Source diff --git a/en_US/deploy/install-debian.md b/en_US/deploy/install-debian.md index 64b6881b0..121f3edc5 100644 --- a/en_US/deploy/install-debian.md +++ b/en_US/deploy/install-debian.md @@ -7,7 +7,6 @@ Supported versions: - Debian 13 - Debian 12 - Debian 11 -- Debian 10 For installation on other systems or to install other versions, visit the [EMQX Enterprise download site](https://www.emqx.com/en/downloads-and-install/enterprise). diff --git a/en_US/deploy/install.md b/en_US/deploy/install.md index 0b845eabd..373357296 100644 --- a/en_US/deploy/install.md +++ b/en_US/deploy/install.md @@ -9,7 +9,7 @@ The table below lists the operating systems and versions that EMQX supports. | Operating system | Versions supported | x86_64/amd64 | arm64 | | :---------------------------------| :----------------------------------------------- | :-- | :-- | | [Ubuntu](./install-ubuntu.md) | Ubuntu 18.04
Ubuntu 20.04
Ubuntu 22.04
Ubuntu 24.04 | Yes | Yes | -| [Debian](./install-debian.md) | Debian 10
Debian 11
Debian 12
Debian 13 | Yes | Yes | +| [Debian](./install-debian.md) | Debian 11
Debian 12
Debian 13 | Yes | Yes | | [CentOS/RHEL](./install-rhel.md) | CentOS 7
Rocky Linux 8
Rocky Linux 9 | Yes | Yes | | [Amazon Linux](./install-rhel.md) | Amazon Linux 2
Amazon Linux 2023 | Yes | Yes | | [macOS](./install-macOS.md) | macOS 13
macOS 14 | Yes | Yes | diff --git a/zh_CN/deploy/install-debian-ce.md b/zh_CN/deploy/install-debian-ce.md index 47ce135ee..6f7e25cc2 100644 --- a/zh_CN/deploy/install-debian-ce.md +++ b/zh_CN/deploy/install-debian-ce.md @@ -7,7 +7,6 @@ - Debian 13 - Debian 12 - Debian 11 -- Debian 10 ## 通过 Apt 源安装 diff --git a/zh_CN/deploy/install-debian.md b/zh_CN/deploy/install-debian.md index 4ef876f24..cdf915b2c 100644 --- a/zh_CN/deploy/install-debian.md +++ b/zh_CN/deploy/install-debian.md @@ -7,7 +7,6 @@ - Debian 13 - Debian 12 - Debian 11 -- Debian 10 如希望在其他支持系统中进行安装，或体验其他版本，请可前往 [EMQX 企业版下载页面](https://www.emqx.com/zh/downloads-and-install/enterprise)获取安装信息。 diff --git a/zh_CN/deploy/install.md b/zh_CN/deploy/install.md index f513027a0..c83c5849d 100644 --- a/zh_CN/deploy/install.md +++ b/zh_CN/deploy/install.md @@ -23,7 +23,7 @@ EMQX 可以跨平台的在多种操作系统和硬件平台上运行，以下是 | 操作系统 | 支持版本 | x86_64/amd64 | arm64 | | :-------------------------------- | :----------------------- | :----------- | :---- | | Ubuntu | Ubuntu 18.04
Ubuntu 20.04
Ubuntu 22.04
Ubuntu 24.04 | 是 | 是 | -| Debian | Debian 10
Debian 11
Debian 12
Debian 13 | 是 | 是 | +| Debian | Debian 11
Debian 12
Debian 13 | 是 | 是 | | CentOS/RHEL | CentOS 7
Rocky Linux 8
Rocky Linux 9 | 是 | 是 | | Amazon Linux | Amazon Linux 2
Amazon Linux 2023 | 是 | 是 | | macOS | macOS 13
macOS 14 | 是 | 是 | From e0a9629d2c429086d61a2d6ed4c37dca1c520aa0 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 16:14:22 +0800 Subject: [PATCH 22/49] Update en_US/data-integration/s3-tables.md Co-authored-by: Ivan Dyachkov --- en_US/data-integration/s3-tables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/en_US/data-integration/s3-tables.md b/en_US/data-integration/s3-tables.md index f7ac196f3..5a8de7ed7 100644 --- a/en_US/data-integration/s3-tables.md +++ b/en_US/data-integration/s3-tables.md @@ -71,7 +71,7 @@ The S3 Tables connector supports two ways to obtain credentials. Choose based on To create and manage access keys for an IAM user, see the [AWS documentation on managing access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). - **Option 2: Automatically obtain temporary credentials (EC2 only)** - If EMQX runs on an AWS EC2 instance and the instance has an attached IAM role with the necessary permissions, you can leave **Access Key ID** and **Secret Access Key** blank in the connector. EMQX will use IMDSv2 to fetch temporary credentials associated with that role. + If EMQX runs on an AWS EC2 instance and the instance has an attached IAM role with the necessary permissions, you can leave **Access Key ID** and **Secret Access Key** blank in the connector. EMQX will use IMDSv2 API to fetch temporary credentials associated with that role. To learn how to assign an IAM role to an EC2 instance, see the [AWS documentation on IAM roles for Amazon EC2](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html). From 73dc4e1d87d454c706c44b4c6685209374135af9 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 16:14:38 +0800 Subject: [PATCH 23/49] Update en_US/data-integration/s3-tables.md Co-authored-by: Ivan Dyachkov --- en_US/data-integration/s3-tables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/en_US/data-integration/s3-tables.md b/en_US/data-integration/s3-tables.md index 5a8de7ed7..179dc5100 100644 --- a/en_US/data-integration/s3-tables.md +++ b/en_US/data-integration/s3-tables.md @@ -78,7 +78,7 @@ The S3 Tables connector supports two ways to obtain credentials. Choose based on ::: tip Note - Ensure the instance role has sufficient permissions to the target S3 Tables (bucket/table) and Athena; otherwise, **Test Connectivity** may fail. -- It’s recommended to enforce IMDSv2-only on the instance; if you are not on EC2 or no role is attached, use **Option 1** and enter access keys manually. +- It’s recommended to use an IAM role attached to EC2 instance to manage temporary credentials; if you are not on EC2 or no role is attached, use **Option 1** and enter access keys manually. ::: ### Prepare an S3 Tables Bucket From 92870450baa685c0512bcd894df6af92341d6009 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 16:14:50 +0800 Subject: [PATCH 24/49] Update en_US/data-integration/s3.md Co-authored-by: Ivan Dyachkov --- en_US/data-integration/s3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/en_US/data-integration/s3.md b/en_US/data-integration/s3.md index cddb5e191..6ae2633c1 100644 --- a/en_US/data-integration/s3.md +++ b/en_US/data-integration/s3.md @@ -64,7 +64,7 @@ EMQX supports Amazon S3 and other S3-compatible storage services. You can use AW 2. Set bucket permissions. After the bucket is created successfully, select the bucket and click the **Permissions** tab. Based on your needs, you can set the bucket to public read/write, private, or other permissions. 3. Obtain access keys. - **Manual Configuration**: In the AWS Console, search for and select the **IAM** service. Create a new user for S3 and obtain the Access Key ID and Secret Access Key. See [AWS guide: Managing access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html). - - **Automatic Retrieval (EC2 only)**: If EMQX is running on **AWS EC2**, attach an **IAM role** with sufficient permissions. EMQX can automatically fetch temporary credentials from **IMDSv2**. See [AWS guide: IAM roles for Amazon EC2](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html). + - **Automatic Retrieval (EC2 only)**: If EMQX is running on **AWS EC2**, [attach an **IAM role**](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html) with sufficient permissions. EMQX can automatically fetch temporary credentials from Instance Metadata via [**IMDSv2** API](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-metadata-security-credentials.html). With the Amazon S3 bucket created and configured, you are now ready to create an Amazon S3 Sink in EMQX. From f8b5e8832ab9b76198d1105fd86a5ae78b5f4c7f Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 16:20:51 +0800 Subject: [PATCH 25/49] Update s3-tables.md --- zh_CN/data-integration/s3-tables.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/zh_CN/data-integration/s3-tables.md b/zh_CN/data-integration/s3-tables.md index 935617145..abc46c8dc 100644 --- a/zh_CN/data-integration/s3-tables.md +++ b/zh_CN/data-integration/s3-tables.md @@ -78,7 +78,7 @@ S3 Tables 连接器支持两种凭证获取方式，请根据 EMQX 的部署环 ::: tip 提示 - 请确保实例角色对目标 S3 表（桶/表）与 Athena 拥有足够权限，否则“测试连接”可能失败。 -- 建议在实例上仅允许 IMDSv2；若非 EC2 或未绑定角色，请使用“方式一”手动填写访问密钥。 +- 建议使用附加到 EC2 实例的 IAM 角色来管理临时凭证；若非 EC2 或未绑定角色，请使用“方式一”手动填写访问密钥。 ::: ### 准备 S3 表存储桶 @@ -144,7 +144,7 @@ S3 Tables 连接器支持两种凭证获取方式，请根据 EMQX 的部署环 - **表资源名称（ARN）**：输入您在 AWS 控制台中 S3 表存储桶列表中找到的 Amazon Resource Name (ARN)。 - **访问密钥 ID 和访问密钥** （可选）： - **手动配置**：输入与具有访问 S3 表和 Athena 权限的 IAM 用户或角色关联的 AWS 访问凭证。 - - **自动获取**：若 EMQX 部署在 AWS EC2 实例上，且实例已关联具备所需权限的 IAM 角色，可将本项留空。EMQX 将通过 IMDSv2 自动获取临时凭证。详情与前置条件见[部署前提与凭证获取方式](#部署前提与凭证获取方式)。 + - **自动获取**：若 EMQX 部署在 AWS EC2 实例上，且实例已关联具备所需权限的 IAM 角色，可将本项留空。EMQX 将通过 IMDSv2 API 自动获取临时凭证。详情与前置条件见[部署前提与凭证获取方式](#部署前提与凭证获取方式)。 - **启用 TLS**：连接到 S3 表数据存储服务时默认启用 TLS。有关 TLS 连接选项的详细信息，请参阅[外部资源访问的 TLS](../network/overview.md#启用-tls-加密访问外部资源)。 - **健康检查超时**：指定对与 S3 表数据存储服务的连接执行自动健康检查的超时时间。 6. 其余设置保持默认值。 From f3da0662ff344e08732ababcdf400ebee178b534 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 16:28:34 +0800 Subject: [PATCH 26/49] Update s3.md --- zh_CN/data-integration/s3.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/zh_CN/data-integration/s3.md b/zh_CN/data-integration/s3.md index 8311bd4c5..bca1ce89d 100644 --- a/zh_CN/data-integration/s3.md +++ b/zh_CN/data-integration/s3.md @@ -63,7 +63,7 @@ EMQX 支持 Amazon S3 以及兼容 S3 的存储服务，您可以使用 AWS 云 2. 设置存储桶权限：在存储桶创建成功后，选择该存储桶，并点击**权限**选项卡，根据需求可以为存储桶选择公共读写、私有等权限。 3. 获取访问密钥： - **手动配置**：在 AWS 控制台中搜索并选择 **IAM** 服务，为 S3 创建新的用户，并获取**访问密钥 ID** 和**访问密钥**。参见 [AWS 指南：管理访问密钥](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html)。 - - **自动获取（仅适用于 EC2）**：如果 EMQX 运行在 **AWS EC2** 上，请为实例绑定具备足够权限的 **IAM 角色**。EMQX 将通过 **IMDSv2** 自动获取临时凭证。参见 [AWS 指南：对 Amazon EC2 上的应用程序使用角色](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html)。 + - **自动获取（仅适用于 EC2）**：如果 EMQX 运行在 **AWS EC2** 上，请为实例绑定具备足够权限的 **[IAM 角色](https://docs.aws.amazon.com/zh_cn/IAM/latest/UserGuide/id_roles_use_switch-role-ec2.html)**。EMQX 将通过 **IMDSv2 API** 自动[从实例元数据中获取临时凭证](https://docs.aws.amazon.com/zh_cn/AWSEC2/latest/UserGuide/instance-metadata-security-credentials.html)。 至此，您已经完成了 S3 存储桶的创建与配置，接下来我们将在 EMQX 中创建 Amazon S3 Sink。 From 95f1f240e76f6d19cc68718a92e4f1e2bf10ce18 Mon Sep 17 00:00:00 2001 From: Shawn <506895667@qq.com> Date: Mon, 15 Sep 2025 16:33:11 +0800 Subject: [PATCH 27/49] chore: improve some descriptions --- zh_CN/emqx-ai/multimedia-ai/architecture.md | 2 ++ zh_CN/emqx-ai/multimedia-ai/message-protocol.md | 2 -- zh_CN/emqx-ai/multimedia-ai/overview.md | 8 ++++---- zh_CN/emqx-ai/sdks/mcp-sdk-python.md | 2 +- zh_CN/emqx-ai/sdks/multimedia-ai/overview.md | 4 ++-- 5 files changed, 9 insertions(+), 9 deletions(-) diff --git a/zh_CN/emqx-ai/multimedia-ai/architecture.md b/zh_CN/emqx-ai/multimedia-ai/architecture.md index e1701ee6b..3b8de65ca 100644 --- a/zh_CN/emqx-ai/multimedia-ai/architecture.md +++ b/zh_CN/emqx-ai/multimedia-ai/architecture.md @@ -1,5 +1,7 @@ # 架构 +多媒体服务器使用 WebRTC 与客户端进行点对点的音视频数据传输，使用 MQTT 协议传输普通消息和 WebRTC 信令消息。用户可以使用 Python 编写 AI 代理程序，通过 STDIO 与多媒体服务器进行通信，以实现更复杂的业务逻辑。 + 下图展示了使用多媒体服务器的 AI 系统的整体架构： ```mermaid diff --git a/zh_CN/emqx-ai/multimedia-ai/message-protocol.md b/zh_CN/emqx-ai/multimedia-ai/message-protocol.md index 8cd35a90b..06bc6bc0c 100644 --- a/zh_CN/emqx-ai/multimedia-ai/message-protocol.md +++ b/zh_CN/emqx-ai/multimedia-ai/message-protocol.md @@ -82,8 +82,6 @@ mqttClient.on('message', (topic, message) => { } ``` -完整的客户端信令实现示例可参考 [signaling_mqtt.js](https://github.com/emqx/emqx-multimedia-proxy/blob/main/apps/emqx_media_proxy_web/assets/js/signaling_mqtt.js)。你可以访问 http://localhost:4000/webrtc_mqtt 体验演示。 - ## 通过 MQTT 发送普通消息 多媒体服务器可通过以下 MQTT 主题向设备发送普通消息： diff --git a/zh_CN/emqx-ai/multimedia-ai/overview.md b/zh_CN/emqx-ai/multimedia-ai/overview.md index a9953bc34..860b6e756 100644 --- a/zh_CN/emqx-ai/multimedia-ai/overview.md +++ b/zh_CN/emqx-ai/multimedia-ai/overview.md @@ -5,11 +5,11 @@ EMQX 多媒体服务器是一个基于 WebRTC 技术构建的高性能多媒体 ## 核心功能 - **实时音视频处理**: 支持高质量的音视频流传输，确保低延迟和高可靠性的通信体验。 -- **自动语音识别（ASR）**: 提供精准的语音转文本功能，适用于语音助手、智能客服等应用。 +- **自动语音识别（ASR）**: 提供语音转文本功能，适用于语音助手、智能客服等应用。 - **文本转语音（TTS）**: 支持多种语言和声音风格的文本转语音服务，提升用户交互体验。 -- **图像理解**: 集成先进的图像识别和分析技术，支持多种图像处理应用。 -- **大模型支持**: 利用大模型能力，实现复杂的语音对话和工具调用，满足多样化的业务需求。 -- **高度灵活的架构**: 适应不同规模和复杂度的应用场景，支持横向扩展和定制化配置。 +- **图像理解**: 集成图像识别和分析技术，支持多种图像处理应用。 +- **大模型支持**: 利用大模型能力，实现语音对话和工具调用，满足多样化的业务需求。 +- **高度灵活的架构**: 适应不同规模和复杂度的应用场景，支持横向扩展和定制化配置。TTS、ASR、图像理解、LLM 等模型服务商可灵活选择和集成。 - **高可靠性**: 采用分布式架构设计，确保系统的高可用性和稳定性。 - **低延迟**: 优化的网络传输和大模型处理机制，确保实时交互的流畅性。 diff --git a/zh_CN/emqx-ai/sdks/mcp-sdk-python.md b/zh_CN/emqx-ai/sdks/mcp-sdk-python.md index c713ce90f..4a25ce4b4 100644 --- a/zh_CN/emqx-ai/sdks/mcp-sdk-python.md +++ b/zh_CN/emqx-ai/sdks/mcp-sdk-python.md @@ -13,7 +13,7 @@ cd mcp_over_mqtt_demo ## 创建一个简单的 MCP 服务器 -在 `mcp_over_mqtt_demo` 项目中，让我们创建一个简单的 MCP 服务器，暴露一个计算器工具和一些资源。创建一个名为 `demo_mcp_server.py` 的文件，并添加以下代码： +在 `mcp_over_mqtt_demo` 项目中，让我们创建一个简单的 MCP 服务器，暴露一个工具做整数的加法运算，和一个简单的资源返回字符串。创建一个名为 `demo_mcp_server.py` 的文件，代码如下： ```python # demo_mcp_server.py diff --git a/zh_CN/emqx-ai/sdks/multimedia-ai/overview.md b/zh_CN/emqx-ai/sdks/multimedia-ai/overview.md index d8c64ea20..93b4d97ec 100644 --- a/zh_CN/emqx-ai/sdks/multimedia-ai/overview.md +++ b/zh_CN/emqx-ai/sdks/multimedia-ai/overview.md @@ -1,6 +1,6 @@ # 与多媒体服务适配的客户端 -支持 WebRTC 协议的客户端均可与多媒体 AI 服务进行交互。常见的客户端包括： +多媒体 AI 服务器使用 WebRTC 技术传输音视频流，并且使用 MQTT 协议传输普通消息和 WebRTC 信令消息，实际上客户端无需安装特定的 SDK，任何支持 WebRTC 和 MQTT 协议的客户端均可与多媒体 AI 服务进行交互。常见的客户端包括： - **Web 浏览器**: 现代浏览器（如 Chrome、Firefox、Edge、Safari）均支持 WebRTC，可以直接通过浏览器访问多媒体 AI 服务。 @@ -8,4 +8,4 @@ - **嵌入式设备**: 物联网设备可以通过集成与设备适配的 WebRTC 库来实现与多媒体 AI 服务的连接。如 [ESP32 WebRTC](https://github.com/espressif/esp-webrtc-solution) -这里我们提供了基于 Web 浏览器的客户端代码示例，演示如何与多媒体服务进行交互：[Typescript WebRTC 示例](./webrtc-typescript.md)。 +具体的 WebRTC 信令和 MQTT 消息格式请参考 [多媒体 AI 消息协议](../../multimedia-ai/message-protocol.md)。这里我们提供了基于 Web 浏览器的客户端代码示例，演示如何与多媒体服务进行交互：[Typescript WebRTC 示例](./webrtc-typescript.md)。 From ca535eb10bb3d795d98ffc041fec47337de66474 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Mon, 15 Sep 2025 18:00:59 +0800 Subject: [PATCH 28/49] update(mqtt integration): Update static clientID setting https://emqx.atlassian.net/browse/EMQX-14679 --- en_US/data-integration/data-bridge-mqtt.md | 41 ++++++++++++++++------ 1 file changed, 31 insertions(+), 10 deletions(-) diff --git a/en_US/data-integration/data-bridge-mqtt.md b/en_US/data-integration/data-bridge-mqtt.md index ac176291d..7918b4453 100644 --- a/en_US/data-integration/data-bridge-mqtt.md +++ b/en_US/data-integration/data-bridge-mqtt.md @@ -58,19 +58,37 @@ When EMQX is running in cluster mode or when a connection pool is enabled, using This section guides you on how to configure a connection with a remote MQTT server, using EMQX's [online MQTT server](https://www.emqx.com/en/mqtt/public-mqtt5-broker) as an example. 1. Go to the **Integration** -> **Connector** page on the Dashboard. + 2. Click **Create** at the top right corner of the page. + 3. Select **MQTT Broker** from the list of connector types and click **Next**. + 4. Enter a **name** for the connector, which must be a combination of upper/lower case letters and numbers, for example, `my_mqtt_bridge`. + 5. Configure the connection information: - - **MQTT Broker**: Only supports MQTT over TCP/TLS. Set this to `broker.emqx.io:1883`. + - **MQTT Broker**: Only supports MQTT over TCP/TLS. For exmaple, `broker.emqx.io:1883`. + - **ClientID Prefix**: This can be left blank. In actual use, specifying a client ID prefix can facilitate client management. EMQX will automatically generate client IDs based on the client ID prefix and the size of the connection pool. For more information, see [Connection Pool and Client ID Generation Rules](#connection-pool-and-client-id-generation-rules). - - **Username** and **Password**: These can be left blank, as authentication is not required for this server. + + - **Username** and **Password**: If the MQTT broker requires authentication, enter the username and password associated with the client ID. If no authentication is required by the broker (such as public brokers), these fields can be left blank. + + - **Keepalive**: Specify the desired keepalive interval. + + - **MQTT Version**: Choose the appropriate version for the broker connection. + + - **Static ClientId Entries**: This section allows you to configure static client IDs for the connector to ensure stable connections, especially when connecting to services like Azure IoT Hub. See the [Configure Static Client IDs](#configure-static-client-ids) section for more details on how to configure this. + + ::: tip + + If static client ID entries are defined, only the EMQX nodes that are assigned those client IDs will establish MQTT connections using them. + + ::: Leave the other configurations as default and click the **Create** button to complete the creation of the Connector. The Connector can be used for both Sink and Source. Next, you can create data bridge rules based on this Connector. ### Connection Pool and Client ID Generation Rules -EMQX enables multiple clients to simultaneously connect to the bridged MQTT service. When creating a Connector, you can set up an MQTT client connection pool and configure its size to indicate the number of client connections in the pool. The connection pool maximizes server resources for greater message throughput and better concurrent performance, which is crucial for handling high-load, high-concurrency scenarios. +EMQX enables multiple clients to connect to the bridged MQTT service simultaneously. When creating a Connector, you can set up an MQTT client connection pool and configure its size to indicate the number of client connections in the pool. The connection pool maximizes server resources for greater message throughput and better concurrent performance, which is crucial for handling high-load, high-concurrency scenarios. As the MQTT protocol requires clients connected to an MQTT server to have a unique client ID, and since EMQX can be deployed in a cluster, each client in MQTT bridging is assigned a unique client ID. EMQX automatically generates client IDs according to the following pattern: @@ -95,15 +113,18 @@ To address this issue, from version 5.7.1 onwards, EMQX has implemented the foll ### Configure Static Client IDs -In some use cases, you may only have a finite set of client IDs to use in an integration. In this case, it is possible to assign static client ID sets to individual nodes while configuring the connector. To configure static client IDs, provide a list of client IDs for each node in your EMQX cluster during the Connector setup. Below is an example configuration: +In some use cases, you only have a finite set of client IDs to use in an integration. In this case, it is possible to assign static client ID sets to individual nodes while configuring the connector. To configure static client IDs, provide a list of client IDs for each node in your EMQX cluster. For each client ID, you can specify the corresponding username and password. This is particularly useful for scenarios like connecting to Azure IoT Hub, where each device (client ID) requires a unique set of credentials. + +To configure static client IDs, follow these steps: -| Node | Client IDs | -| :-------------- | ------------------------ | -| `emqx@10.0.0.1` | `clientid1`, `clientid3` | -| `emqx@10.0.0.2` | `clientid2` | -| `emqx@10.0.0.3` | `clientid4`, `clientid5` | +1. In the **Static ClientId Entries** section, click the **Add** button to add a new static client ID entry. You can add multiple entries for different nodes as needed. +2. For each entry, fill in the following fields: + - **Node Name**: Specify the node where the client ID will be assigned. For example, `emqx@10.0.0.1`. + - **Client ID**: Enter the static client ID. For example, `device1`. You can add multiple client IDs for a node as needed by clicking **Add** . + - **Username**: (optional) Provide the username associated with this client ID for authentication. + - **Password**: (optional) Enter the password associated with this client ID. This is the credential used to authenticate the device or client, which may be a device-specific key, secret, or certificate, depending on the platform (e.g., an authentication key in Azure IoT Hub). -Static client IDs can only be configured through the configuration file and are not available for setup through the Dashboard UI. You can define the `static_clientids` parameter for each node individually in configuration files. +You can also define the `static_clientids` parameter for each node individually in configuration files. If static client IDs are configured, only MQTT connections using these client IDs will be started. Any configurations for dynamic client IDs, such as `pool_size` or `clientid_prefix`, will not take effect. From 6cfa0d971d6edd98a79465dbd153cdff60a1e589 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Tue, 16 Sep 2025 10:09:24 +0800 Subject: [PATCH 29/49] Update zh file --- en_US/data-integration/data-bridge-mqtt.md | 24 +++++++--- zh_CN/data-integration/data-bridge-mqtt.md | 53 ++++++++++++++++------ 2 files changed, 58 insertions(+), 19 deletions(-) diff --git a/en_US/data-integration/data-bridge-mqtt.md b/en_US/data-integration/data-bridge-mqtt.md index 7918b4453..31c01f036 100644 --- a/en_US/data-integration/data-bridge-mqtt.md +++ b/en_US/data-integration/data-bridge-mqtt.md @@ -37,9 +37,9 @@ Make sure you know the following: ::: -Before creating an MQTT Broker data integration, you need to obtain the connection information for the remote MQTT service, including: +Before creating an MQTT Broker data integration, you need to obtain the connection information for the remote MQTT service, using EMQX's [online MQTT server](https://www.emqx.com/en/mqtt/public-mqtt5-broker) as an example: -- **MQTT Service Address**: The address and port of the target MQTT service, for example, `broker.emqx.io:1883`. +- **MQTT Service Address**: The address and port of the target MQTT service; in this example, it is `broker.emqx.io:1883`. - **Username**: The username required for the connection. If the target service does not require authentication, this can be left blank. - **Password**: The password required for the connection. If the target service does not require authentication, this can also be left blank. - **Protocol Type**: It is important to determine whether the target service has enabled TLS and whether it is using MQTT over TCP/TLS protocol. Note that the EMQX MQTT bridge currently does not support protocols like MQTT over WebSocket and MQTT over QUIC. @@ -55,7 +55,7 @@ When EMQX is running in cluster mode or when a connection pool is enabled, using ## Create a Connector -This section guides you on how to configure a connection with a remote MQTT server, using EMQX's [online MQTT server](https://www.emqx.com/en/mqtt/public-mqtt5-broker) as an example. +This section guides you on how to configure a connection with a remote MQTT server. 1. Go to the **Integration** -> **Connector** page on the Dashboard. @@ -80,7 +80,7 @@ This section guides you on how to configure a connection with a remote MQTT serv ::: tip - If static client ID entries are defined, only the EMQX nodes that are assigned those client IDs will establish MQTT connections using them. + If static client ID entries are defined, only the EMQX nodes that have been explicitly assigned static client IDs will start MQTT connections. ::: @@ -117,13 +117,25 @@ In some use cases, you only have a finite set of client IDs to use in an integra To configure static client IDs, follow these steps: -1. In the **Static ClientId Entries** section, click the **Add** button to add a new static client ID entry. You can add multiple entries for different nodes as needed. +1. In the **Static ClientId Entries** section, click the **Add** button to add a new static client ID entry. You can add multiple entries of different nodes as needed. + 2. For each entry, fill in the following fields: + - **Node Name**: Specify the node where the client ID will be assigned. For example, `emqx@10.0.0.1`. - - **Client ID**: Enter the static client ID. For example, `device1`. You can add multiple client IDs for a node as needed by clicking **Add** . + - **Client ID**: Enter the static client ID. For example, `device1`. You can add multiple client IDs for a node as needed by clicking **Add**. - **Username**: (optional) Provide the username associated with this client ID for authentication. - **Password**: (optional) Enter the password associated with this client ID. This is the credential used to authenticate the device or client, which may be a device-specific key, secret, or certificate, depending on the platform (e.g., an authentication key in Azure IoT Hub). + **Configuration Example**: + + | Node | Client ID | Username (optional) | Password (optional) | + | --------------- | ----------- | ------------------- | ------------------- | + | `emqx@10.0.0.1` | `clientid1` | `username1` | `secret1` | + | | `clientid3` | | | + | `emqx@10.0.0.2` | `clientid2` | `username2` | | + | `emqx@10.0.0.3` | `clientid4` | | | + | | `clientid5` | | | + You can also define the `static_clientids` parameter for each node individually in configuration files. If static client IDs are configured, only MQTT connections using these client IDs will be started. Any configurations for dynamic client IDs, such as `pool_size` or `clientid_prefix`, will not take effect. diff --git a/zh_CN/data-integration/data-bridge-mqtt.md b/zh_CN/data-integration/data-bridge-mqtt.md index 1b661d2a6..3166d42f7 100644 --- a/zh_CN/data-integration/data-bridge-mqtt.md +++ b/zh_CN/data-integration/data-bridge-mqtt.md @@ -37,9 +37,9 @@ EMQX 的 MQTT 消息桥接具有以下特性和优势： ::: -在创建 MQTT 消息桥接之前，您需要获取远程 MQTT 服务的连接信息，包括： +在创建 MQTT 消息桥接之前，您需要获取远程 MQTT 服务（例如 EMQX 的[在线 MQTT 服务器](https://www.emqx.com/zh/mqtt/public-mqtt5-broker)）的连接信息，包括： -- **MQTT 服务地址**：目标 MQTT 服务的地址和端口，例如 `broker.emqx.io:1883`。 +- **MQTT 服务地址**：目标 MQTT 服务的地址和端口，例如：`broker.emqx.io:1883`。 - **用户名**：进行连接所需的用户名，如果目标服务不需要认证，此项可留空。 - **密码**：进行连接所需的密码，如果目标服务不需要认证，此项也可留空。 - **协议类型**：需要明确目标服务是否启用了 TLS，以及是否使用的是 MQTT over TCP/TLS 协议。值得注意的是，EMQX MQTT 桥接暂时不支持 MQTT over WebSocket 和 MQTT over QUIC 类型的协议。 @@ -53,7 +53,7 @@ EMQX 运行在集群模式下或启用连接池时，多个节点使用相同的 ## 创建连接器 -本节将以 EMQX 的[在线 MQTT 服务器](https://www.emqx.com/zh/mqtt/public-mqtt5-broker)作为远程 MQTT 服务器，指导您如何配置与远程 MQTT 服务器的连接。 +本节将指导您如何配置与远程 MQTT 服务器的连接。 1. 转到 Dashboard **集成** -> **连接器**页面。 @@ -61,13 +61,27 @@ EMQX 运行在集群模式下或启用连接池时，多个节点使用相同的 3. 在**连接器类型**中选择 **MQTT 服务**，点击**下一步**。 -4. 输入连接器 **名称**，要求是大小写英文字母或数字组合，例如 `my_mqtt_bridge`。 +4. 输入连接器**名称**，要求是大小写英文字母或数字组合，例如 `my_mqtt_bridge`。 5. 填写连接相关配置： - - **MQTT 服务地址**：仅支持 MQTT over TCP/TLS，此处设为 `broker.emqx.io:1883`。 + - **MQTT 服务地址**：仅支持 MQTT over TCP/TLS，例如：`broker.emqx.io:1883`。 + - **客户端 ID 前缀**：此处可以留空，实际使用中，指定客户端 ID 前缀可以便于客户端管理，EMQX 会根据客户端 ID 前缀和连接池大小自动生成客户端 ID，具体规则，参考[连接池与客户端 ID 生成规则](#连接池与客户端-id-生成规则)。 - - **用户名**和**密码**：此处可以留空，该服务器不需要认证。 + + - **用户名**和**密码**：如果 MQTT 服务器要求身份认证，请输入与该 Client ID 关联的用户名和密码。如果不需要认证（例如公共 MQTT 服务器），则可以留空。 + + - **Keepalive**：指定需要的 keepalive 间隔。 + + - **MQTT 协议版本**：选择适合该 MQTT 服务器连接的 MQTT 协议版本。 + + - **静态客户端 ID 映射表**：此部分允许为连接器配置静态 Client ID，以确保连接稳定，特别是在连接 Azure IoT Hub 等服务时。更多配置方法参见[配置静态客户端 ID](#配置静态客户端-id)。 + + ::: tip + + 当配置了映射关系后，只有在映射中指定的 EMQX 节点才会创建对应的 MQTT 客户端连接。 + + ::: 其他配置保持默认即可，点击**创建**按钮完成连接器创建。连接器能够同时用于 Sink 和 Source。接下来，您可以基于此连接器创建数据桥接规则。 @@ -98,15 +112,28 @@ myprefix:foo2bd61c44:1 ### 配置静态客户端 ID -在某些使用场景中，您可能只需要在集成中使用有限的一组客户端 ID。在这种情况下，可以在配置连接器时，将静态客户端 ID 分配给集群中每个节点。以下是一个示例配置： +在某些使用场景中，集成时可用的客户端 ID 数量是有限的。在这种情况下，可以在配置连接器时，为集群中的各个节点分配静态的客户端 ID 集合。对于每个客户端 ID，可以指定对应的用户名和密码。这在连接 Azure IoT Hub 等场景中特别有用，因为每个设备（Client ID）都需要一组唯一的凭证。 + +配置静态 Client ID 的步骤如下： + +1. 在**静态客户端 ID 映射表**部分，点击**添加**按钮添加一个新的节点与静态客户端 ID 的映射条目。根据需要，可以添加多个不同节点的条目。 +2. 对于每个条目，填写以下字段： + - **节点名称**：指定客户端 ID 所分配的节点，例如 `emqx@10.0.0.1`。 + - **客户端 ID**：输入静态 Client ID，例如 `device1`。如需为某个节点添加多个客户端 ID，可点击**添加**按钮继续添加。 + - **用户名**：（可选）输入与该客户端 ID 关联的用户名，用于身份认证。 + - **密码**：（可选）输入与该客户端 ID 关联的密码。这是用于设备或客户端认证的凭证，可能是设备专用密钥、密文或证书，具体取决于所连接的平台（例如，在 Azure IoT Hub 中是认证密钥）。 + +**配置示例**: -| 节点 | 客户端 ID | -| --------------- | ------------------------ | -| `emqx@10.0.0.1` | `clientid1`, `clientid3` | -| `emqx@10.0.0.2` | `clientid2` | -| `emqx@10.0.0.3` | `clientid4`, `clientid5` | +| 节点 | 客户端 ID | 用户名（可选） | 密码（可选） | +| --------------- | ----------- | -------------- | ------------ | +| `emqx@10.0.0.1` | `clientid1` | `username1` | `secret1` | +| | `clientid3` | | | +| `emqx@10.0.0.2` | `clientid2` | `username2` | | +| `emqx@10.0.0.3` | `clientid4` | | | +| | `clientid5` | | | -静态客户端 ID 只能通过配置文件进行设置，无法通过 Dashboard UI 配置。您可以在配置文件中为每个节点单独定义 `static_clientids` 参数。 +你也可以在配置文件中为每个节点单独定义 `static_clientids` 参数。 如果配置了静态客户端 ID，则只有使用这些客户端 ID 的 MQTT 连接会被启动。任何动态客户端 ID 的配置（例如 `pool_size` 或 `clientid_prefix`）将不再生效。 From 6c05669a16485294f1d32210efc5edf21ce0d52b Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Tue, 16 Sep 2025 10:16:54 +0800 Subject: [PATCH 30/49] Update en_US/data-integration/data-bridge-mqtt.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- en_US/data-integration/data-bridge-mqtt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/en_US/data-integration/data-bridge-mqtt.md b/en_US/data-integration/data-bridge-mqtt.md index 31c01f036..5f5496590 100644 --- a/en_US/data-integration/data-bridge-mqtt.md +++ b/en_US/data-integration/data-bridge-mqtt.md @@ -66,7 +66,7 @@ This section guides you on how to configure a connection with a remote MQTT serv 4. Enter a **name** for the connector, which must be a combination of upper/lower case letters and numbers, for example, `my_mqtt_bridge`. 5. Configure the connection information: - - **MQTT Broker**: Only supports MQTT over TCP/TLS. For exmaple, `broker.emqx.io:1883`. + - **MQTT Broker**: Only supports MQTT over TCP/TLS. For example, `broker.emqx.io:1883`. - **ClientID Prefix**: This can be left blank. In actual use, specifying a client ID prefix can facilitate client management. EMQX will automatically generate client IDs based on the client ID prefix and the size of the connection pool. For more information, see [Connection Pool and Client ID Generation Rules](#connection-pool-and-client-id-generation-rules). From 7301ac5888cd1886cb68a981fa36beef07debc34 Mon Sep 17 00:00:00 2001 From: YuShifan <894402575bt@gmail.com> Date: Tue, 16 Sep 2025 14:07:56 +0800 Subject: [PATCH 31/49] feat: add TypeScript MCP SDK documentation with calculator demo --- zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md | 4 +- zh_CN/emqx-ai/sdks/mcp-sdk-typescript.md | 380 +++++++++++++++++++++++ 2 files changed, 382 insertions(+), 2 deletions(-) diff --git a/zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md b/zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md index 9ca69e428..e8154c2bc 100644 --- a/zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md +++ b/zh_CN/emqx-ai/sdks/mcp-sdk-paho-c.md @@ -13,7 +13,7 @@ const char* get_temperature_callback(int n_args, property_t *args) { // Read sensor data float temp = read_temperature_sensor(); - + // Return JSON formatted result static char result[64]; snprintf(result, sizeof(result), "{\"temperature\": %.2f}", temp); @@ -61,4 +61,4 @@ CMake 使用可以参考 [paho-mcp-over-mqtt](https://github.com/mqtt-ai/paho-mc ```bash ./demo_mcp_server -``` \ No newline at end of file +``` diff --git a/zh_CN/emqx-ai/sdks/mcp-sdk-typescript.md b/zh_CN/emqx-ai/sdks/mcp-sdk-typescript.md index e69de29bb..d045ca5b0 100644 --- a/zh_CN/emqx-ai/sdks/mcp-sdk-typescript.md +++ b/zh_CN/emqx-ai/sdks/mcp-sdk-typescript.md @@ -0,0 +1,380 @@ +# TypeScript SDK + +本文将使用 [@emqx-ai/mcp-mqtt-sdk](https://github.com/emqx/mcp-typescript-sdk) 创建一个 MCP over MQTT 服务端和客户端。该 SDK 支持浏览器和 Node.js 环境，提供完整的 TypeScript 类型安全支持。 + +本文为了方便快速演示，我们将在 Node.js 环境中运行示例。您也可以轻松地将其集成到浏览器环境中，在 Vue、React 等前端框架中使用。 + +## 创建演示项目 + +首先创建一个新的 Node.js 项目（需要 Node.js >= 18）： + +```bash +mkdir mcp_typescript_demo +cd mcp_typescript_demo +npm init -y +``` + +## 安装依赖 + +安装 TypeScript MCP SDK： + +```bash +# 使用 npm +npm install @emqx-ai/mcp-mqtt-sdk +npm install -D typescript @types/node ts-node + +# 或使用 yarn +yarn add @emqx-ai/mcp-mqtt-sdk +yarn add -D typescript @types/node ts-node + +# 或使用 pnpm +pnpm add @emqx-ai/mcp-mqtt-sdk +pnpm add -D typescript @types/node ts-node +``` + +## 创建简单的 MCP 服务器 + +在 `mcp_typescript_demo` 项目中，创建一个暴露计算器工具和资源的简单 MCP 服务器。创建一个名为 `demo_mcp_server.ts` 的文件，并添加以下代码： + +```typescript +// demo_mcp_server.ts +import { McpMqttServer } from "@emqx-ai/mcp-mqtt-sdk"; + +// 创建 MCP 服务器 +const server = new McpMqttServer({ + host: "mqtt://broker.emqx.io:1883", + serverId: "demo-calculator-server", + serverName: "demo_server/calculator", + name: "Calculator MCP Server", + version: "1.0.0", + description: "一个简单的计算器 MCP 服务器", + capabilities: { + tools: { listChanged: true }, + resources: { listChanged: true }, + }, +}); + +// 添加加法工具 +server.tool( + "add", + "两个数字相加", + { + type: "object", + properties: { + a: { type: "number", description: "第一个数字" }, + b: { type: "number", description: "第二个数字" }, + }, + required: ["a", "b"], + }, + async (params: Record) => { + const { a, b } = params as { a: number; b: number }; + const result = a + b; + return { + content: [ + { + type: "text", + text: `${a} + ${b} = ${result}`, + }, + ], + }; + }, +); + +// 添加乘法工具 +server.tool( + "multiply", + "两个数字相乘", + { + type: "object", + properties: { + a: { type: "number", description: "第一个数字" }, + b: { type: "number", description: "第二个数字" }, + }, + required: ["a", "b"], + }, + async (params: Record) => { + const { a, b } = params as { a: number; b: number }; + const result = a * b; + return { + content: [ + { + type: "text", + text: `${a} × ${b} = ${result}`, + }, + ], + }; + }, +); + +// 添加具体的问候资源 +const names = ["Alice", "Bob", "Charlie", "Diana", "World"]; +names.forEach((name) => { + server.resource( + `greeting://${name}`, + `个性化问候 - ${name}`, + async () => { + return { + contents: [ + { + uri: `greeting://${name}`, + mimeType: "text/plain", + text: `Hello, ${name}! Welcome to our calculator service.`, + }, + ], + }; + }, + { + description: `生成给${name}的个性化问候消息`, + mimeType: "text/plain", + }, + ); +}); + +// 添加服务器状态资源 +server.resource( + "status://server", + "服务器状态", + async () => { + return { + contents: [ + { + uri: "status://server", + mimeType: "application/json", + text: JSON.stringify( + { + name: "Calculator MCP Server", + status: "running", + uptime: process.uptime(), + availableTools: ["add", "multiply"], + timestamp: new Date().toISOString(), + }, + null, + 2, + ), + }, + ], + }; + }, + { + description: "服务器运行状态信息", + mimeType: "application/json", + }, +); + +// 事件处理 +server.on("ready", () => { + console.log("Calculator MCP Server 已启动"); +}); + +server.on("error", (error) => { + console.error("服务器错误:", error); +}); + +// 启动服务器 +async function startServer() { + try { + await server.start(); + } catch (error) { + console.error("启动服务器失败:", error); + process.exit(1); + } +} + +// 优雅关闭 +process.on("SIGINT", async () => { + console.log("正在关闭服务器..."); + await server.stop(); + process.exit(0); +}); + +startServer(); +``` + +## 创建简单的 MCP 客户端 + +在同一个项目中，创建一个连接到服务器并列出可用工具和资源的简单 MCP 客户端。在项目目录中创建一个名为 `demo_mcp_client.ts` 的文件，并添加以下代码： + +```typescript +// demo_mcp_client.ts +import { McpMqttClient } from "@emqx-ai/mcp-mqtt-sdk"; + +// 创建 MCP 客户端 +const client = new McpMqttClient({ + host: "mqtt://broker.emqx.io:1883", + name: "Demo MCP Client", + version: "1.0.0", +}); + +async function onServerDiscovered(server: any) { + console.log(`发现服务器 ${server.name}，正在连接...`); + await client.initializeServer(server.serverId); +} + +async function onServerConnected(server: any, initResult: any) { + if (!initResult) { + console.error(`连接到 ${server.name} 失败`); + return; + } + + console.log(`已连接到 ${server.name}`); + const capabilities = initResult.capabilities; + + // 列出工具 + if (capabilities.tools) { + try { + const tools = await client.listTools(server.serverId); + console.log( + `${server.name} 的工具:`, + tools.map((t) => t.name), + ); + + // 测试加法工具 + if (tools.some((t) => t.name === "add")) { + const result = await client.callTool(server.serverId, "add", { + a: 1, + b: 2, + }); + console.log("调用 add(a=1, b=2) 工具结果:", result.content[0]?.text); + } + + // 测试乘法工具 + if (tools.some((t) => t.name === "multiply")) { + const result = await client.callTool(server.serverId, "multiply", { + a: 3, + b: 4, + }); + console.log( + "调用 multiply(a=3, b=4) 工具结果:", + result.content[0]?.text, + ); + } + } catch (error) { + console.error("工具操作错误:", error); + } + } + + // 列出和读取资源 + if (capabilities.resources) { + try { + const resources = await client.listResources(server.serverId); + console.log( + `${server.name} 的资源:`, + resources.map((r) => r.uri), + ); + + // 读取服务器状态 + if (resources.some((r) => r.uri === "status://server")) { + const status = await client.readResource( + server.serverId, + "status://server", + ); + console.log("服务器状态:", status.contents[0]?.text); + } + + // 读取动态问候资源 + const greeting = await client.readResource( + server.serverId, + "greeting://Alice", + ); + console.log("问候资源:", greeting.contents[0]?.text); + } catch (error) { + console.error("资源操作错误:", error); + } + } +} + +async function onServerDisconnected(serverId: string) { + console.log(`与服务器 ${serverId} 断开连接`); +} + +// 设置事件处理器 +client.on("serverDiscovered", onServerDiscovered); +client.on("serverInitialized", (server) => { + // 这里我们需要获取初始化结果，简化处理 + onServerConnected(server, { capabilities: { tools: true, resources: true } }); +}); +client.on("serverDisconnected", onServerDisconnected); +client.on("error", (error) => { + console.error("客户端错误:", error); +}); + +// 启动客户端 +async function startClient() { + try { + await client.connect(); + console.log("Demo MCP Client 已启动"); + + // 保持运行 + while (true) { + // 模拟其他工作，同时 MQTT 传输客户端在后台运行 + await new Promise((resolve) => setTimeout(resolve, 20000)); + } + } catch (error) { + console.error("启动客户端失败:", error); + process.exit(1); + } +} + +// 优雅关闭 +process.on("SIGINT", async () => { + console.log("正在关闭客户端..."); + await client.disconnect(); + process.exit(0); +}); + +startClient(); +``` + +## 配置项目 + +由于 SDK 使用 ES 模块，需要配置项目支持现代 JavaScript 模块系统。 + +在 `package.json` 中添加模块类型和运行脚本： + +```json +{ + "type": "module", + "scripts": { + "start:server": "ts-node --esm demo_mcp_server.ts", + "start:client": "ts-node --esm demo_mcp_client.ts" + } +} +``` + +创建 `tsconfig.json` 配置文件： + +```json +{ + "compilerOptions": { + "target": "ES2022", + "module": "ESNext", + "moduleResolution": "node", + "esModuleInterop": true, + "allowSyntheticDefaultImports": true, + "strict": false, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true + }, + "ts-node": { + "esm": true + } +} +``` + +## 运行演示 + +1. 首先运行客户端： + +```bash +npm run start:client +``` + +2. 打开新终端并运行服务器： + +```bash +npm run start:server +``` + +即便客户端先于服务器启动，它也会发现服务器并连接到它。客户端将列出可用的工具，并使用参数 `a=1` 和 `b=2` 调用 `add` 工具，使用参数 `a=3` 和 `b=4` 调用 `multiply` 工具。 + +通过这个完整的演示，您已经成功创建了一个功能完整的 MCP over MQTT 系统。现在，大模型（如 DeepSeek、Claude、GPT、Gemini 等）就可以通过 MCP 协议发现并调用您暴露的计算器工具，实现与外部服务的无缝集成和智能交互。 From 473c480d5dab9ebf426d9d636bc2803aa00865f4 Mon Sep 17 00:00:00 2001 From: YuShifan <894402575bt@gmail.com> Date: Tue, 16 Sep 2025 14:46:07 +0800 Subject: [PATCH 32/49] feat: add TypeScript WebRTC integration documentation with multimedia AI services --- .../sdks/multimedia-ai/webrtc-typescript.md | 459 +++++++++++++++++- 1 file changed, 458 insertions(+), 1 deletion(-) diff --git a/zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md b/zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md index 2f2cae53b..3346cb854 100644 --- a/zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md +++ b/zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md @@ -1 +1,458 @@ -# Typescript WebRTC 示例 +# TypeScript WebRTC 示例 + +本文档演示如何使用 TypeScript 通过 MQTT 信令与 EMQX 多媒体服务进行 WebRTC 交互，实现音视频通话、语音识别和文字转语音功能。 + +EMQX 多媒体服务通过 MQTT 协议进行 WebRTC 信令交换，客户端可以轻松集成音视频通话功能，并享受 AI 驱动的语音识别（ASR）和文字转语音（TTS）服务。 + +## 核心架构 + +```shell +TypeScript Client ←→ MQTT 信令 ←→ EMQX 多媒体服务 + ↓ + WebRTC P2P 连接 +``` + +**关键 MQTT 主题**: +- `$webrtc/{device_id}` - 接收服务端的 SDP answer 和 ICE candidates +- `$webrtc/{device_id}/multimedia_proxy` - 发送 SDP offer 和 ICE candidates +- `$message/{device_id}` - 接收 ASR、TTS 和消息数据 + +## 核心实现 + +### WebRTC 信令类详解 + +`MqttWebRTCSignaling` 是核心的信令管理类，负责处理 MQTT 消息传递、WebRTC 连接建立和多媒体服务交互。该类封装了复杂的信令逻辑，提供简洁的 API 供开发者使用。 + +```typescript +import type { MqttClient } from 'mqtt' + +interface SignalingMessage { + type: 'sdp_offer' | 'sdp_answer' | 'ice_candidate' | 'webrtc_terminated' + | 'asr_response' | 'tts_begin' | 'tts_text' | 'tts_complete' | 'tts_terminate' + | 'chat' | 'message' + data?: any + payload?: any + reason?: string + results?: string + text?: string + task_id?: string +} + +interface WebRTCCallbacks { + onLocalStream?: (stream: MediaStream) => void + onRemoteStream?: (stream: MediaStream) => void + onConnectionStateChange?: (state: string) => void + onASRResponse?: (text: string) => void + onTTSText?: (text: string) => void + onMessage?: (message: string) => void + onError?: (error: Error) => void +} + +export class MqttWebRTCSignaling { + private mqttClient: MqttClient + private pc: RTCPeerConnection | null = null + private localStream: MediaStream | null = null + private remoteStream: MediaStream | null = null + private clientId: string + private callbacks: WebRTCCallbacks + private messageHandler: ((topic: string, message: Buffer) => void) | null = null + private ttsText: string = '' + + constructor(options: { + mqttClient: MqttClient + clientId: string + callbacks?: WebRTCCallbacks + }) { + this.mqttClient = options.mqttClient + this.clientId = options.clientId + this.callbacks = options.callbacks || {} + } + + async connect(): Promise { + if (!this.mqttClient.connected) { + throw new Error('MQTT client is not connected') + } + + // 订阅主题 + await this.subscribeToTopics() + + // 设置消息处理 + this.setupMessageHandler() + + // 初始化 WebRTC + await this.setupWebRTC() + } + + private async subscribeToTopics(): Promise { + const topics = [ + `$webrtc/${this.clientId}`, + `$message/${this.clientId}` + ] + + return new Promise((resolve, reject) => { + this.mqttClient.subscribe(topics, { qos: 0 }, (err) => { + if (err) reject(err) + else resolve() + }) + }) + } + + private setupMessageHandler(): void { + this.messageHandler = (topic: string, message: Buffer) => { + if (topic === `$webrtc/${this.clientId}` || topic === `$message/${this.clientId}`) { + try { + const payload = JSON.parse(message.toString()) as SignalingMessage + this.handleSignalingMessage(topic, payload) + } catch (error) { + console.error('Failed to parse message:', error) + } + } + } + + this.mqttClient.on('message', this.messageHandler) + } + + private async handleSignalingMessage(topic: string, message: SignalingMessage): Promise { + if (topic === `$webrtc/${this.clientId}`) { + // WebRTC 信令处理 + switch (message.type) { + case 'sdp_answer': + if (this.pc && message.data) { + const answer = new RTCSessionDescription({ + type: 'answer', + sdp: message.data.sdp || message.data + }) + await this.pc.setRemoteDescription(answer) + } + break + + case 'ice_candidate': + if (this.pc && message.data) { + const candidate = new RTCIceCandidate(message.data) + await this.pc.addIceCandidate(candidate) + } + break + + case 'webrtc_terminated': + this.callbacks.onError?.(new Error(`WebRTC terminated: ${message.reason}`)) + this.disconnect() + break + } + } else if (topic === `$message/${this.clientId}`) { + // 多媒体服务消息处理 + switch (message.type) { + case 'asr_response': + this.callbacks.onASRResponse?.(message.results || '') + break + + case 'tts_begin': + this.ttsText = '' + break + + case 'tts_text': + this.ttsText += message.text || '' + this.callbacks.onTTSText?.(this.ttsText) + break + + case 'tts_complete': + console.log('TTS complete:', message.task_id) + break + + case 'tts_terminate': + console.log('TTS terminated:', message.task_id) + break + + case 'message': + this.callbacks.onMessage?.(message.payload || '') + break + } + } + } + + private async setupWebRTC(): Promise { + // 获取用户媒体 + this.localStream = await navigator.mediaDevices.getUserMedia({ + video: true, + audio: { + echoCancellation: true, + noiseSuppression: true, + autoGainControl: true, + sampleRate: 48000 + } + }) + + this.callbacks.onLocalStream?.(this.localStream) + + // 创建 RTCPeerConnection + this.pc = new RTCPeerConnection({ + iceServers: [{ urls: 'stun:stun.l.google.com:19302' }] + }) + this.remoteStream = new MediaStream() + + // 添加本地流 + this.localStream.getTracks().forEach(track => { + this.pc!.addTrack(track, this.localStream!) + }) + + // 设置事件处理 + this.setupPeerConnectionHandlers() + + // 创建并发送 offer + const offer = await this.pc.createOffer() + await this.pc.setLocalDescription(offer) + this.sendSignal('sdp_offer', offer) + } + + private setupPeerConnectionHandlers(): void { + if (!this.pc) return + + // ICE 候选处理 + this.pc.onicecandidate = (event) => { + if (event.candidate) { + this.sendSignal('ice_candidate', event.candidate) + } + } + + // 连接状态变化 + this.pc.onconnectionstatechange = () => { + if (this.pc) { + this.callbacks.onConnectionStateChange?.(this.pc.connectionState) + + if (this.pc.connectionState === 'failed') { + this.callbacks.onError?.(new Error('Connection failed')) + } + } + } + + // 远程流处理 + this.pc.ontrack = (event) => { + if (this.remoteStream) { + this.remoteStream.addTrack(event.track) + this.callbacks.onRemoteStream?.(this.remoteStream) + } + } + } + + private sendSignal(type: string, data: any): void { + const message = JSON.stringify({ type, data }) + const topic = `$webrtc/${this.clientId}/multimedia_proxy` + + this.mqttClient.publish(topic, message, { qos: 0 }, (err) => { + if (err) { + console.error(`Failed to send ${type}:`, err) + } + }) + } + + // 音视频控制 + toggleAudio(enabled?: boolean): void { + if (!this.localStream) return + + this.localStream.getAudioTracks().forEach(track => { + track.enabled = enabled !== undefined ? enabled : !track.enabled + }) + } + + toggleVideo(enabled?: boolean): void { + if (!this.localStream) return + + this.localStream.getVideoTracks().forEach(track => { + track.enabled = enabled !== undefined ? enabled : !track.enabled + }) + } + + // 发送聊天消息 + sendChatMessage(message: string): void { + this.sendSignal('chat', message) + } + + disconnect(): void { + // 停止媒体流 + if (this.localStream) { + this.localStream.getTracks().forEach(track => track.stop()) + this.localStream = null + } + + if (this.remoteStream) { + this.remoteStream.getTracks().forEach(track => track.stop()) + this.remoteStream = null + } + + // 关闭 peer connection + if (this.pc) { + this.pc.close() + this.pc = null + } + + // 清理消息处理器 + if (this.mqttClient && this.messageHandler) { + this.mqttClient.removeListener('message', this.messageHandler) + this.messageHandler = null + } + + // 取消订阅 + const topics = [`$webrtc/${this.clientId}`, `$message/${this.clientId}`] + this.mqttClient.unsubscribe(topics) + + this.callbacks.onConnectionStateChange?.('disconnected') + } +} +``` + +## 使用示例 + +以下是基础的使用示例，展示核心 API 的使用方法。该 WebRTC 信令类可以在各种前端环境中使用，包括 Vue、React 等框架，也可以直接在原生 JavaScript 和 HTML 中实现。 + +完整的实现示例可以查看：[MCP AI Companion Demo](https://github.com/emqx/mcp-ai-companion-demo) 项目中的 web 部分。 + +### 基础连接和回调设置 + +创建 WebRTC 连接需要先建立 MQTT 连接，然后配置各种回调函数来处理音视频流和 AI 服务响应。 + +```typescript +import mqtt from 'mqtt' +import { MqttWebRTCSignaling } from './mqtt-webrtc-signaling' + +// 连接 MQTT +const mqttClient = mqtt.connect('ws://broker.emqx.io:8083/mqtt', { + clientId: 'device_123', + username: 'your-username', + password: 'your-password' +}) + +// 创建 WebRTC 信令 +const signaling = new MqttWebRTCSignaling({ + mqttClient, + clientId: 'device_123', + callbacks: { + onLocalStream: (stream) => { + document.getElementById('localVideo').srcObject = stream + }, + onRemoteStream: (stream) => { + document.getElementById('remoteVideo').srcObject = stream + }, + onASRResponse: (text) => { + console.log('语音识别:', text) + }, + onTTSText: (text) => { + console.log('AI 回复:', text) + }, + onConnectionStateChange: (state) => { + console.log('连接状态:', state) + } + } +}) + +// 开始连接 +await signaling.connect() +``` + +### 多媒体功能控制 + +连接建立后，可以通过简单的 API 调用来控制音视频设备、发送消息和管理连接状态。 + +```typescript +// 音视频控制 +signaling.toggleAudio(false) // 静音 +signaling.toggleVideo(true) // 开启摄像头 + +// 发送聊天消息 +signaling.sendChatMessage('Hello AI!') + +// 断开连接 +signaling.disconnect() +``` + +## 关键交互说明 + +### MQTT 信令交换流程 + +WebRTC 连接建立通过 MQTT 主题进行信令交换，遵循标准的 offer/answer 模式： + +``` +1. 客户端创建 offer → 发送到 $webrtc/{device_id}/multimedia_proxy +2. 服务端处理 offer → 返回 answer 到 $webrtc/{device_id} +3. 双方交换 ICE candidates → 通过上述两个主题进行 +4. 建立 P2P 连接 → 开始音视频传输 +``` + +### 多媒体 AI 服务 + +EMQX 多媒体服务提供三种核心 AI 功能，通过 `$message/{device_id}` 主题进行数据交换： + +- **ASR (语音识别)**: 实时识别用户语音并返回文本结果，支持流式识别 +- **TTS (文字转语音)**: 将文本转换为自然语音，支持流式文本推送 +- **智能对话**: 结合 ASR 和 TTS，实现完整的语音交互体验 + +### 错误处理和最佳实践 + +在实际应用中，需要处理各种可能出现的错误情况： + +```typescript +// 常见错误处理 +callbacks: { + onError: (error) => { + if (error.message.includes('getUserMedia')) { + alert('无法访问摄像头/麦克风，请检查权限') + } else if (error.message.includes('Connection failed')) { + // 自动重连逻辑 + setTimeout(() => signaling?.connect(), 2000) + } + } +} +``` + +**常见问题解决方案**： +- **媒体权限问题**: 确保浏览器已授权摄像头和麦克风访问 +- **网络连接问题**: 实现自动重连机制，处理网络波动 +- **信令超时**: 设置合理的连接超时时间，提供用户友好的错误提示 + +## 高级配置 + +### ICE 服务器配置 + +为确保 WebRTC 连接在各种网络环境下正常工作，需要配置适当的 ICE 服务器： + +```typescript +const iceServers = [ + { urls: 'stun:stun.l.google.com:19302' }, // 公共 STUN 服务器 + { + urls: 'turn:your-turn-server.com:3478', // 私有 TURN 服务器（生产环境推荐） + username: 'turnuser', + credential: 'turnpass' + } +] +``` + +### 媒体约束优化 + +根据应用场景调整音视频质量参数，平衡质量和带宽消耗： + +```typescript +const mediaConstraints = { + video: { + width: { ideal: 1280 }, // 视频宽度 + height: { ideal: 720 }, // 视频高度 + frameRate: { ideal: 30 } // 帧率 + }, + audio: { + echoCancellation: true, // 回声消除 + noiseSuppression: true, // 噪声抑制 + autoGainControl: true, // 自动增益控制 + sampleRate: 48000, // 采样率 + sampleSize: 16, // 采样位深 + channelCount: 2 // 声道数 + } +} +``` + +## 总结 + +通过 EMQX 多媒体服务的 TypeScript WebRTC 集成，你可以轻松实现： + +- ✅ **音视频通话**: 高质量的实时音视频传输 +- ✅ **AI 语音识别**: 实时语音转文字，支持多语言 +- ✅ **智能语音合成**: 自然流畅的文字转语音 +- ✅ **跨平台兼容**: 支持各种前端框架和原生 JavaScript + +这为构建现代化的 AI 驱动通信应用提供了完整的技术基础。 From bcdc1f0ab38601cd57a1657232a7510bbe104a86 Mon Sep 17 00:00:00 2001 From: Shawn <506895667@qq.com> Date: Tue, 16 Sep 2025 15:40:12 +0800 Subject: [PATCH 33/49] remove en docs and we will add it back after zh docs ready --- en_US/emqx-ai/multimedia-ai/architecture.md | 47 -------------------- en_US/emqx-ai/multimedia-ai/overview.md | 23 ---------- en_US/emqx-ai/sdks/multimedia-ai/overview.md | 11 ----- 3 files changed, 81 deletions(-) diff --git a/en_US/emqx-ai/multimedia-ai/architecture.md b/en_US/emqx-ai/multimedia-ai/architecture.md index 2d75f91f3..c79bec1ac 100644 --- a/en_US/emqx-ai/multimedia-ai/architecture.md +++ b/en_US/emqx-ai/multimedia-ai/architecture.md @@ -1,48 +1 @@ # Architecture - -The following diagram illustrates the architecture of a typical multimedia AI system built using EMQX AI. - -```mermaid -flowchart LR - Devices <-- WebRTC --> MultimediaServer - MultimediaServer <-- STDIO --> AIAgents - MultimediaServer <--> ASRTTS - AIAgents <--> LLM - - Devices[Devices] - MultimediaServer[Multimedia
Server] - AIAgents[AI Agents] - ASRTTS[ASR/TTS] - LLM[LLM] -``` - -The following diagram illustrates the interaction flow between the components: - -```mermaid -sequenceDiagram - actor Customer as Device - participant LoginPage as Multimedia Server - participant P1 as AI Agents - participant P2 as LLM - - Customer ->>+ LoginPage: WebRTC Audio - LoginPage ->> LoginPage: ASR - LoginPage ->> P1: ASR Results - P1 ->> P2: Process ASR with MCP Tools - P2 ->> P1: LLM Results - P1 ->> P1: Process LLM Results - P1 ->> LoginPage: TTS and send to Device - LoginPage ->> LoginPage: TTS - LoginPage ->> Customer: WebRTC Audio - Customer ->> LoginPage: WebRTC Video - P1 ->> LoginPage: Realtime Image Analysis - LoginPage ->> LoginPage: Image Analysis - LoginPage ->> P1: Image Analysis Result - P1 ->> P2: Summary the Analysis Reuslt - P2 ->> P1: Summary - P1 ->> LoginPage: TTS and send to Device - LoginPage ->> Customer: WebRTC Audio - P1 ->> P1: Some other processing - P1 ->> LoginPage: Send message to Device - LoginPage ->> Customer: MQTT message -``` diff --git a/en_US/emqx-ai/multimedia-ai/overview.md b/en_US/emqx-ai/multimedia-ai/overview.md index 66a3a2570..e69de29bb 100644 --- a/en_US/emqx-ai/multimedia-ai/overview.md +++ b/en_US/emqx-ai/multimedia-ai/overview.md @@ -1,23 +0,0 @@ -# Multimedia Server - -The EMQX Multimedia Server is a high-performance multimedia processing platform built on WebRTC technology. It can receive RTP/SRTP audio and video streams from clients and integrates various AI features such as Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and image understanding. Leveraging large model capabilities, the EMQX Multimedia Server supports complex voice conversations and tool invocation, providing robust technical support for AI applications requiring audio and video capabilities. - -## Core Features - -- **Real-time Audio and Video Processing**: Supports high-quality audio and video streaming, ensuring low latency and highly reliable communication. -- **Automatic Speech Recognition (ASR)**: Delivers accurate speech-to-text functionality, suitable for voice assistants, intelligent customer service, and more. -- **Text-to-Speech (TTS)**: Offers text-to-speech services in multiple languages and voice styles, enhancing user interaction experiences. -- **Image Understanding**: Integrates advanced image recognition and analysis technologies, supporting various image processing applications. -- **Large Model Support**: Utilizes large model capabilities to enable complex voice conversations and tool invocation, meeting diverse business needs. -- **Highly Flexible Architecture**: Adapts to applications of different scales and complexities, supporting horizontal scaling and customizable configurations. -- **High Reliability**: Employs distributed architecture design to ensure high availability and system stability. -- **Low Latency**: Optimized network transmission and large model processing mechanisms guarantee smooth real-time interactions. - -## Application Scenarios - -EMQX Multimedia Server is suitable for the following scenarios: - -- **Emotional Companionship**: Provides personalized emotional companionship services through voice conversations and emotion recognition technologies. -- **Intelligent Customer Service**: Uses ASR and TTS technologies to enable efficient voice interactions and enhance customer service experiences. -- **Intelligent Device Control**: Enables convenient control of smart devices through voice commands and image recognition. - diff --git a/en_US/emqx-ai/sdks/multimedia-ai/overview.md b/en_US/emqx-ai/sdks/multimedia-ai/overview.md index d240fe096..e69de29bb 100644 --- a/en_US/emqx-ai/sdks/multimedia-ai/overview.md +++ b/en_US/emqx-ai/sdks/multimedia-ai/overview.md @@ -1,11 +0,0 @@ -# Clients Compatible with Multimedia Services - -Clients that support the WebRTC protocol can interact with multimedia AI services. Common clients include: - -- **Web Browsers**: Modern browsers (such as Chrome, Firefox, Edge, Safari) support WebRTC and can directly access multimedia AI services. - -- **Mobile Applications**: By integrating WebRTC SDKs (such as [Pion](https://pion.ly)), mobile apps can interact with multimedia AI services. - -- **Embedded Devices**: IoT devices can connect to multimedia AI services by integrating device-adapted WebRTC libraries, such as [ESP32 WebRTC](https://github.com/espressif/esp-webrtc-solution). - -Here we provide a sample client code based on a web browser, demonstrating how to interact with multimedia services: [Typescript WebRTC Example](./webrtc-typescript.md). From 74137392421a4a36160a550dbe8ad868d94f247e Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Tue, 16 Sep 2025 16:45:00 +0800 Subject: [PATCH 34/49] Update changes-ee-v5.md --- en_US/changes/changes-ee-v5.md | 56 ++++++++++++++++------------------ 1 file changed, 27 insertions(+), 29 deletions(-) diff --git a/en_US/changes/changes-ee-v5.md b/en_US/changes/changes-ee-v5.md index 77fef2c98..ced6d55de 100644 --- a/en_US/changes/changes-ee-v5.md +++ b/en_US/changes/changes-ee-v5.md @@ -62,11 +62,27 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ * `error: {function_clause,[{gen_tcp,send,[closed,[]],[{file,“gen_tcp.erl”},{line,966}]},{cowboy_websocket_linger,commands,3,[{file,“cowboy_websocket_linger.erl”},{line,665}]},...` * `message: {tcp,#Port<0.364>,<<136,130,...>>}, msg: emqx_session_mem_unknown_message` +- [#15872](https://github.com/emqx/emqx/pull/15872) Eliminated warning log `unclean_terminate` when disconnected after CONNACK is sent with a non-zero reason code. + #### Data Integration - [#15394](https://github.com/emqx/emqx/pull/15394) Fixed a rare race condition where Action metrics could become inconsistent due to unexpected asynchronous replies. - [#15603](https://github.com/emqx/emqx/pull/15603) Fixed an issue in the MQTT bridge where a stale connection could be shown as `Connected` and would not automatically reconnect. +- [#15826](https://github.com/emqx/emqx/pull/15826) Improved Kafka consumer connector health check behavior with restricted ACLs. .Previously, Kafka Consumer Connector health checks could fail if the configured user lacked permission to access the internal `____emqx_consumer_probe` consumer group used for the check. With this fix, if the Kafka broker returns an "ACL denied" response, EMQX will treat the connection as healthy. +- [#15827](https://github.com/emqx/emqx/pull/15827) Fixed atom and process leaks in the GreptimeDB driver. + + Fixed a `function_clause` error that could arise if certain incorrect write syntaxes were used in GreptimeDB Actions. + +- [#15836](https://github.com/emqx/emqx/pull/15836) Enriched the returned information when a Kafka Consumer Source fails to be added, for example, due to denied topic ACLs. + +- [#15850](https://github.com/emqx/emqx/pull/15850) Fixed an issue with the MQTT bridge when a stale connection was displayed as `Connected` and the connection was not re-established. + +- [#15866](https://github.com/emqx/emqx/pull/15866) Upgrade Kafka producer lib wollf to 4.0.12 to improve handling of temporarily missing partitions in Kafka metadata responses. + + In rare race conditions, Kafka may return an incomplete partition list. + Previously, this was only handled when a topic was recreated with fewer partitions, but not when partitions were temporarily missing. + This gap could cause the partition producer to stall and block shutdown indefinitely. #### Deployment @@ -78,36 +94,30 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ In previous EMQX versions (before 5.9), a bug in the ZIP timestamp encoder could store an invalid “seconds” value in archive entries (values corresponding to the 30th or 31st 2-second slot in DOS time format). -#### Clustering +- [#15863](https://github.com/emqx/emqx/pull/15863) Fixed license quota alarm text. +#### Clustering -- [#15788](https://github.com/emqx/emqx/pull/15788) Etcd cluster discovery issue - Resolved an issue where EMQX nodes from different clusters could mistakenly join each other when using a shared etcd server. - This was caused by a bug in the etcd client library. +- [#15788](https://github.com/emqx/emqx/pull/15788) Fixed etcd cluster discovery issue. Resolved an issue where EMQX nodes from different clusters could mistakenly join each other when using a shared etcd server. This was caused by a bug in the etcd client library. -- [#15794](https://github.com/emqx/emqx/pull/15794) Ensure that any changes to connection rate limits take effect immediately after the listener update has completed. Previously, parts of internal limiter state were not directly affected by configuration changes. For example, after increasing the burst rate, the effective rate limit could appear stricter than expected. +#### Rate Limit -- [#15810](https://github.com/emqx/emqx/pull/15810) The original `sparkplug_{en,de}code` rule functions, when handling `bytes_value` metrics values, did not base64 decode/encode the data, respectively, thus not following the Protobuf spec. - https://protobuf.dev/programming-guides/json/ +- [#15794](https://github.com/emqx/emqx/pull/15794) Improved the behavior of connection rate limit updates to ensure that changes (e.g., to burst rate or rate thresholds) are applied immediately after the listener configuration is updated. Previously, parts of the internal limiter state were not refreshed correctly, which could result in rate limits appearing stricter than configured. - Thus, here, we introduce new `spb_{en,de}code` rule functions that translate such fields, to avoid breaking backwards compatibility, and deprecate the old `sparkplug_{en,de}code` rule functions. +#### Smart Data Hub -- [#15818](https://github.com/emqx/emqx/pull/15818) Corrected handling of `{allow|deny, all}` ACL rules. - Previously, these rules were internally translated to match `#`, which incorrectly failed to match topics prefixed with `$` (e.g. `$testtopic/1`) due to MQTT spec restrictions. - Now, a special internal value is used to ensure `{allow|deny, all}` rules correctly match any topic, including `$`-prefixed ones. +- [#15810](https://github.com/emqx/emqx/pull/15810) Introduced `spb_{en,de}code` functions to ccorrect handling of `bytes_value` Metrics. Fixed an issue with the original `sparkplug_{en,de}code` functions, which did not base64 encode/decode `bytes_value` metric values as required by the [Protobuf specification](https://protobuf.dev/programming-guides/json/). To address this, new `spb_{en,de}code` functions have been introduced for correct encoding/decoding of such fields. The old `sparkplug_{en,de}code` functions are now deprecated to maintain backward compatibility. - +#### Access Control -- [#15826](https://github.com/emqx/emqx/pull/15826) Previously, if the user used in a Kafka Consumer Connector did not have permissions to read the special `____emqx_consumer_probe` group used for health checks, the health check would fail. Now, if the Kafka broker returns an ACL denied response, the connection is considered healthy. -- [#15827](https://github.com/emqx/emqx/pull/15827) Fixed atom and process leaks in the GreptimeDB driver. - - Fixed a `function_clause` error that could arise if certain incorrect write syntaxes were used in GreptimeDB Actions. +- [#15818](https://github.com/emqx/emqx/pull/15818) Corrected handling of `{allow|deny, all}` ACL rules. -- [#15836](https://github.com/emqx/emqx/pull/15836) Enriched the returned information when a Kafka Consumer Source fails to be added, for example, due to denied topic ACLs. + Previously, these rules were internally translated to match `#`, which incorrectly failed to match topics prefixed with `$` (e.g. `$testtopic/1`) due to MQTT spec restrictions. + Now, a special internal value is used to ensure `{allow|deny, all}` rules correctly match any topic, including `$`-prefixed ones. - [#15844](https://github.com/emqx/emqx/pull/15844) Added validation to forbid adding empty usernames to the built-in database authenticator. Such users cannot be deleted via the HTTP API later, since they mess up the API path. @@ -117,18 +127,6 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ mria:transaction(emqx_authn_shard, fun() -> mnesia:delete(emqx_authn_mnesia, {'mqtt:global',<<>>}, write) end). ``` -- [#15850](https://github.com/emqx/emqx/pull/15850) Fixed an issue with the MQTT bridge when a stale connection was displayed as `Connected' and the connection was not re-established. - -- [#15863](https://github.com/emqx/emqx/pull/15863) Fix license quota alarm text. - -- [#15866](https://github.com/emqx/emqx/pull/15866) Upgrade Kafka producer lib wollf to 4.0.12 to improve handling of temporarily missing partitions in Kafka metadata responses. - - In rare race conditions, Kafka may return an incomplete partition list. - Previously, this was only handled when a topic was recreated with fewer partitions, but not when partitions were temporarily missing. - This gap could cause the partition producer to stall and block shutdown indefinitely. - -- [#15872](https://github.com/emqx/emqx/pull/15872) Eliminate warning log 'unclean_terminate' when disconnected after CONNACK is sent with a non-zero reason code. - ## 5.10.0 *Release Date: 2025-06-10* From 79017eeebfe332d35d5a9f2bc2913897ed914046 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Tue, 16 Sep 2025 16:48:05 +0800 Subject: [PATCH 35/49] Update en_US/changes/changes-ee-v5.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- en_US/changes/changes-ee-v5.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/en_US/changes/changes-ee-v5.md b/en_US/changes/changes-ee-v5.md index ced6d55de..c32f1a14f 100644 --- a/en_US/changes/changes-ee-v5.md +++ b/en_US/changes/changes-ee-v5.md @@ -109,7 +109,7 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ #### Smart Data Hub -- [#15810](https://github.com/emqx/emqx/pull/15810) Introduced `spb_{en,de}code` functions to ccorrect handling of `bytes_value` Metrics. Fixed an issue with the original `sparkplug_{en,de}code` functions, which did not base64 encode/decode `bytes_value` metric values as required by the [Protobuf specification](https://protobuf.dev/programming-guides/json/). To address this, new `spb_{en,de}code` functions have been introduced for correct encoding/decoding of such fields. The old `sparkplug_{en,de}code` functions are now deprecated to maintain backward compatibility. +- [#15810](https://github.com/emqx/emqx/pull/15810) Introduced `spb_{en,de}code` functions to correct handling of `bytes_value` Metrics. Fixed an issue with the original `sparkplug_{en,de}code` functions, which did not base64 encode/decode `bytes_value` metric values as required by the [Protobuf specification](https://protobuf.dev/programming-guides/json/). To address this, new `spb_{en,de}code` functions have been introduced for correct encoding/decoding of such fields. The old `sparkplug_{en,de}code` functions are now deprecated to maintain backward compatibility. #### Access Control From 5434e04a93f73afb513229c79024e2ce19b9e72c Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Tue, 16 Sep 2025 16:49:33 +0800 Subject: [PATCH 36/49] Update en_US/changes/changes-ee-v5.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- en_US/changes/changes-ee-v5.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/en_US/changes/changes-ee-v5.md b/en_US/changes/changes-ee-v5.md index c32f1a14f..6d230bc20 100644 --- a/en_US/changes/changes-ee-v5.md +++ b/en_US/changes/changes-ee-v5.md @@ -31,7 +31,7 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ Also introduced a new `resource_opts.health_check_interval_jitter`, which adds a uniform random delay to `resource_opts.health_check_interval` to reduce the chance of multiple Actions under the same Connector running health checks at the same time. -- [#15542](https://github.com/emqx/emqx/pull/15542) Upgraded our `erlcoud` library to `3.8.3.0`. This allows one to setup a S3 Connector without specifying Access Key Id and Secret Access Key, so long as the EC2 instance EMQX is running in has the correct IAM permissions to read/write to the configured bucket(s). +- [#15542](https://github.com/emqx/emqx/pull/15542) Upgraded our `erlcloud` library to `3.8.3.0`. This allows one to setup a S3 Connector without specifying Access Key Id and Secret Access Key, so long as the EC2 instance EMQX is running in has the correct IAM permissions to read/write to the configured bucket(s). - [#15845](https://github.com/emqx/emqx/pull/15845) The `static_clientids` configuration for the MQTT Connector now supports specifying a username and password for each client ID. This is particularly useful for scenarios like connecting to Azure IoT Hub, where each device (client ID) requires a unique set of credentials. This enhancement helps ensure successful connections across multiple nodes in a clustered environment. #### CLI From e6d95dfc0bcca0352c1225d0e8caf22d2726de4c Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Tue, 16 Sep 2025 16:49:44 +0800 Subject: [PATCH 37/49] Update en_US/changes/changes-ee-v5.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- en_US/changes/changes-ee-v5.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/en_US/changes/changes-ee-v5.md b/en_US/changes/changes-ee-v5.md index 6d230bc20..4fd4a9692 100644 --- a/en_US/changes/changes-ee-v5.md +++ b/en_US/changes/changes-ee-v5.md @@ -69,7 +69,7 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ - [#15394](https://github.com/emqx/emqx/pull/15394) Fixed a rare race condition where Action metrics could become inconsistent due to unexpected asynchronous replies. - [#15603](https://github.com/emqx/emqx/pull/15603) Fixed an issue in the MQTT bridge where a stale connection could be shown as `Connected` and would not automatically reconnect. -- [#15826](https://github.com/emqx/emqx/pull/15826) Improved Kafka consumer connector health check behavior with restricted ACLs. .Previously, Kafka Consumer Connector health checks could fail if the configured user lacked permission to access the internal `____emqx_consumer_probe` consumer group used for the check. With this fix, if the Kafka broker returns an "ACL denied" response, EMQX will treat the connection as healthy. +- [#15826](https://github.com/emqx/emqx/pull/15826) Improved Kafka consumer connector health check behavior with restricted ACLs. Previously, Kafka Consumer Connector health checks could fail if the configured user lacked permission to access the internal `____emqx_consumer_probe` consumer group used for the check. With this fix, if the Kafka broker returns an "ACL denied" response, EMQX will treat the connection as healthy. - [#15827](https://github.com/emqx/emqx/pull/15827) Fixed atom and process leaks in the GreptimeDB driver. Fixed a `function_clause` error that could arise if certain incorrect write syntaxes were used in GreptimeDB Actions. From f2c0f78a4f28a4c30c7215e30d764a1d0826e11a Mon Sep 17 00:00:00 2001 From: "emqx-ci-bot[bot]" Date: Tue, 16 Sep 2025 14:17:34 +0000 Subject: [PATCH 38/49] chore: update changelog for e5.10.1-rc.2 --- en_US/changes/breaking-changes-ee-5.10.md | 6 ++++++ en_US/changes/changes-ee-v5.md | 22 ++++++++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/en_US/changes/breaking-changes-ee-5.10.md b/en_US/changes/breaking-changes-ee-5.10.md index e2293b700..399615748 100644 --- a/en_US/changes/breaking-changes-ee-5.10.md +++ b/en_US/changes/breaking-changes-ee-5.10.md @@ -1,5 +1,11 @@ # Incompatible Changes in EMQX 5.10 +## e5.10.1 + +- [#15753](https://github.com/emqx/emqx/pull/15753) Listener connection rate limits (`max_conn_rate` and `max_conn_burst`) are now enforced per listener rather than per acceptor, restoring the pre-5.9.0 behavior. As a result, configurations from versions 5.9.0 and 5.9.1 are incompatible: specified rates must be scaled up by the number of acceptors configured for respective listeners. + +- [#15752](https://github.com/emqx/emqx/pull/15752) Listener connection rate limits (`max_conn_rate` and `max_conn_burst`) are now enforced per listener rather than per acceptor, restoring the pre-5.9.0 behavior. As a result, configurations from versions 5.9.0, 5.9.1 and 5.10.0 are incompatible: specified rates must be scaled up by the number of acceptors configured for respective listeners. + ## e5.10.0 - [#15289](https://github.com/emqx/emqx/pull/15289) Added a new `resource_opts.health_check_timeout` configuration to all Connectors, Actions and Sources, with default value of 60 seconds. If a health check takes more than this to return a response, the Connector/Action/Source will be deemed `disconnected`. diff --git a/en_US/changes/changes-ee-v5.md b/en_US/changes/changes-ee-v5.md index 4fd4a9692..93ef730d0 100644 --- a/en_US/changes/changes-ee-v5.md +++ b/en_US/changes/changes-ee-v5.md @@ -8,6 +8,10 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ ### Enhancements +- [#15911](https://github.com/emqx/emqx/pull/15911) Now, for the HTTP Action, the HTTP request timeout is taken to be the same as `resource_opts.request_ttl`. Previously, it was a fixed, non-configurable value of 30 seconds. + +- [#15371](https://github.com/emqx/emqx/pull/15371) Added `tags` fields to the return of `GET /actions_summary` and `GET /sources_summary`, and to the fallback actions returned in `GET /actions/:id`. + #### Observability - [#15499](https://github.com/emqx/emqx/pull/15499) Added a force deactivate alarm API endpoint to allow administrators to forcibly deactivate active alarms. @@ -40,6 +44,24 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ ### Bug Fixes +- [#15910](https://github.com/emqx/emqx/pull/15910) Fixed an issue with Connectors where a pool of workers could fail to recover from a failure if multiple workers crashed simultaneously in large worker pools. + + Connectors affected and fixed: + + - MySQL + - PostgreSQL + - Oracle + - SQLServer + - TDEngine + - Cassandra + - Dynamo + +- [#15906](https://github.com/emqx/emqx/pull/15906) Upgraded Kafka producer library Wolff from 4.0.12 to 4.0.13`, which adds handling for the record_list_too_large error in ProduceResponse. + +- [#15899](https://github.com/emqx/emqx/pull/15899) Improved memory usage: authorization (authz) cache is now cleared immediately when a client disconnects, reducing unnecessary memory consumption. + +- [#15518](https://github.com/emqx/emqx/pull/15518) Resolve a race condition that may lead to accumulating inconsistencies in the routing table and shared subscriptions state in the cluster when a large number of shared subscribers disconnect simultaneously. + #### API - [#15547](https://github.com/emqx/emqx/pull/15547) Resolved an issue where EMQX would fail to process HTTP requests with large bodies (e.g., 10MB) in the REST API. From 7ed4023edca55bba7fd70bf6f9215da2b1790755 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Wed, 17 Sep 2025 11:29:14 +0800 Subject: [PATCH 39/49] Add zh translation to "changes-ee-v5.md" --- en_US/changes/all-changes-ee.md | 1 + en_US/changes/breaking-changes-ee-5.10.md | 21 ---- en_US/changes/changes-ee-v5.md | 131 +++++++++++----------- zh_CN/changes/all-changes-ee.md | 1 + zh_CN/changes/breaking-changes-5.10.md | 4 + zh_CN/changes/changes-ee-v5.md | 110 ++++++++++++++++++ 6 files changed, 180 insertions(+), 88 deletions(-) delete mode 100644 en_US/changes/breaking-changes-ee-5.10.md diff --git a/en_US/changes/all-changes-ee.md b/en_US/changes/all-changes-ee.md index e2b074f25..ecc5e9b02 100644 --- a/en_US/changes/all-changes-ee.md +++ b/en_US/changes/all-changes-ee.md @@ -4,6 +4,7 @@ The release notes page for EMQX Enterprise provides a comprehensive and detailed ## v5.10 +- [5.10.1](./changes-ee-v5.md#_5-10-1): 2025-09-17 - [5.10.0](./changes-ee-v5.md#_5-10-0): 2025-06-10 ## v5.9 diff --git a/en_US/changes/breaking-changes-ee-5.10.md b/en_US/changes/breaking-changes-ee-5.10.md deleted file mode 100644 index 399615748..000000000 --- a/en_US/changes/breaking-changes-ee-5.10.md +++ /dev/null @@ -1,21 +0,0 @@ -# Incompatible Changes in EMQX 5.10 - -## e5.10.1 - -- [#15753](https://github.com/emqx/emqx/pull/15753) Listener connection rate limits (`max_conn_rate` and `max_conn_burst`) are now enforced per listener rather than per acceptor, restoring the pre-5.9.0 behavior. As a result, configurations from versions 5.9.0 and 5.9.1 are incompatible: specified rates must be scaled up by the number of acceptors configured for respective listeners. - -- [#15752](https://github.com/emqx/emqx/pull/15752) Listener connection rate limits (`max_conn_rate` and `max_conn_burst`) are now enforced per listener rather than per acceptor, restoring the pre-5.9.0 behavior. As a result, configurations from versions 5.9.0, 5.9.1 and 5.10.0 are incompatible: specified rates must be scaled up by the number of acceptors configured for respective listeners. - -## e5.10.0 - -- [#15289](https://github.com/emqx/emqx/pull/15289) Added a new `resource_opts.health_check_timeout` configuration to all Connectors, Actions and Sources, with default value of 60 seconds. If a health check takes more than this to return a response, the Connector/Action/Source will be deemed `disconnected`. - - Note: since the default is 60 seconds, this means that if a Connector/Action/Source previously could take more than that to return a healthy response, now it'll be deemed disconnected in such situations. - -- [#15286](https://github.com/emqx/emqx/pull/15286) Configuration option `broker.routing.storage_schema` is now deprecated and ignored. Legacy `v1` routing storage schema is no longer supported, and EMQX will refuse to start in a cluster running older versions that still use it. - -- [#15239](https://github.com/emqx/emqx/pull/15239) The type for the `multi_tenancy.default_max_sessions` is now either `infinity` or a positive integer. Previously, `0` would be accepted. - -- [#15156](https://github.com/emqx/emqx/pull/15156) Schema validation was added to `dashboard.sso.oidc.issuer` field. Now, this value is checked to be a valid URL. - - diff --git a/en_US/changes/changes-ee-v5.md b/en_US/changes/changes-ee-v5.md index 93ef730d0..fada9fb75 100644 --- a/en_US/changes/changes-ee-v5.md +++ b/en_US/changes/changes-ee-v5.md @@ -4,19 +4,10 @@ *Release Date: 2025-09-17* -Make sure to check the breaking changes and known issues before upgrading to EMQX 5.10.0. +Make sure to check the breaking changes and known issues before upgrading to EMQX 5.10.1. ### Enhancements -- [#15911](https://github.com/emqx/emqx/pull/15911) Now, for the HTTP Action, the HTTP request timeout is taken to be the same as `resource_opts.request_ttl`. Previously, it was a fixed, non-configurable value of 30 seconds. - -- [#15371](https://github.com/emqx/emqx/pull/15371) Added `tags` fields to the return of `GET /actions_summary` and `GET /sources_summary`, and to the fallback actions returned in `GET /actions/:id`. - -#### Observability - -- [#15499](https://github.com/emqx/emqx/pull/15499) Added a force deactivate alarm API endpoint to allow administrators to forcibly deactivate active alarms. -- [#15364](https://github.com/emqx/emqx/pull/15364) Added HTTP header configuration items to the OpenTelemetry integration to adapt to collectors with HTTP authentication. - #### Access Control - [#15294](https://github.com/emqx/emqx/pull/15294) Enhanced LDAP authentication and authorization. @@ -37,6 +28,13 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ - [#15542](https://github.com/emqx/emqx/pull/15542) Upgraded our `erlcloud` library to `3.8.3.0`. This allows one to setup a S3 Connector without specifying Access Key Id and Secret Access Key, so long as the EC2 instance EMQX is running in has the correct IAM permissions to read/write to the configured bucket(s). - [#15845](https://github.com/emqx/emqx/pull/15845) The `static_clientids` configuration for the MQTT Connector now supports specifying a username and password for each client ID. This is particularly useful for scenarios like connecting to Azure IoT Hub, where each device (client ID) requires a unique set of credentials. This enhancement helps ensure successful connections across multiple nodes in a clustered environment. +- [#15911](https://github.com/emqx/emqx/pull/15911) The HTTP request timeout for the HTTP Action is now configurable via the `resource_opts.request_ttl` setting. Previously, this timeout was fixed at 30 seconds and could not be adjusted. +- [#15371](https://github.com/emqx/emqx/pull/15371) Added `tags` fields to the responses of `GET /actions_summary` and `GET /sources_summary` endpoints, and to the fallback actions returned by the `GET /actions/:id` endpoint. + +#### Observability + +- [#15499](https://github.com/emqx/emqx/pull/15499) Added a force deactivate alarm API endpoint to allow administrators to forcibly deactivate active alarms. +- [#15364](https://github.com/emqx/emqx/pull/15364) Added HTTP header configuration items to the OpenTelemetry integration to adapt to collectors with HTTP authentication. #### CLI @@ -44,38 +42,7 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ ### Bug Fixes -- [#15910](https://github.com/emqx/emqx/pull/15910) Fixed an issue with Connectors where a pool of workers could fail to recover from a failure if multiple workers crashed simultaneously in large worker pools. - - Connectors affected and fixed: - - - MySQL - - PostgreSQL - - Oracle - - SQLServer - - TDEngine - - Cassandra - - Dynamo - -- [#15906](https://github.com/emqx/emqx/pull/15906) Upgraded Kafka producer library Wolff from 4.0.12 to 4.0.13`, which adds handling for the record_list_too_large error in ProduceResponse. - -- [#15899](https://github.com/emqx/emqx/pull/15899) Improved memory usage: authorization (authz) cache is now cleared immediately when a client disconnects, reducing unnecessary memory consumption. - -- [#15518](https://github.com/emqx/emqx/pull/15518) Resolve a race condition that may lead to accumulating inconsistencies in the routing table and shared subscriptions state in the cluster when a large number of shared subscribers disconnect simultaneously. - -#### API - -- [#15547](https://github.com/emqx/emqx/pull/15547) Resolved an issue where EMQX would fail to process HTTP requests with large bodies (e.g., 10MB) in the REST API. -- [#15797](https://github.com/emqx/emqx/pull/15797) To improve compatibility with EMQX 4.x, the `encoding` parameter has been reintroduced in the batch publish HTTP API (`/api/v5/publish/bulk`) as an alias for `payload_encoding`. This change addresses migration issues for users relying on the original `encoding` parameter, and ensures existing integrations using EMQX v4 APIs can continue working without requiring software-level changes. - -#### Observability - -- [#15785](https://github.com/emqx/emqx/pull/15785) Resolved a crash that occurred when MQTT usernames containing non-ASCII characters were used in formatting network congestion alarm messages. - -#### Gateway - -- [#15342](https://github.com/emqx/emqx/pull/15342) Fixed a crash in the NATS gateway caused by client info override templates referencing undefined packet fields. The system now returns an empty binary instead of undefined atom. - -#### Core MQTT Functions +#### Core MQTT Functionalities - [#15361](https://github.com/emqx/emqx/pull/15361) Fixed a `function_clause` error when parsing a malformed `User-Property` pair with invalid (too short) length. @@ -86,25 +53,25 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ - [#15872](https://github.com/emqx/emqx/pull/15872) Eliminated warning log `unclean_terminate` when disconnected after CONNACK is sent with a non-zero reason code. -#### Data Integration +- [#15518](https://github.com/emqx/emqx/pull/15518) Resolve a race condition that could cause accumulating inconsistencies in the routing table and shared subscriptions state across the cluster when a large number of shared subscribers disconnect simultaneously. +#### Access Control -- [#15394](https://github.com/emqx/emqx/pull/15394) Fixed a rare race condition where Action metrics could become inconsistent due to unexpected asynchronous replies. -- [#15603](https://github.com/emqx/emqx/pull/15603) Fixed an issue in the MQTT bridge where a stale connection could be shown as `Connected` and would not automatically reconnect. -- [#15826](https://github.com/emqx/emqx/pull/15826) Improved Kafka consumer connector health check behavior with restricted ACLs. Previously, Kafka Consumer Connector health checks could fail if the configured user lacked permission to access the internal `____emqx_consumer_probe` consumer group used for the check. With this fix, if the Kafka broker returns an "ACL denied" response, EMQX will treat the connection as healthy. -- [#15827](https://github.com/emqx/emqx/pull/15827) Fixed atom and process leaks in the GreptimeDB driver. - Fixed a `function_clause` error that could arise if certain incorrect write syntaxes were used in GreptimeDB Actions. +- [#15818](https://github.com/emqx/emqx/pull/15818) Corrected handling of `{allow|deny, all}` ACL rules. -- [#15836](https://github.com/emqx/emqx/pull/15836) Enriched the returned information when a Kafka Consumer Source fails to be added, for example, due to denied topic ACLs. + Previously, these rules were internally translated to match `#`, which incorrectly failed to match topics prefixed with `$` (e.g. `$testtopic/1`) due to MQTT spec restrictions. + Now, a special internal value is used to ensure `{allow|deny, all}` rules correctly match any topic, including `$`-prefixed ones. -- [#15850](https://github.com/emqx/emqx/pull/15850) Fixed an issue with the MQTT bridge when a stale connection was displayed as `Connected` and the connection was not re-established. +- [#15844](https://github.com/emqx/emqx/pull/15844) Added validation to forbid adding empty usernames to the built-in database authenticator. Such users cannot be deleted via the HTTP API later, since they mess up the API path. -- [#15866](https://github.com/emqx/emqx/pull/15866) Upgrade Kafka producer lib wollf to 4.0.12 to improve handling of temporarily missing partitions in Kafka metadata responses. + If you have such an user and wish to delete it, run the following in an EMQX console: - In rare race conditions, Kafka may return an incomplete partition list. - Previously, this was only handled when a topic was recreated with fewer partitions, but not when partitions were temporarily missing. - This gap could cause the partition producer to stall and block shutdown indefinitely. + ```erlang + mria:transaction(emqx_authn_shard, fun() -> mnesia:delete(emqx_authn_mnesia, {'mqtt:global',<<>>}, write) end). + ``` + +- [#15899](https://github.com/emqx/emqx/pull/15899) Improved memory management by ensuring that the authorization (authz) cache is cleared immediately when a client disconnects, reducing unnecessary memory consumption. #### Deployment @@ -123,31 +90,61 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ - [#15788](https://github.com/emqx/emqx/pull/15788) Fixed etcd cluster discovery issue. Resolved an issue where EMQX nodes from different clusters could mistakenly join each other when using a shared etcd server. This was caused by a bug in the etcd client library. -#### Rate Limit +#### Smart Data Hub -- [#15794](https://github.com/emqx/emqx/pull/15794) Improved the behavior of connection rate limit updates to ensure that changes (e.g., to burst rate or rate thresholds) are applied immediately after the listener configuration is updated. Previously, parts of the internal limiter state were not refreshed correctly, which could result in rate limits appearing stricter than configured. +- [#15810](https://github.com/emqx/emqx/pull/15810) Introduced `spb_{en,de}code` functions to correct handling of `bytes_value` Metrics. Fixed an issue with the original `sparkplug_{en,de}code` functions, which did not base64 encode/decode `bytes_value` metric values as required by the [Protobuf specification](https://protobuf.dev/programming-guides/json/). To address this, new `spb_{en,de}code` functions have been introduced for correct encoding/decoding of such fields. The old `sparkplug_{en,de}code` functions are now deprecated to maintain backward compatibility. -#### Smart Data Hub +#### Data Integration -- [#15810](https://github.com/emqx/emqx/pull/15810) Introduced `spb_{en,de}code` functions to correct handling of `bytes_value` Metrics. Fixed an issue with the original `sparkplug_{en,de}code` functions, which did not base64 encode/decode `bytes_value` metric values as required by the [Protobuf specification](https://protobuf.dev/programming-guides/json/). To address this, new `spb_{en,de}code` functions have been introduced for correct encoding/decoding of such fields. The old `sparkplug_{en,de}code` functions are now deprecated to maintain backward compatibility. +- [#15394](https://github.com/emqx/emqx/pull/15394) Fixed a rare race condition where Action metrics could become inconsistent due to unexpected asynchronous replies. -#### Access Control +- [#15603](https://github.com/emqx/emqx/pull/15603) Fixed an issue in the MQTT bridge where a stale connection could be shown as `Connected` and would not automatically reconnect. + +- [#15826](https://github.com/emqx/emqx/pull/15826) Improved Kafka consumer connector health check behavior with restricted ACLs. Previously, Kafka Consumer Connector health checks could fail if the configured user lacked permission to access the internal `____emqx_consumer_probe` consumer group used for the check. With this fix, if the Kafka broker returns an "ACL denied" response, EMQX will treat the connection as healthy. +- [#15827](https://github.com/emqx/emqx/pull/15827) Fixed atom and process leaks in the GreptimeDB driver. -- [#15818](https://github.com/emqx/emqx/pull/15818) Corrected handling of `{allow|deny, all}` ACL rules. + Fixed a `function_clause` error that could arise if certain incorrect write syntaxes were used in GreptimeDB Actions. - Previously, these rules were internally translated to match `#`, which incorrectly failed to match topics prefixed with `$` (e.g. `$testtopic/1`) due to MQTT spec restrictions. - Now, a special internal value is used to ensure `{allow|deny, all}` rules correctly match any topic, including `$`-prefixed ones. +- [#15836](https://github.com/emqx/emqx/pull/15836) Enriched the returned information when a Kafka Consumer Source fails to be added, for example, due to denied topic ACLs -- [#15844](https://github.com/emqx/emqx/pull/15844) Added validation to forbid adding empty usernames to the built-in database authenticator. Such users cannot be deleted via the HTTP API later, since they mess up the API path. +- [#15866](https://github.com/emqx/emqx/pull/15866) Upgrade Kafka producer lib wollf to 4.0.12 to improve handling of temporarily missing partitions in Kafka metadata responses. - If you have such an user and wish to delete it, run the following in an EMQX console: + In rare race conditions, Kafka may return an incomplete partition list. Previously, this was only handled when a topic was recreated with fewer partitions, but not when partitions were temporarily missing. This gap could cause the partition producer to stall and block shutdown indefinitely. - ```erlang - mria:transaction(emqx_authn_shard, fun() -> mnesia:delete(emqx_authn_mnesia, {'mqtt:global',<<>>}, write) end). - ``` +- [#15906](https://github.com/emqx/emqx/pull/15906) Upgraded Kafka producer library Wolff from 4.0.12 to 4.0.13, which adds handling for the `record_list_too_large` error in `ProduceResponse`. + +- [#15910](https://github.com/emqx/emqx/pull/15910) Fixed an issue with Connectors where a pool of workers could fail to recover from a failure if multiple workers crashed simultaneously in large worker pools. + + Connectors affected and fixed: + + - MySQL + - PostgreSQL + - Oracle + - SQLServer + - TDEngine + - Cassandra + - Dynamo + +#### API + +- [#15547](https://github.com/emqx/emqx/pull/15547) Resolved an issue where EMQX would fail to process HTTP requests with large bodies (e.g., 10MB) in the REST API. +- [#15797](https://github.com/emqx/emqx/pull/15797) To improve compatibility with EMQX 4.x, the `encoding` parameter has been reintroduced in the batch publish HTTP API (`/api/v5/publish/bulk`) as an alias for `payload_encoding`. This change addresses migration issues for users relying on the original `encoding` parameter, and ensures existing integrations using EMQX v4 APIs can continue working without requiring software-level changes. + +#### Rate Limit + + +- [#15794](https://github.com/emqx/emqx/pull/15794) Improved the behavior of connection rate limit updates to ensure that changes (e.g., to burst rate or rate thresholds) are applied immediately after the listener configuration is updated. Previously, parts of the internal limiter state were not refreshed correctly, which could result in rate limits appearing stricter than configured. + +#### Observability + +- [#15785](https://github.com/emqx/emqx/pull/15785) Resolved a crash that occurred when MQTT usernames containing non-ASCII characters were used in formatting network congestion alarm messages. + +#### Gateway + +- [#15342](https://github.com/emqx/emqx/pull/15342) Fixed a crash in the NATS gateway caused by client info override templates referencing undefined packet fields. The system now returns an empty binary instead of undefined atom. ## 5.10.0 diff --git a/zh_CN/changes/all-changes-ee.md b/zh_CN/changes/all-changes-ee.md index ca37b3927..f839e9b09 100644 --- a/zh_CN/changes/all-changes-ee.md +++ b/zh_CN/changes/all-changes-ee.md @@ -4,6 +4,7 @@ EMQX 企业版版本发布页面全面详细地记录了 EMQX 企业版每个版 ## v5.10 +- [5.10.1](./changes-ee-v5.md#_5-10-1): 2025-09-17 - [5.10.0](./changes-ee-v5.md#_5-10-0): 2025-06-10 ## v5.9 diff --git a/zh_CN/changes/breaking-changes-5.10.md b/zh_CN/changes/breaking-changes-5.10.md index 47ed0267c..eec48f5e7 100644 --- a/zh_CN/changes/breaking-changes-5.10.md +++ b/zh_CN/changes/breaking-changes-5.10.md @@ -1,5 +1,9 @@ # EMQX 5.10 中的不兼容变更 +## 5.10.1 + + + ## 5.10.0 - [#15289](https://github.com/emqx/emqx/pull/15289) 为所有连接器、动作和数据源新增配置项 `resource_opts.health_check_timeout`，默认值为 60 秒。 diff --git a/zh_CN/changes/changes-ee-v5.md b/zh_CN/changes/changes-ee-v5.md index e3f14458c..a56222063 100644 --- a/zh_CN/changes/changes-ee-v5.md +++ b/zh_CN/changes/changes-ee-v5.md @@ -1,5 +1,115 @@ # EMQX 企业版 v5 版本 +## 5.10.1 + +*发布日期：2025-09-17* + +升级前请查看已知问题列表和不兼容变更列表。 + +### 增强 + +#### 访问控制 + +- [#15294](https://github.com/emqx/emqx/pull/15294) 增强了 LDAP 认证和授权功能。LDAP 授权现在支持使用 JSON 格式的扩展 ACL 规则，除了现有的简单主题列表外，还可以在认证过程中基于客户端信息从 LDAP 获取 ACL 规则，并将其缓存在客户端的元数据中，以避免在授权过程中重复进行 LDAP 查询。 +- [#15349](https://github.com/emqx/emqx/pull/15349) 优化了认证和授权的外部资源管理。此前，EMQX 在禁用认证或授权源的情况下，仍可能与配置的资源保持连接。 + +#### 数据集成 + +- [#15360](https://github.com/emqx/emqx/pull/15360) Amazon S3 Tables 动作现在支持以 Parquet 格式写入数据文件。 +- [#15387](https://github.com/emqx/emqx/pull/15387) 为 Kinesis 生产者连接器和动作的健康检查增加了限速机制，以遵守 AWS API 限额并提升集群行为一致性： + - 对 `ListStreams` 和 `DescribeStream` 接口的调用分别限制为每个连接器每秒 5 次和 10 次； + - 集群中的核心节点协调分布式限速器，以确保限速一致。 + - 若健康检查被限速或超时，连接器或动作将保留原状态，而不是被标记为已断开。 + - 新增配置项 `resource_opts.health_check_interval_jitter`，在健康检查间隔基础上引入一个均匀随机延迟，减少同一连接器下多个动作同时发起健康检查的可能性。 +- [#15542](https://github.com/emqx/emqx/pull/15542) 将 `erlcloud` 库升级到 `3.8.3.0`。升级后，如果 EMQX 运行的 EC2 实例具有正确的 IAM 权限来读写配置的 S3 存储桶，就可以在不指定访问密钥 ID 和私有访问密钥的情况下配置 S3 连接器。 +- [#15845](https://github.com/emqx/emqx/pull/15845) MQTT 连接器的 `static_clientids` 配置项现支持为每个客户端 ID 分别指定用户名和密码，适用于如 Azure IoT Hub 等要求每个设备使用唯一凭证的场景。此增强提升了在集群部署中多节点连接的兼容性与稳定性。 +- [#15911](https://github.com/emqx/emqx/pull/15911) HTTP 动作的 HTTP 请求超时时间现在可以通过 `resource_opts.request_ttl` 设置进行配置。此前，此超时时间固定为 30 秒且不可调整。 +- [#15371](https://github.com/emqx/emqx/pull/15371) 为 `GET /actions_summary` 和 `GET /sources_summary` 接口的响应以及 `GET /actions/:id` 接口返回的备选动作添加了 `tags` 字段。 + +#### 可观测性 + +- [#15499](https://github.com/emqx/emqx/pull/15499) 添加了强制停用告警的 API 接口，允许管理员强制停用活动告警。 +- [#15364](https://github.com/emqx/emqx/pull/15364) 为 OpenTelemetry 集成添加了 HTTP 头配置项，以适应带有 HTTP 认证的 collector。 + +#### CLI + +- [#15399](https://github.com/emqx/emqx/pull/15399) `node_dump` 工具现在导出当前系统配置为 HOCON 格式，并自动对敏感信息（如密码和密钥）进行脱敏处理，以确保安全。 + +### 修复 + +#### 核心 MQTT 功能 + +- [#15361](https://github.com/emqx/emqx/pull/15361) 修复了在解析格式错误的 `User-Property` 键值对时产生的 `function_clause` 错误，特别是当键值对的长度无效（过短）时。 +- [#15396](https://github.com/emqx/emqx/pull/15396) 移除了已断开连接客户端的共享订阅的冗余清理操作。这些操作在高频断开情况下容易导致崩溃，并可能导致全局代理状态不一致。 +- [#15416](https://github.com/emqx/emqx/pull/15416) 修复了 WebSocket 连接会话过期时偶尔出现的 warning 级别日志和崩溃问题。该问题由近期的 WebSocket 性能优化引入，虽然不会影响 Broker 的容量，但会在日志中产生如下错误信息： + - `error: {function_clause,[{gen_tcp,send,[closed,[]],[{file,“gen_tcp.erl”},{line,966}]},{cowboy_websocket_linger,commands,3,[{file,“cowboy_websocket_linger.erl”},{line,665}]},...` + - `message: {tcp,#Port<0.364>,<<136,130,...>>}, msg: emqx_session_mem_unknown_message` +- [#15872](https://github.com/emqx/emqx/pull/15872) 消除了在 CONNACK 后因非零原因代码断开连接时的 warning 日志 `unclean_terminate`。 +- [#15518](https://github.com/emqx/emqx/pull/15518) 修复了一个竞争条件，该问题在大量共享订阅者同时断开连接时，可能导致集群中路由表和共享订阅状态持续出现不一致。 + +#### 部署 + +- [#15553](https://github.com/emqx/emqx/pull/15553) 修复了 EMQX Helm chart 的一个问题：在使用默认配置部署 EMQX 时，会启动多个副本，并导致除一个节点外其余节点全部崩溃。现在 Helm chart 默认改为单副本，因为集群部署需要商业 License。 +- [#15712](https://github.com/emqx/emqx/pull/15712) 修复了从旧版本（5.9 之前）进行滚动升级时，节点启动失败的问题。在 EMQX 的早期版本中（5.9 之前），ZIP 时间戳编码器中的错误可能会在归档条目中存储无效的 "秒" 值（值对应于 DOS 时间格式中的第 30 或 31 个 2 秒槽）。 +- [#15863](https://github.com/emqx/emqx/pull/15863) 修复了许可证配额报警文本。 + +#### 访问控制 + +- [#15818](https://github.com/emqx/emqx/pull/15818) 修正了 `{allow|deny, all}` ACL 规则的处理。以前，这些规则被内部转换为匹配 `#`，但由于 MQTT 规范的限制，未能正确匹配以 `$` 为前缀的主题（例如 `$testtopic/1`）。现在，使用了一个特殊的内部值，确保 `{allow|deny, all}` 规则能够正确匹配所有主题，包括以 `$` 为前缀的主题。 + +- [#15844](https://github.com/emqx/emqx/pull/15844) 添加了验证机制，禁止向内置数据库认证器添加空用户名。此类用户稍后无法通过 HTTP API 删除，因为它们会导致 API 路径混乱。 + 如果您有此类用户并希望删除，请在 EMQX 控制台中运行以下命令： + + ``` + mria:transaction(emqx_authn_shard, fun() -> mnesia:delete(emqx_authn_mnesia, {'mqtt:global',<<>>}, write) end). + ``` + +- [#15899](https://github.com/emqx/emqx/pull/15899) 通过确保在客户端断开时立即清除授权（authz）缓存来改进内存管理，减少不必要的内存消耗。 + +#### 集群 + +- [#15788](https://github.com/emqx/emqx/pull/15788) 修复了 etcd 集群发现问题。解决了使用共享 etcd 服务器时，EMQX 节点可能错误地加入到不同集群中的问题。此问题是由 etcd 客户端库中的 bug 引起的。 + +#### 智能数据中心 + +- [#15810](https://github.com/emqx/emqx/pull/15810) 引入了 `spb_{en,de}code` 函数来修正 `bytes_value` 指标的处理。修复了原始的 `sparkplug_{en,de}code` 函数的问题，因为它们没有根据 [Protobuf 规范](https://protobuf.dev/programming-guides/json/) 对 `bytes_value` 指标值进行 base64 编码/解码。为此，引入了新的 `spb_{en,de}code` 函数来正确编码/解码这些字段。旧的 `sparkplug_{en,de}code` 函数已被弃用，以保持向后兼容性。 + +#### 数据集成 + +- [#15394](https://github.com/emqx/emqx/pull/15394) 修复了一个罕见的竞争条件，导致动作指标因意外的异步回复而变得不一致。 +- [#15603](https://github.com/emqx/emqx/pull/15603) 修复了 MQTT 桥接中的一个问题：过期的连接可能仍显示为 `Connected` 状态，且不会自动重连。 +- [#15826](https://github.com/emqx/emqx/pull/15826) 改进了 Kafka 消费者连接器健康检查行为，尤其是在 ACL 限制的情况下。此前，若配置的用户缺少访问内部 `____emqx_consumer_probe` 消费者组的权限，则 Kafka 消费者连接器的健康检查可能会失败。通过此修复，如果 Kafka broker 返回 "ACL denied" 响应，EMQX 将视该连接为健康连接。 +- [#15827](https://github.com/emqx/emqx/pull/15827) 修复了 GreptimeDB 驱动中的原子泄漏和进程泄漏问题。同时修复了在 GreptimeDB 动作中使用某些错误的写入语法时可能出现的 `function_clause` 错误。 +- [#15836](https://github.com/emqx/emqx/pull/15836) 丰富了 Kafka 消费者源添加失败时的返回信息，例如因主题 ACL 被拒导致的失败。 +- [#15866](https://github.com/emqx/emqx/pull/15866) 升级 Kafka 生产者库 `wolff` 到 4.0.12，以改善 Kafka 元数据响应中临时缺失分区的处理。 + 在罕见的竞争条件下，Kafka 可能返回不完整的分区列表。此前，这种情况仅在主题被重新创建且分区较少时得到处理，而在分区临时缺失时未被处理。此修复解决了此问题，防止分区生产者挂起并无限期阻止关闭。 +- [#15906](https://github.com/emqx/emqx/pull/15906) 将 Kafka 生产者库 Wolff 从 4.0.12 升级到 4.0.13，新增了处理 `ProduceResponse` 中 `record_list_too_large` 错误的功能。 +- [#15910](https://github.com/emqx/emqx/pull/15910) 修复了连接器在多个工作线程同时崩溃时，无法从失败中恢复的问题。受影响并已修复的连接器包括： + - MySQL + - PostgreSQL + - Oracle + - SQLServer + - TDEngine + - Cassandra + - Dynamo + +#### API + +- [#15547](https://github.com/emqx/emqx/pull/15547) 修复了 EMQX 在处理包含大体积请求体（例如 10MB）的 REST API 请求时可能失败的问题。 +- [#15797](https://github.com/emqx/emqx/pull/15797) 为了提高与 EMQX 4.x 的兼容性，`batch publish` HTTP API (`/api/v5/publish/bulk`) 中的 `encoding` 参数已重新引入，并作为 `payload_encoding` 的别名。此更改解决了依赖于原始 `encoding` 参数的用户的迁移问题，确保现有的 EMQX v4 API 集成可以继续工作，无需软件级别的更改。 + +#### 速率限制 + +- [#15794](https://github.com/emqx/emqx/pull/15794) 改进了连接速率限制更新的行为，确保在监听器配置更新后，速率限制的更改（例如突发速率或速率阈值）会立即生效。此前，内部限速器状态未能正确刷新，可能导致速率限制比配置的严格。 + +#### 可观测性 + +- [#15785](https://github.com/emqx/emqx/pull/15785) 修复了在格式化网络拥塞告警消息时，若 MQTT 用户名包含非 ASCII 字符，可能导致崩溃的问题。 + +#### 网关 + +- [#15342](https://github.com/emqx/emqx/pull/15342) 修复了 NATS 网关中的崩溃问题，该问题由客户端信息覆盖模板引用了未定义的报文字段引起。系统现在会返回空二进制而非未定义的原子值。 + ## 5.10.0 *发布日期：2025-06-09* From ae610244a6887d44cd779d59260cb2e8aa40ecb2 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Wed, 17 Sep 2025 11:55:22 +0800 Subject: [PATCH 40/49] Update "breaking-changes-5.10.md" --- en_US/changes/breaking-changes-5.10.md | 2 -- zh_CN/changes/breaking-changes-5.10.md | 2 +- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/en_US/changes/breaking-changes-5.10.md b/en_US/changes/breaking-changes-5.10.md index fd80c7434..03da561b9 100644 --- a/en_US/changes/breaking-changes-5.10.md +++ b/en_US/changes/breaking-changes-5.10.md @@ -4,8 +4,6 @@ - [#15752](https://github.com/emqx/emqx/pull/15752) Listener connection rate limits (`max_conn_rate` and `max_conn_burst`) are now enforced per listener rather than per acceptor, restoring the pre-5.9.0 behavior. As a result, configurations from versions 5.9.0, 5.9.1, and 5.10.0 are incompatible: specified rates must be scaled up by the number of acceptors configured for respective listeners. -- [#15753](https://github.com/emqx/emqx/pull/15753) Listener connection rate limits (`max_conn_rate` and `max_conn_burst`) are now enforced per listener rather than per acceptor, restoring the pre-5.9.0 behavior. As a result, configurations from versions 5.9.0 and 5.9.1 are incompatible: specified rates must be scaled up by the number of acceptors configured for respective listeners. - ## 5.10.0 - [#15289](https://github.com/emqx/emqx/pull/15289) Added a new `resource_opts.health_check_timeout` configuration to all Connectors, Actions, and Sources, with a default value of 60 seconds. If a health check takes more than this to return a response, the Connector/Action/Source will be deemed `disconnected`. diff --git a/zh_CN/changes/breaking-changes-5.10.md b/zh_CN/changes/breaking-changes-5.10.md index eec48f5e7..8d3ddf63b 100644 --- a/zh_CN/changes/breaking-changes-5.10.md +++ b/zh_CN/changes/breaking-changes-5.10.md @@ -2,7 +2,7 @@ ## 5.10.1 - +- [#15752](https://github.com/emqx/emqx/pull/15752) 监听器的连接速率限制（`max_conn_rate` 和 `max_conn_burst`）现在按监听器维度生效，而非按接收器（acceptor）维度生效，恢复了 5.9.0 之前的行为。因此，来自 5.9.0、5.9.1 和 5.10.0 的配置不兼容：需要将配置的速率乘以对应监听器配置的接收器数量。 ## 5.10.0 From 3274cb9c1ae2af97371a3052d2b401cd5c1fe0c4 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Wed, 17 Sep 2025 17:59:22 +0800 Subject: [PATCH 41/49] Update dir.yaml and add en translations --- dir.yaml | 46 +- en_US/emqx-ai/multimedia-ai/architecture.md | 63 ++- .../emqx-ai/multimedia-ai/message-protocol.md | 58 ++- en_US/emqx-ai/multimedia-ai/overview.md | 33 ++ en_US/emqx-ai/sdks/mcp-sdk-erlang.md | 4 +- en_US/emqx-ai/sdks/mcp-sdk-esp32-c.md | 59 +++ en_US/emqx-ai/sdks/multimedia-ai/overview.md | 11 + .../sdks/multimedia-ai/webrtc-typescript.md | 465 +++++++++++++++++- en_US/emqx-ai/sdks/overview.md | 3 +- zh_CN/emqx-ai/mcp-over-mqtt/architecture.md | 26 +- zh_CN/emqx-ai/mcp-over-mqtt/overview.md | 21 +- zh_CN/emqx-ai/mcp-over-mqtt/specification.md | 35 +- zh_CN/emqx-ai/multimedia-ai/architecture.md | 13 +- .../emqx-ai/multimedia-ai/message-protocol.md | 2 +- zh_CN/emqx-ai/multimedia-ai/overview.md | 2 +- zh_CN/emqx-ai/overview.md | 5 +- zh_CN/emqx-ai/sdks/multimedia-ai/overview.md | 4 +- .../sdks/multimedia-ai/webrtc-typescript.md | 8 +- 18 files changed, 775 insertions(+), 83 deletions(-) diff --git a/dir.yaml b/dir.yaml index 364e611c4..bafbf7abc 100644 --- a/dir.yaml +++ b/dir.yaml @@ -965,7 +965,51 @@ path: gateway/gbt32960 #- gateway/tcp - client-attributes/client-attributes - +- title_en: EMQX AI + title_cn: EMQX AI + title_ja: EMQX AI + path: emqx-ai/overview + collapsed: true + children: + - title_en: MCP over MQTT + title_cn: MCP over MQTT + title_ja: MCP over MQTT + path: emqx-ai/mcp-over-mqtt/overview + collapsed: true + children: + - emqx-ai/mcp-over-mqtt/architecture + - emqx-ai/mcp-over-mqtt/specification + - title_en: Multimedia Server + title_cn: 多媒体服务器 + title_ja: Multimedia Server + path: emqx-ai/multimedia-ai/overview + collapsed: true + children: + - emqx-ai/multimedia-ai/architecture + - emqx-ai/multimedia-ai/message-protocol + - title_en: MCP over MQTT & Multimedia Service SDKs + title_cn: MCP over MQTT 和多媒体服务 SDK + title_ja: MCP over MQTT & Multimedia Service SDKs + path: emqx-ai/sdks/overview + collapsed: true + children: + - title_en: Clients Compatible with Multimedia Services + title_cn: 与多媒体服务适配的客户端 + title_ja: Clients Compatible with Multimedia Services + path: emqx-ai/sdks/multimedia-ai/overview + collapsed: true + children: + - emqx-ai/sdks/multimedia-ai/webrtc-typescript + - title_en: MCP over MQTT SDKs + title_cn: MCP over MQTT SDKs + title_ja: MCP over MQTT SDKs + collapsed: true + children: + - emqx-ai/sdks/mcp-sdk-erlang + - emqx-ai/sdks/mcp-sdk-esp32-c + - emqx-ai/sdks/mcp-sdk-paho-c + - emqx-ai/sdks/mcp-sdk-python + - emqx-ai/sdks/mcp-sdk-typescript - title_en: Tutorials title_cn: 实用教程 title_ja: 実用的なチュートリアル diff --git a/en_US/emqx-ai/multimedia-ai/architecture.md b/en_US/emqx-ai/multimedia-ai/architecture.md index c79bec1ac..f41d616e8 100644 --- a/en_US/emqx-ai/multimedia-ai/architecture.md +++ b/en_US/emqx-ai/multimedia-ai/architecture.md @@ -1 +1,62 @@ -# Architecture +# Multimedia Server Architecture + +The EMQX Multimedia Server uses WebRTC to establish peer-to-peer audio and video data transmission with clients, while MQTT is used to carry general messages and WebRTC signaling. + +Developers can build AI agent programs in Python, which communicate with the Multimedia Server via Standard Input/Output (STDIO), enabling more sophisticated business logic and AI-driven workflows. + +The following diagram illustrates the overall architecture of an AI system built with the Multimedia Server: + +```mermaid +flowchart LR + Device <-- WebRTC --> MediaServer + MediaServer <-- STDIO --> AIAgent + MediaServer <--> ASRTTS + AIAgent <--> LLM + + Device[Device] + MediaServer[Multimedia Server] + AIAgent[AI Agent] + ASRTTS[ASR/TTS] + LLM[LLM] +``` + +### Components + +- **Device**: Exchanges audio and video data with the Multimedia Server via WebRTC. +- **Multimedia Server**: Processes audio and video streams from devices, provides Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) services, and communicates with the AI Agent. +- **AI Agent**: Receives ASR results from the Multimedia Server over STDIO. It encapsulates the core business logic of the AI application, calls the Large Language Model (LLM) for natural language understanding, and uses Multimedia Server APIs to send text or audio streams back to devices. +- **ASR/TTS**: External AI service providers that offer speech recognition and text-to-speech capabilities. +- **LLM**: Large Language Model, responsible for natural language processing and text generation. + +## Workflow + +The diagram below shows how the components interact in a typical workflow: + +```mermaid +sequenceDiagram + actor Device + participant Media as Multimedia Server + participant Agent as AI Agent + participant LLM + + Device ->>+ Media: WebRTC Audio + Media ->> Media: Run ASR + Media ->> Agent: ASR Result + Agent ->> LLM: Process Input with MCP Tools + LLM ->> Agent: LLM Output + Agent ->> Agent: Apply Business Logic + Agent ->> Media: TTS Request + Media ->> Media: Run TTS + Media ->> Device: WebRTC Audio + Device ->> Media: WebRTC Video + Agent ->> Media: Image Analysis Request + Media ->> Media: Run Image Analysis + Media ->> Agent: Image Analysis Result + Agent ->> LLM: Summarize Analysis Result + LLM ->> Agent: Summary + Agent ->> Media: TTS Request + Media ->> Device: WebRTC Audio + Agent ->> Agent: Additional Processing + Agent ->> Media: Send Control Message + Media ->> Device: MQTT Message +``` diff --git a/en_US/emqx-ai/multimedia-ai/message-protocol.md b/en_US/emqx-ai/multimedia-ai/message-protocol.md index c6e02bf66..457d6e407 100644 --- a/en_US/emqx-ai/multimedia-ai/message-protocol.md +++ b/en_US/emqx-ai/multimedia-ai/message-protocol.md @@ -1,4 +1,4 @@ -## Message Protocol +# Multimedia AI Messaging Protocol This document describes the message protocol used for interaction between the multimedia server, clients (devices), and AI agents. @@ -83,19 +83,17 @@ The multimedia proxy will send the `webrtc_terminated` message to the client whe } ``` -See [signaling_mqtt.js](https://github.com/emqx/emqx-multimedia-proxy/blob/main/apps/emqx_media_proxy_web/assets/js/signaling_mqtt.js) for a complete example of the client-side signaling implementation using MQTT. You can go to http://localhost:4000/webrtc_mqtt to try the demo. +## Send General Messages via MQTT -## Normal Messages via MQTT +The multimedia server and devices exchange general messages through the following MQTT topics: -The multimedia proxy can send normal messages to the device via the following MQTT topic: +- **`$message/`**: Topic for the multimedia server to send general messages to a device. +- **`$message//multimedia_proxy`**: Topic for a device to send arbitrary messages to the multimedia server. These messages are forwarded to the AI Agent via the `message_from_device` method. -- `$message/`: The MQTT topic for the device to receive normal messages from the multimedia proxy. -- `$message//multimedia_proxy`: The MQTT topic for the multimedia proxy to receive arbitrary messages from the device, will be sent to the AI agents using the `message_from_device` method. +### Messages Sent from Multimedia Server to Devices -### The format of the normal messages sent to the device: - -The multimedia proxy can send the following types of messages to the device via the `$message/` topic. +The multimedia server can publish the following message types on the `$message/` topic: A `asr_response` message is sent when ASR results are available: @@ -153,9 +151,10 @@ A `message` message is sent to device when the agent sends an arbitrary message } ``` -### The format of the arbitrary messages sent from the device to the multimedia proxy: +### Messages Sent from Devices to the Multimedia Server -The device can send arbitrary messages to the multimedia proxy via the `$message//multimedia_proxy` topic. The format of the messages is: +Devices can publish arbitrary messages to the **`$message//multimedia_proxy`** topic. + The multimedia server will forward these messages to the AI Agent: ```json { @@ -165,14 +164,15 @@ The device can send arbitrary messages to the multimedia proxy via the `$message ``` -## The Interaction Protocol between Multimedia Proxy and AI Agents +## Interaction Protocol between Multimedia Server and AI Agent -Using AI agents can enhance the capabilities of the multimedia proxy, such as processing ASR results according to specific business logic or sending messages in arbitrary formats to devices. +Using AI agents can can extend the capabilities of the Multimedia Server, for example by processing ASR results according to business logic or sending custom messages to devices. -The multimedia proxy interacts with AI agents using a simple JSON RPC 2.0 based protocol. The messages are sent over STDIO (standard input/output). Messages are delimited by newlines (`\n`), and MUST NOT contain embedded newlines. +The multimedia proxy interacts with AI agents using a simple JSON RPC 2.0 based protocol. The messages are sent over Standard Input/Output (STDIO). Messages are delimited by newlines (`\n`), and MUST NOT contain embedded newlines. - **Initialization**: - After the STDIO connection is established, the agent must send an initialization message to the multimedia proxy, to negotiate the protocol version and configs: + After the STDIO connection is established, the agent must send an initialization message to the multimedia proxy, to negotiate the protocol version and configuration: + ```json { "jsonrpc": "2.0", @@ -189,7 +189,7 @@ The multimedia proxy interacts with AI agents using a simple JSON RPC 2.0 based } } ``` - + The multimedia proxy will respond with an acknowledgment: ```json { @@ -198,7 +198,7 @@ The multimedia proxy interacts with AI agents using a simple JSON RPC 2.0 based "result": "ok" } ``` - + - **ASR Result**: The multimedia proxy sends the ASR results as notifications to the AI agents in the following format: ```json @@ -215,9 +215,12 @@ The multimedia proxy interacts with AI agents using a simple JSON RPC 2.0 based - **TTS and Send**: - The AI agents can request the multimedia proxy to perform TTS and send the audio back to the specific device. + The AI Agent can request the Multimedia Server to perform **TTS** and send the audio to a target device. - First the agent should send a `tts_and_send_start` message to start a TTS task, and then send one or more `tts_and_send` messages to send the texts to be converted to speech. The texts of the same task can be sent in one batch or in separate messages, but they must have the same `task_id`. Finally, the agent should send a `tts_and_send_finish` message to indicate the end of the TTS task. + 1. The agent sends a `tts_and_send_start` message to initiate the task. + 2. The agent sends one or more `tts_and_send` messages to provide the text to be synthesized. + - Multiple texts for the same task can be sent in batches or sequentially, but must use the same `task_id`. + 3. Finally, the agent sends a `tts_and_send_finish` message to signal the end of the task. The start message: ```json @@ -276,9 +279,10 @@ The multimedia proxy interacts with AI agents using a simple JSON RPC 2.0 based ] ``` - The `tts_and_send_start` and `tts_and_send_finish` messages canbe sent in the same batch with the `tts_and_send` messages, or in separate messages. + The `tts_and_send_start` and `tts_and_send_finish` messages can be sent either in the same batch as `tts_and_send` messages or separately. + + The Multimedia Server confirms each message with `"ok"` or returns an error: - The multimedia proxy will acknowledge the request with "ok" or errors: ```json [ { @@ -330,7 +334,8 @@ The multimedia proxy interacts with AI agents using a simple JSON RPC 2.0 based ``` - **Forward Messages Received from Device**: - The multimedia proxy will send messages received from `$message//multimedia_proxy` topic to the AI agents via the `message_from_device` method: + The Multimedia Server forwards messages received on the `$message//multimedia_proxy` topic to the AI Agent using the `message_from_device` method: + ```json { "jsonrpc": "2.0", @@ -342,9 +347,10 @@ The multimedia proxy interacts with AI agents using a simple JSON RPC 2.0 based } } ``` - + - **Send Message to Device**: - The AI agents can send arbitrary messages to the device via the multimedia proxy: + The AI Agent can send arbitrary messages to devices through the Multimedia Server: + ```json { "jsonrpc": "2.0", @@ -361,8 +367,8 @@ The multimedia proxy interacts with AI agents using a simple JSON RPC 2.0 based } } ``` - - The multimedia proxy will acknowledge the request with: + + The Multimedia Server responds with a confirmation: ```json { "jsonrpc": "2.0", diff --git a/en_US/emqx-ai/multimedia-ai/overview.md b/en_US/emqx-ai/multimedia-ai/overview.md index e69de29bb..d543bea2d 100644 --- a/en_US/emqx-ai/multimedia-ai/overview.md +++ b/en_US/emqx-ai/multimedia-ai/overview.md @@ -0,0 +1,33 @@ +# EMQX Multimedia Server + +The EMQX Multimedia Server is a high-performance audio and video processing platform built on WebRTC technology. It can receive RTP/SRTP audio and video streams from clients and integrates multiple AI capabilities, including Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and Image Understanding. By leveraging large language models (LLMs), the EMQX Multimedia Server enables advanced voice interactions and tool invocation, providing robust technical support for AI applications that require audio and video capabilities. + +## Key Features + +- **Real-time Audio and Video Processing** + Supports high-quality audio and video streaming with low latency and high reliability. +- **Automatic Speech Recognition (ASR)** + Converts speech to text with high accuracy, suitable for voice assistants, intelligent customer service, and more. +- **Text-to-Speech (TTS)** + Generates natural-sounding speech in multiple languages and voice styles, enhancing interactive experiences. +- **Image Understanding** + Integrates image recognition and analysis to support diverse visual processing tasks such as object detection and scene analysis. +- **LLM Integration** + Harnesses large model capabilities to enable complex voice conversations and tool invocation, meeting diverse business needs. +- **Flexible Architecture** + Designed for scalability and customization, supporting horizontal expansion. Developers can flexibly integrate services from different providers, including TTS, ASR, image processing, and LLMs. +- **High Reliability** + Built on a distributed architecture to ensure high availability and system stability under large-scale workloads. +- **Low Latency** + Optimized network transmission and model processing pipelines deliver smooth real-time interactions. + +## Use Cases + +The EMQX Multimedia Server can be applied to a wide range of AI-driven scenarios, including: + +- **Emotional Companionship** + Provides personalized companion services through conversational AI and emotion recognition technologies. +- **Intelligent Customer Service** + Enables efficient voice-based interactions with ASR and TTS, improving customer service quality. +- **Smart Device Control** + Facilitates intuitive device control through voice commands and image recognition. \ No newline at end of file diff --git a/en_US/emqx-ai/sdks/mcp-sdk-erlang.md b/en_US/emqx-ai/sdks/mcp-sdk-erlang.md index 5109cd5a4..06048b30c 100644 --- a/en_US/emqx-ai/sdks/mcp-sdk-erlang.md +++ b/en_US/emqx-ai/sdks/mcp-sdk-erlang.md @@ -4,7 +4,7 @@ This document demonstrates how to use the [MCP over MQTT Erlang SDK](https://git ## Example -### Creating a Simple MCP Client +### Create a Simple MCP Client ```erlang -module(mcp_mqtt_erl_client_demo). @@ -50,7 +50,7 @@ start_link() -> Here, `server_name_filter` is used to subscribe to the MQTT topic filter for MCP servers, and `mqtt_options` are options passed to the underlying MQTT client. -### Creating a Simple MCP Server +### Create a Simple MCP Server Below is a simple MCP server implementation that supports two tools: `tool1` and `tool2`. diff --git a/en_US/emqx-ai/sdks/mcp-sdk-esp32-c.md b/en_US/emqx-ai/sdks/mcp-sdk-esp32-c.md index e69de29bb..b16694429 100644 --- a/en_US/emqx-ai/sdks/mcp-sdk-esp32-c.md +++ b/en_US/emqx-ai/sdks/mcp-sdk-esp32-c.md @@ -0,0 +1,59 @@ +# ESP32 C SDK + +This guide demonstrates how to use the [MCP over MQTT C SDK for ESP32](https://github.com/mqtt-ai/esp-mcp-over-mqtt) to create a simple MCP over MQTT server. + Currently, only the MCP server is supported. You can create an MCP client using the Python SDK for interaction. + +The SDK uses the MQTT library included in ESP-IDF, making it suitable for ESP32 devices. Therefore, it must be used within the ESP-IDF environment. + +## Create an MCP Server + +Following the instructions in the [ESP32 C SDK README](https://github.com/mqtt-ai/esp-mcp-over-mqtt), create a new file named `mcp_server_example.c` in your ESP-IDF project and add the following code: + +```c +#include "mcp_server.h" + +const char* get_temperature_callback(int n_args, property_t *args) { + // Read sensor data + float temp = read_temperature_sensor(); + + // Return JSON formatted result + static char result[64]; + snprintf(result, sizeof(result), "{\"temperature\": %.2f}", temp); + return result; +} + +// Define MCP tools +mcp_tool_t my_tools[] = { + { + .name = "get_temperature", + .description = "Get device temperature", + .property_count = 0, + .properties = NULL, + .call = get_temperature_callback + } +}; + +// Initialize MCP server +mcp_server_t *server = mcp_server_init( + "esp32_sensor", // Server name + "ESP32 Sensor MCP Server", // Description + "mqtt://broker.example.com",// MQTT Broker URI + "esp32_client_001", // Client ID + "username", // Username + "password", // Password + NULL // Certificate (optional) +); + +// Register tools +mcp_server_register_tool(server, 1, my_tools); + +// Start server +mcp_server_run(server); +``` + +## Use MCP Server in an ESP-IDF Project + +For detailed usage, see the [ESP32 MCP Demo](https://github.com/mqtt-ai/esp32-mcp-mqtt-tutorial/tree/main/samples/blog_3) project. + This example demonstrates how to integrate the MCP over MQTT C SDK for ESP32 into an ESP-IDF project, set up an MCP server, and interact with it using an MCP client implemented in the Python SDK. + +After building the project with ESP-IDF and flashing it onto the ESP32 device, the MCP server will start automatically. \ No newline at end of file diff --git a/en_US/emqx-ai/sdks/multimedia-ai/overview.md b/en_US/emqx-ai/sdks/multimedia-ai/overview.md index e69de29bb..9b9adde02 100644 --- a/en_US/emqx-ai/sdks/multimedia-ai/overview.md +++ b/en_US/emqx-ai/sdks/multimedia-ai/overview.md @@ -0,0 +1,11 @@ +# Clients Compatible with Multimedia Services + +The Multimedia AI Server uses WebRTC for audio and video streaming, and MQTT for general messages and WebRTC signaling. In practice, clients do not need a dedicated SDK. Any client that supports WebRTC and MQTT protocols can interact with the Multimedia AI Service. Common client types include: + +- **Web Browsers**: Modern browsers such as Chrome, Firefox, Edge, and Safari support WebRTC natively and can directly access the Multimedia AI Service. +- **Mobile Applications**: Mobile apps can integrate a WebRTC SDK (e.g., [Pion](https://pion.ly)) to communicate with the Multimedia AI Service. +- **Embedded Devices**: IoT devices can integrate WebRTC libraries adapted for their platforms, such as [ESP32 WebRTC](https://github.com/espressif/esp-webrtc-solution), to connect to the Multimedia AI Service. + +For details on WebRTC signaling and MQTT message formats, see the [Multimedia AI Messaging Protocol](../../multimedia-ai/message-protocol.md). + +We also provide a browser-based client code example demonstrating how to interact with the Multimedia Service: [TypeScript WebRTC Example](./webrtc-typescript.md). \ No newline at end of file diff --git a/en_US/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md b/en_US/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md index 53f4bbc98..00ed5cd08 100644 --- a/en_US/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md +++ b/en_US/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md @@ -1 +1,464 @@ -# Typescript WebRTC Example +# TypeScript WebRTC Example + +This document demonstrates how to use TypeScript to interact with the EMQX Multimedia Service via MQTT signaling for WebRTC, enabling audio/video calls, speech recognition (ASR), and text-to-speech (TTS). + +The EMQX Multimedia Service uses the MQTT protocol to exchange WebRTC signaling. Clients can easily integrate real-time audio/video communication and leverage AI-powered services such as ASR and TTS. + +## Core Architecture + +```shell +TypeScript Client ←→ MQTT Signaling ←→ EMQX Multimedia Service + ↓ + WebRTC P2P Connection +``` + +**Key MQTT Topics**: + +- `$webrtc/{device_id}` — receive SDP answers and ICE candidates from the server. +- `$webrtc/{device_id}/multimedia_proxy` — send SDP offers and ICE candidates. +- `$message/{device_id}` — receive ASR, TTS, and general messages. + +## Core Implementation + +### WebRTC Signaling Class + +`MqttWebRTCSignaling` is the core signaling management class. It handles MQTT message exchange, WebRTC connection setup, and multimedia service integration. The class abstracts complex signaling logic into a simple API for developers. + +```typescript +import type { MqttClient } from 'mqtt' + +interface SignalingMessage { + type: 'sdp_offer' | 'sdp_answer' | 'ice_candidate' | 'webrtc_terminated' + | 'asr_response' | 'tts_begin' | 'tts_text' | 'tts_complete' | 'tts_terminate' + | 'chat' | 'message' + data?: any + payload?: any + reason?: string + results?: string + text?: string + task_id?: string +} + +interface WebRTCCallbacks { + onLocalStream?: (stream: MediaStream) => void + onRemoteStream?: (stream: MediaStream) => void + onConnectionStateChange?: (state: string) => void + onASRResponse?: (text: string) => void + onTTSText?: (text: string) => void + onMessage?: (message: string) => void + onError?: (error: Error) => void +} + +export class MqttWebRTCSignaling { + private mqttClient: MqttClient + private pc: RTCPeerConnection | null = null + private localStream: MediaStream | null = null + private remoteStream: MediaStream | null = null + private clientId: string + private callbacks: WebRTCCallbacks + private messageHandler: ((topic: string, message: Buffer) => void) | null = null + private ttsText: string = '' + + constructor(options: { + mqttClient: MqttClient + clientId: string + callbacks?: WebRTCCallbacks + }) { + this.mqttClient = options.mqttClient + this.clientId = options.clientId + this.callbacks = options.callbacks || {} + } + + async connect(): Promise { + if (!this.mqttClient.connected) { + throw new Error('MQTT client is not connected') + } + + // Subscribe to topics + await this.subscribeToTopics() + + // Set up message handler + this.setupMessageHandler() + + // Initialize WebRTC + await this.setupWebRTC() + } + + private async subscribeToTopics(): Promise { + const topics = [ + `$webrtc/${this.clientId}`, + `$message/${this.clientId}` + ] + + return new Promise((resolve, reject) => { + this.mqttClient.subscribe(topics, { qos: 0 }, (err) => { + if (err) reject(err) + else resolve() + }) + }) + } + + private setupMessageHandler(): void { + this.messageHandler = (topic: string, message: Buffer) => { + if (topic === `$webrtc/${this.clientId}` || topic === `$message/${this.clientId}`) { + try { + const payload = JSON.parse(message.toString()) as SignalingMessage + this.handleSignalingMessage(topic, payload) + } catch (error) { + console.error('Failed to parse message:', error) + } + } + } + + this.mqttClient.on('message', this.messageHandler) + } + + private async handleSignalingMessage(topic: string, message: SignalingMessage): Promise { + if (topic === `$webrtc/${this.clientId}`) { + // WebRTC signaling handling + switch (message.type) { + case 'sdp_answer': + if (this.pc && message.data) { + const answer = new RTCSessionDescription({ + type: 'answer', + sdp: message.data.sdp || message.data + }) + await this.pc.setRemoteDescription(answer) + } + break + + case 'ice_candidate': + if (this.pc && message.data) { + const candidate = new RTCIceCandidate(message.data) + await this.pc.addIceCandidate(candidate) + } + break + + case 'webrtc_terminated': + this.callbacks.onError?.(new Error(`WebRTC terminated: ${message.reason}`)) + this.disconnect() + break + } + } else if (topic === `$message/${this.clientId}`) { + // Multimedia service message handling + switch (message.type) { + case 'asr_response': + this.callbacks.onASRResponse?.(message.results || '') + break + + case 'tts_begin': + this.ttsText = '' + break + + case 'tts_text': + this.ttsText += message.text || '' + this.callbacks.onTTSText?.(this.ttsText) + break + + case 'tts_complete': + console.log('TTS complete:', message.task_id) + break + + case 'tts_terminate': + console.log('TTS terminated:', message.task_id) + break + + case 'message': + this.callbacks.onMessage?.(message.payload || '') + break + } + } + } + + private async setupWebRTC(): Promise { + // Get user media + this.localStream = await navigator.mediaDevices.getUserMedia({ + video: true, + audio: { + echoCancellation: true, + noiseSuppression: true, + autoGainControl: true, + sampleRate: 48000 + } + }) + + this.callbacks.onLocalStream?.(this.localStream) + + // Create RTCPeerConnection + this.pc = new RTCPeerConnection({ + iceServers: [{ urls: 'stun:stun.l.google.com:19302' }] + }) + this.remoteStream = new MediaStream() + + // Add local stream + this.localStream.getTracks().forEach(track => { + this.pc!.addTrack(track, this.localStream!) + }) + + // Set event handlers + this.setupPeerConnectionHandlers() + + // Create and send offer + const offer = await this.pc.createOffer() + await this.pc.setLocalDescription(offer) + this.sendSignal('sdp_offer', offer) + } + + private setupPeerConnectionHandlers(): void { + if (!this.pc) return + + // ICE candidate handling + this.pc.onicecandidate = (event) => { + if (event.candidate) { + this.sendSignal('ice_candidate', event.candidate) + } + } + + // Connection state changes + this.pc.onconnectionstatechange = () => { + if (this.pc) { + this.callbacks.onConnectionStateChange?.(this.pc.connectionState) + + if (this.pc.connectionState === 'failed') { + this.callbacks.onError?.(new Error('Connection failed')) + } + } + } + + // Remote stream handling + this.pc.ontrack = (event) => { + if (this.remoteStream) { + this.remoteStream.addTrack(event.track) + this.callbacks.onRemoteStream?.(this.remoteStream) + } + } + } + + private sendSignal(type: string, data: any): void { + const message = JSON.stringify({ type, data }) + const topic = `$webrtc/${this.clientId}/multimedia_proxy` + + this.mqttClient.publish(topic, message, { qos: 0 }, (err) => { + if (err) { + console.error(`Failed to send ${type}:`, err) + } + }) + } + + // Audio/video control + toggleAudio(enabled?: boolean): void { + if (!this.localStream) return + + this.localStream.getAudioTracks().forEach(track => { + track.enabled = enabled !== undefined ? enabled : !track.enabled + }) + } + + toggleVideo(enabled?: boolean): void { + if (!this.localStream) return + + this.localStream.getVideoTracks().forEach(track => { + track.enabled = enabled !== undefined ? enabled : !track.enabled + }) + } + + // Send chat message + sendChatMessage(message: string): void { + this.sendSignal('chat', message) + } + + disconnect(): void { + // Stop media streams + if (this.localStream) { + this.localStream.getTracks().forEach(track => track.stop()) + this.localStream = null + } + + if (this.remoteStream) { + this.remoteStream.getTracks().forEach(track => track.stop()) + this.remoteStream = null + } + + // Close peer connection + if (this.pc) { + this.pc.close() + this.pc = null + } + + // Clean up message handler + if (this.mqttClient && this.messageHandler) { + this.mqttClient.removeListener('message', this.messageHandler) + this.messageHandler = null + } + + // Unsubscribe + const topics = [`$webrtc/${this.clientId}`, `$message/${this.clientId}`] + this.mqttClient.unsubscribe(topics) + + this.callbacks.onConnectionStateChange?.('disconnected') + } +} +``` + +## Usage Example + +The following example shows basic usage of the API. The WebRTC signaling class can be used in any frontend environment, including Vue, React, or directly in vanilla JavaScript/HTML. + +For a complete working demo, see the [MCP AI Companion Demo](https://github.com/emqx/mcp-ai-companion-demo) (web client). + +### Basic Connection and Callback Setup + +To establish a WebRTC connection: + +1. First connect to MQTT. +2. Then configure callbacks for handling audio/video streams and AI service responses. + +```typescript +import mqtt from 'mqtt' +import { MqttWebRTCSignaling } from './mqtt-webrtc-signaling' + +// Connect to MQTT +const mqttClient = mqtt.connect('ws://broker.emqx.io:8083/mqtt', { + clientId: 'device_123', + username: 'your-username', + password: 'your-password' +}) + +// Create WebRTC signaling +const signaling = new MqttWebRTCSignaling({ + mqttClient, + clientId: 'device_123', + callbacks: { + onLocalStream: (stream) => { + document.getElementById('localVideo').srcObject = stream + }, + onRemoteStream: (stream) => { + document.getElementById('remoteVideo').srcObject = stream + }, + onASRResponse: (text) => { + console.log('Speech recognition:', text) + }, + onTTSText: (text) => { + console.log('AI reply:', text) + }, + onConnectionStateChange: (state) => { + console.log('Connection state:', state) + } + } +}) + +// Start connection +await signaling.connect() + +``` + +### Multimedia Control + +Once connected, you can easily control audio/video devices, send messages, and manage connection state: + +```typescript +// Audio and video control +signaling.toggleAudio(false) // Mute +signaling.toggleVideo(true) // Enable camera + +// Send a chat message +signaling.sendChatMessage('Hello AI!') + +// Disconnect +signaling.disconnect() +``` + +## Key Interactions + +### MQTT Signaling Workflow + +WebRTC connections use MQTT topics for signaling, following the standard offer/answer model: + +``` +1. Client creates offer → sends to $webrtc/{device_id}/multimedia_proxy +2. Server processes offer → returns answer on $webrtc/{device_id} +3. Both sides exchange ICE candidates via these topics +4. P2P connection established → audio/video streaming begins +``` + +### Multimedia AI Services + +The EMQX Multimedia Service provides three core AI features via the `$message/{device_id}` topic: + +- **ASR (Automatic Speech Recognition)**: real-time speech-to-text, supports streaming recognition. +- **TTS (Text-to-Speech)**: converts text to natural speech, supports incremental text streaming. +- **Intelligent Dialogue**: combines ASR and TTS to enable end-to-end voice interaction. + +## Error Handling and Best Practices + +In real-world applications, you must handle potential errors gracefully: + +```typescript +// Common error handling +callbacks: { + onError: (error) => { + if (error.message.includes('getUserMedia')) { + alert('Unable to access camera/microphone. Please check permissions.') + } else if (error.message.includes('Connection failed')) { + // Auto-reconnect logic + setTimeout(() => signaling?.connect(), 2000) + } + } +} +``` + +**Common solutions**: + +- **Media permission issues**: Ensure camera and microphone permissions are granted. +- **Network instability**: Implement auto-reconnect to handle disruptions. +- **Signaling timeouts**: Configure connection timeouts and show user-friendly error messages. + +## Advanced Configuration + +### ICE Server Configuration + +To ensure connectivity across different network environments, configure STUN/TURN servers: + +```typescript +const iceServers = [ + { urls: 'stun:stun.l.google.com:19302' }, // Public STUN server + { + urls: 'turn:your-turn-server.com:3478', // Private TURN server (recommended for production) + username: 'turnuser', + credential: 'turnpass' + } +] +``` + +### Media Constraints Optimization + +Adjust audio/video quality to balance performance and bandwidth usage: + +```typescript +const mediaConstraints = { + video: { + width: { ideal: 1280 }, // Video width + height: { ideal: 720 }, // Video height + frameRate: { ideal: 30 } // Frame rate + }, + audio: { + echoCancellation: true, // Echo cancellation + noiseSuppression: true, // Noise suppression + autoGainControl: true, // Automatic gain control + sampleRate: 48000, // Sampling rate + sampleSize: 16, // Bit depth + channelCount: 2 // Number of channels + } +} +``` + +## Summary + +By integrating TypeScript WebRTC with the EMQX Multimedia Service, you can easily enable: + +- **Real-time Audio/Video Calls**: high-quality, low-latency communication. +- **AI Speech Recognition**: real-time multilingual speech-to-text. +- **AI Speech Synthesis**: natural text-to-speech conversion. +- **Cross-Platform Compatibility**: works with major frontend frameworks and native JavaScript. + +This provides a complete foundation for building modern, AI-powered communication applications. diff --git a/en_US/emqx-ai/sdks/overview.md b/en_US/emqx-ai/sdks/overview.md index 3f9097ed0..d260bc7a6 100644 --- a/en_US/emqx-ai/sdks/overview.md +++ b/en_US/emqx-ai/sdks/overview.md @@ -1,3 +1,4 @@ # MCP over MQTT and Multimedia Service SDKs -EMQX provides SDKs for various platforms and programming languages to help developers quickly integrate MCP over MQTT and multimedia server features. These SDKs encapsulate the underlying communication details, allowing developers to focus on implementing business logic. +EMQX provides SDKs for various platforms and programming languages to help developers quickly integrate MCP over MQTT and Multimedia Server features. These SDKs abstract the underlying communication details, allowing developers to focus on implementing business logic. + diff --git a/zh_CN/emqx-ai/mcp-over-mqtt/architecture.md b/zh_CN/emqx-ai/mcp-over-mqtt/architecture.md index 152c9a0e8..b54243ee1 100644 --- a/zh_CN/emqx-ai/mcp-over-mqtt/architecture.md +++ b/zh_CN/emqx-ai/mcp-over-mqtt/architecture.md @@ -1,8 +1,10 @@ -# 架构 +# MCP over MQTT 架构 -## MQTT 传输的核心组件 +MCP over MQTT 在整体设计上继承了标准 MCP 架构的核心概念（Host、Client、Server），并通过引入中心化的 MQTT Broker 作为传输层，实现消息的路由、服务注册与发现、认证与授权。这种架构不仅保持了 MCP 原有的上下文交互模式，还利用 MQTT 的轻量化与广泛适用性，为物联网与边缘计算场景中的多对多通信、负载均衡与可扩展性提供了基础。 -MCP over MQTT 引入了中心化的 MQTT Broker，其他组件（Host、Client、Server）保持不变。 +## 核心组件与通信方式 + +在 MCP over MQTT 架构中，引入了中心化的 MQTT Broker 作为消息路由器，其余组件（Host、Client、Server）保持与标准 MCP 架构一致。 ```mermaid graph LR @@ -38,9 +40,9 @@ graph LR end ``` -### Host、Client 和 Server +### Host、Client 与 Server -Host、Client 和 Server 组件保持不变： +Host、Client 和 Server 组件保持不变（详见[核心组件](https://modelcontextprotocol.io/docs/learn/architecture#concepts-of-mcp)）： - Host 进程作为客户端的容器和协调者。 - 每个 Client 由 Host 创建，并维护与 Server 的独立连接。 @@ -48,20 +50,22 @@ Host、Client 和 Server 组件保持不变： 最大的区别是，Client 和 Server 现在通过 MQTT Broker 进行通信，而不是直接相互通信。另外，由于 MQTT Broker 的引入，Client 和 Server 变为多对多的关系，而不是一对一。 -详见 [核心组件](https://modelcontextprotocol.io/docs/learn/architecture#concepts-of-mcp)。 - -### MQTT Broker +### MQTT Broker 的角色 MQTT Broker 作为中心化的消息路由器： -- 促进 Client 与 Server 之间的通信。 +- 转发 Client 与 Server 之间的消息。 - 支持服务发现和服务注册（通过保留消息）。 - 对 Client 和 Server 进行认证和授权。 -## 服务端负载均衡与可扩展性 +## 服务端扩展与负载均衡 为实现 MCP 服务端负载均衡与可扩展性，MCP Server 可以启动多个实例（进程），每个实例使用唯一的 `server-id` 作为 MQTT Client ID 建立独立的 MQTT 连接。所有 MCP Server 实例共享同一个 `server-name`。 -Client 首先需要订阅服务发现主题，以获取指定 `server-name` 下的所有 `server-id` 列表。然后，Client 根据自定义的 Server 选择策略（如随机选择或轮询），向其中一个 `server-id` 发起 `initialize` 请求。初始化完成后，MCP Client 会在特定的 RPC 主题上与选定的 MCP Server 实例进行通信。 +**客户端交互流程**： + +1. Client 订阅服务发现主题，获取目标 `server-name` 下的所有可用 `server-id`。 +2. Client 根据自定义策略（如随机、轮询）选择一个 Server 实例，并向其发起 `initialize` 请求。 +3. 初始化完成后，Client 与选定的 Server 实例在专属 RPC 主题上进行通信。 ```mermaid graph LR diff --git a/zh_CN/emqx-ai/mcp-over-mqtt/overview.md b/zh_CN/emqx-ai/mcp-over-mqtt/overview.md index b2ff7006d..06d967614 100644 --- a/zh_CN/emqx-ai/mcp-over-mqtt/overview.md +++ b/zh_CN/emqx-ai/mcp-over-mqtt/overview.md @@ -1,21 +1,26 @@ # MCP over MQTT -MCP over MQTT 是一种基于 MQTT 协议的 [MCP](https://modelcontextprotocol.io/docs/getting-started/intro) 实现，旨在为 AI 应用提供高效、低延迟的工具调用以及消息通信能力。通过 MCP over MQTT，开发者可以轻松地在物联网系统中集成 AI 模型和服务，实现设备与 AI 之间的无缝交互。 +MCP over MQTT 是一种将 [Model Context Protocol（MCP）](https://modelcontextprotocol.io/docs/getting-started/intro) 运行在 MQTT 协议之上的实现方式。 + 它利用 MQTT 的轻量、高效和广泛应用的特性，将 MCP 的上下文与工具调用能力扩展到物联网与边缘计算场景，使设备与 AI 服务之间能够实现低延迟、无缝的交互。 ## 为什么使用 MQTT 实现 MCP -MQTT 是一种轻量级且广泛使用的 IoT 和边缘计算协议。它旨在应对不可靠的网络和低带宽的情况，因此非常适合边缘设备与云服务之间的通信。 +MQTT 是一种轻量级、发布/订阅模式的消息传输协议，广泛应用于 IoT 和边缘计算领域。其设计目标是应对不可靠网络和低带宽环境，因此非常适合边缘设备与云服务之间的通信需求。 通过使用 MQTT 作为 MCP 的传输层，我们将 MCP 的应用范围扩展到更广泛的场景中，包括边缘计算、物联网和云服务等任何需要使用 MQTT 的地方。 -## 特性 +## 核心特性 -MCP over MQTT 支持 MCP 的所有特性，并增加了以下特性： +在保留 MCP 原有能力的基础上，MCP over MQTT 增强并扩展了以下特性： -- **内置服务注册和发现**: MCP 客户端可以从 MQTT 代理发现可用的 MCP 服务器。 +- **内置服务注册与发现** + MCP 客户端可以通过 MQTT Broker 自动发现可用的 MCP 服务器，简化了服务接入流程。 -- **内置负载均衡和可扩展性**: MCP 服务器可以通过添加更多的 MCP 服务器实例进行水平扩展，同时保持 MCP 服务器端的状态。 +- **负载均衡与水平扩展** + MCP 服务器可通过部署多个实例实现水平扩展，并保持服务端状态一致性，从而提升系统可用性和吞吐能力。 -- **支持集中式的认证和授权**: MCP over MQTT 可以利用 MQTT Broker 的认证和授权机制，确保只有经过授权的客户端可以访问 MCP 服务。 +- **集中式认证与授权** + 依托 MQTT Broker 的认证与权限控制机制，MCP over MQTT 能确保只有经过授权的客户端才能访问指定的 MCP 服务。 -- **支持服务名管理和下发**: 在 MCP 的基础上，EMQX 增加了 MCP 服务名的概念，用于 MCP 服务的标识和分类管理。用户可以在 EMQX 上集中式地设计和下发 MCP 服务名，简化多 MCP 服务的管理和维护。 +- **服务名管理与下发** + 在标准 MCP 协议之上，EMQX 引入了 **MCP 服务名** 概念，用于对 MCP 服务进行标识和分类。管理员可在 EMQX 中集中管理与下发服务名，从而简化多服务场景下的统一管理与维护。 diff --git a/zh_CN/emqx-ai/mcp-over-mqtt/specification.md b/zh_CN/emqx-ai/mcp-over-mqtt/specification.md index cdf103cc8..aced51430 100644 --- a/zh_CN/emqx-ai/mcp-over-mqtt/specification.md +++ b/zh_CN/emqx-ai/mcp-over-mqtt/specification.md @@ -2,11 +2,11 @@ 本文档定义了 MCP 协议在 MQTT 传输层下的特殊要求，包括 MQTT 主题、客户端 ID 格式等，并描述了 MQTT 传输层的生命周期，包括服务发现、初始化、能力列表变更、资源更新和关闭流程。 -请结合 [MCP 规范](https://modelcontextprotocol.io/specification/2025-06-18) 一起阅读。 +请结合 [MCP 规范](https://modelcontextprotocol.io/specification/2025-06-18)一起阅读。 ## 术语 -- **server-name**：MCP 服务器的标识符，将包含在主题中。 +- **server-name**：MCP 服务器的标识符，将包含在 MQTT 消息主题中。 多个使用相同 `server-name` 的连接视为同一个 MCP 服务器的多个实例，提供完全相同的服务。MCP 客户端发送初始化消息时，应根据客户端侧策略选择其中一个。 @@ -83,19 +83,20 @@ MCP 客户端的 Client ID（`mcp-client-id`）为除 `/`、`+`、`#` 外的任 | `$mcp-client/presence/{mcp-client-id}` | 客户端在线状态主题，接收客户端断开通知。 | | `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`| RPC 主题，接收来自客户端的 RPC 请求、响应和通知。 | -::: info -- 服务器订阅 RPC 主题时**必须**设置 **No Local** 选项，避免收到自身消息。 +::: tip 注意 + +服务器订阅 RPC 主题时**必须**设置 **No Local** 选项，避免收到自身消息。 ::: ### MCP 服务器发布 -| 主题名称 | 消息内容 | -|-----------------------------------------------------|--------------------------------------------------------------------------| -| `$mcp-server/capability/{server-id}/{server-name}` | 能力列表变更或资源更新通知。 | -| `$mcp-server/presence/{server-id}/{server-name}` | 服务器在线状态消息，详见[服务发现](#service-discovery)。 | -| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`| RPC 请求、响应和通知。 | +| 主题名称 | 消息内容 | +| ---------------------------------------------------- | ----------------------------------------------- | +| `$mcp-server/capability/{server-id}/{server-name}` | 能力列表变更或资源更新通知。 | +| `$mcp-server/presence/{server-id}/{server-name}` | 服务器在线状态消息，详见[服务发现](#服务发现)。 | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` | RPC 请求、响应和通知。 | -::: info +::: tip 注意 - 服务器发布在线状态消息时，**必须**将 `$mcp-server/presence/{server-id}/{server-name}` 的 RETAIN 标志设为 True。 - 连接 MQTT Broker 时，服务器**必须**将 `$mcp-server/presence/{server-id}/{server-name}` 设为遗嘱主题，负载为空，用于异常断开时清除保留消息。 ::: @@ -108,8 +109,9 @@ MCP 客户端的 Client ID（`mcp-client-id`）为除 `/`、`+`、`#` 外的任 | `$mcp-server/presence/+/{server-name-filter}` | 服务器在线状态主题，接收服务器在线状态消息。 | | `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}`| RPC 主题，接收服务器发送的 RPC 请求、响应和通知。 | -::: info -- 客户端订阅 RPC 主题时**必须**设置 **No Local** 选项，避免收到自身消息。 +::: info 注意 + +客户端订阅 RPC 主题时**必须**设置 **No Local** 选项，避免收到自身消息。 ::: ### MCP 客户端发布 @@ -121,8 +123,9 @@ MCP 客户端的 Client ID（`mcp-client-id`）为除 `/`、`+`、`#` 外的任 | `$mcp-client/presence/{mcp-client-id}` | 发送客户端断开通知。 | | `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`| 向指定服务器发送 RPC 请求/响应。 | -::: info -- 连接 MQTT Broker 时，客户端**必须**将 `$mcp-client/presence/{mcp-client-id}` 设为遗嘱主题，负载为 "disconnected" 通知，用于异常断开时通知服务器。 +::: tip 注意 + +连接 MQTT Broker 时，客户端**必须**将 `$mcp-client/presence/{mcp-client-id}` 设为遗嘱主题，负载为 "disconnected" 通知，用于异常断开时通知服务器。 ::: ## 服务发现 @@ -136,7 +139,7 @@ MCP 服务器启动后，向 MQTT Broker 注册服务。服务发现与注册主 "server/online" 通知**应**只包含简要信息，避免消息过大。客户端可在初始化后请求详细信息。 - MCP 服务器功能简述，便于客户端选择初始化对象。 -- 元数据，如角色与权限，帮助客户端理解访问控制策略。`rbac` 字段可包含角色列表，每个角色含名称、描述、允许的方法、工具和资源，Broker 可据此实现 RBAC。 +- 元数据，如角色与权限，帮助客户端理解访问控制策略。`rbac` 字段可包含角色列表，每个角色含名称、描述、允许的方法、工具和资源，Broker 可据此实现基于角色的访问控制（RBAC）。 ```json { @@ -224,7 +227,7 @@ sequenceDiagram ## 初始化 -本节仅描述 MQTT 传输相关的初始化流程，详细内容见 [生命周期](https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle)。 +本节仅描述 MQTT 传输相关的初始化流程，详细内容见[生命周期](https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle)。 初始化阶段**必须**是客户端与服务器的首次交互。 diff --git a/zh_CN/emqx-ai/multimedia-ai/architecture.md b/zh_CN/emqx-ai/multimedia-ai/architecture.md index 3b8de65ca..0229facae 100644 --- a/zh_CN/emqx-ai/multimedia-ai/architecture.md +++ b/zh_CN/emqx-ai/multimedia-ai/architecture.md @@ -1,4 +1,4 @@ -# 架构 +# 多媒体服务器架构 多媒体服务器使用 WebRTC 与客户端进行点对点的音视频数据传输，使用 MQTT 协议传输普通消息和 WebRTC 信令消息。用户可以使用 Python 编写 AI 代理程序，通过 STDIO 与多媒体服务器进行通信，以实现更复杂的业务逻辑。 @@ -18,10 +18,11 @@ flowchart LR LLM[LLM] ``` -- 设备: 通过 WebRTC 与多媒体服务器进行音视频数据的交互。 -- 多媒体服务器: 负责处理来自设备的音视频数据，提供 ASR（自动语音识别）、 TTS（文本转语音）等功能，并与 AI 代理进行通信。 -- AI 代理: 通过 STDIO 接受多媒体服务器传递的 ASR 结果，包含了 AI 应用的核心业务逻辑，调用 LLM（大语言模型）处理文本自然语言，并调用多媒体服务提供的 API 发送文字或音频流给设备。 -- ASR/TTS: 语音和视觉模型提供商，提供自动语音识别和文本转语音服务。 +- **设备**: 通过 WebRTC 与多媒体服务器进行音视频数据的交互。 +- **多媒体服务器**: 负责处理来自设备的音视频数据，提供 ASR（自动语音识别）、 TTS（文本转语音）等功能，并与 AI 代理进行通信。 +- **AI 代理**: 通过 STDIO 接受多媒体服务器传递的 ASR 结果，包含了 AI 应用的核心业务逻辑，调用 LLM（大语言模型）处理文本自然语言，并调用多媒体服务提供的 API 发送文字或音频流给设备。 +- **ASR/TTS**: 语音和视觉模型提供商，提供自动语音识别和文本转语音服务。 +- **LLM**：大语言模型，用于自然语言处理与文本生成。 ## 工作流程 @@ -47,7 +48,7 @@ sequenceDiagram P1 ->> Media: Realtime Image Analysis Media ->> Media: Image Analysis Media ->> P1: Image Analysis Result - P1 ->> P2: Summary the Analysis Reuslt + P1 ->> P2: Summarize the Analysis Reuslt P2 ->> P1: Summary P1 ->> Media: TTS and send to Device Media ->> Customer: WebRTC Audio diff --git a/zh_CN/emqx-ai/multimedia-ai/message-protocol.md b/zh_CN/emqx-ai/multimedia-ai/message-protocol.md index 06bc6bc0c..b263aefa9 100644 --- a/zh_CN/emqx-ai/multimedia-ai/message-protocol.md +++ b/zh_CN/emqx-ai/multimedia-ai/message-protocol.md @@ -1,4 +1,4 @@ -# 消息协议 +# 多媒体 AI 消息协议 本文档描述了多媒体服务器与客户端（设备）和 AI 代理之间交互所使用的消息协议。 diff --git a/zh_CN/emqx-ai/multimedia-ai/overview.md b/zh_CN/emqx-ai/multimedia-ai/overview.md index 860b6e756..38bfbdd66 100644 --- a/zh_CN/emqx-ai/multimedia-ai/overview.md +++ b/zh_CN/emqx-ai/multimedia-ai/overview.md @@ -1,6 +1,6 @@ # 多媒体服务器 -EMQX 多媒体服务器是一个基于 WebRTC 技术构建的高性能多媒体处理平台。它能够接收来自客户端的 RTP/SRTP 音视频流，并集成了多种 AI 功能，如自动语音识别（ASR）、文本转语音（TTS）以及图像理解等。通过利用大模型能力，EMQX 多媒体服务器支持复杂的语音对话和工具调用，为需要音视频能力的 AI 应用提供了强大的技术支持。 +EMQX 多媒体服务器是一个基于 WebRTC 技术构建的高性能音视频处理平台。它能够接收来自客户端的 RTP/SRTP 音视频流，并集成了多种 AI 功能，如自动语音识别（ASR）、文本转语音（TTS）以及图像理解等。通过利用大模型能力，EMQX 多媒体服务器支持复杂的语音对话和工具调用，为需要音视频能力的 AI 应用提供了强大的技术支持。 ## 核心功能 diff --git a/zh_CN/emqx-ai/overview.md b/zh_CN/emqx-ai/overview.md index 438ff7441..a6377ed77 100644 --- a/zh_CN/emqx-ai/overview.md +++ b/zh_CN/emqx-ai/overview.md @@ -1,9 +1,10 @@ # EMQX AI -EMQX AI 是 EMQX 在人工智能领域的创新实践，致力于为 AI 应用提供高效、可靠的通信基础设施，分为以下两个方面： +EMQX AI 是 EMQX 在人工智能领域的创新实践，致力于为 AI 应用提供高效、可靠的通信基础设施，主要包括以下两个方向： - **消息通信：** 通过 MCP over MQTT 协议，EMQX 为 AI 应用提供了轻量、低延迟的工具调用和消息传递方式，满足 AI 应用在模型推理、实时反馈、智能化数据采集和智能化控制等场景下的通信需求。详见 [MCP over MQTT](./mcp-over-mqtt/overview.md)。 -- **多媒体流传输：** 通过基于 WebRTC 的多媒体服务器，EMQX 为 AI 应用提供了实时音视频流的传输能力，适用于智能语音、视频分析、图像理解、远程协作等场景。详见 [多媒体智能](./multimedia-ai/overview.md)。 +- **多媒体流传输：** 通过基于 WebRTC 的多媒体服务器，EMQX 为 AI 应用提供了实时音视频流的传输能力，适用于智能语音、视频分析、图像理解、远程协作等场景。详见[多媒体服务器](./multimedia-ai/overview.md)。 以上两个方案相结合，为 AI 应用构建了坚实的通信底座，助力开发者快速搭建智能化、实时化的 AI 应用。 + diff --git a/zh_CN/emqx-ai/sdks/multimedia-ai/overview.md b/zh_CN/emqx-ai/sdks/multimedia-ai/overview.md index 93b4d97ec..7dc3e0b55 100644 --- a/zh_CN/emqx-ai/sdks/multimedia-ai/overview.md +++ b/zh_CN/emqx-ai/sdks/multimedia-ai/overview.md @@ -6,6 +6,6 @@ - **移动应用**: 通过集成 WebRTC SDK（如 [Pion](https://pion.ly)）来实现与多媒体 AI 服务的交互。 -- **嵌入式设备**: 物联网设备可以通过集成与设备适配的 WebRTC 库来实现与多媒体 AI 服务的连接。如 [ESP32 WebRTC](https://github.com/espressif/esp-webrtc-solution) +- **嵌入式设备**: 物联网设备可以通过集成与设备适配的 WebRTC 库来实现与多媒体 AI 服务的连接，如 [ESP32 WebRTC](https://github.com/espressif/esp-webrtc-solution)。 -具体的 WebRTC 信令和 MQTT 消息格式请参考 [多媒体 AI 消息协议](../../multimedia-ai/message-protocol.md)。这里我们提供了基于 Web 浏览器的客户端代码示例，演示如何与多媒体服务进行交互：[Typescript WebRTC 示例](./webrtc-typescript.md)。 +具体的 WebRTC 信令和 MQTT 消息格式请参考[多媒体 AI 消息协议](../../multimedia-ai/message-protocol.md)。这里我们提供了基于 Web 浏览器的客户端代码示例，演示如何与多媒体服务进行交互：[Typescript WebRTC 示例](./webrtc-typescript.md)。 diff --git a/zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md b/zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md index 3346cb854..e4490db4a 100644 --- a/zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md +++ b/zh_CN/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md @@ -450,9 +450,9 @@ const mediaConstraints = { 通过 EMQX 多媒体服务的 TypeScript WebRTC 集成，你可以轻松实现： -- ✅ **音视频通话**: 高质量的实时音视频传输 -- ✅ **AI 语音识别**: 实时语音转文字，支持多语言 -- ✅ **智能语音合成**: 自然流畅的文字转语音 -- ✅ **跨平台兼容**: 支持各种前端框架和原生 JavaScript +- **音视频通话**: 高质量的实时音视频传输 +- **AI 语音识别**: 实时语音转文字，支持多语言 +- **智能语音合成**: 自然流畅的文字转语音 +- **跨平台兼容**: 支持各种前端框架和原生 JavaScript 这为构建现代化的 AI 驱动通信应用提供了完整的技术基础。 From 5ccb06b016bace8cb3742c310132899f2a902d85 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Thu, 18 Sep 2025 11:35:58 +0800 Subject: [PATCH 42/49] Add en translations --- en_US/emqx-ai/mcp-over-mqtt/architecture.md | 46 ++- en_US/emqx-ai/mcp-over-mqtt/overview.md | 27 +- en_US/emqx-ai/mcp-over-mqtt/specification.md | 10 +- en_US/emqx-ai/sdks/mcp-sdk-paho-c.md | 65 ++++ en_US/emqx-ai/sdks/mcp-sdk-python.md | 2 +- en_US/emqx-ai/sdks/mcp-sdk-typescript.md | 383 +++++++++++++++++++ en_US/emqx-ai/sdks/mcp-sdks-overview.md | 0 en_US/emqx-ai/sdks/overview.md | 17 + zh_CN/emqx-ai/sdks/mcp-sdks-overview.md | 0 zh_CN/emqx-ai/sdks/overview.md | 18 + 10 files changed, 531 insertions(+), 37 deletions(-) delete mode 100644 en_US/emqx-ai/sdks/mcp-sdks-overview.md delete mode 100644 zh_CN/emqx-ai/sdks/mcp-sdks-overview.md diff --git a/en_US/emqx-ai/mcp-over-mqtt/architecture.md b/en_US/emqx-ai/mcp-over-mqtt/architecture.md index 89eef53e3..43772dc89 100644 --- a/en_US/emqx-ai/mcp-over-mqtt/architecture.md +++ b/en_US/emqx-ai/mcp-over-mqtt/architecture.md @@ -1,8 +1,12 @@ -# Architecture +# MCP over MQTT Architecture + +MCP over MQTT inherits the core concepts of the standard MCP architecture (Host, Client, Server), while introducing a centralized MQTT Broker as the transport layer. The broker enables message routing, service registration and discovery, authentication, and authorization. + +This architecture not only preserves MCP’s original context interaction model but also leverages MQTT’s lightweight and broadly applicable design, providing the foundation for many-to-many communication, load balancing, and scalability in IoT and edge computing scenarios. ## Core Components of the MQTT Transport -MCP over MQTT introduces a centralized MQTT broker, while other components (Hosts, Clients, Servers) remain unchanged. +In the MCP over MQTT architecture, a centralized MQTT Broker is introduced as the message router, while other components (Host, Client, Server) remain consistent with the standard MCP design. ```mermaid graph LR @@ -40,28 +44,31 @@ graph LR ### Host, Client, and Server -The Host, Client, and Server components remain unchanged: +The Host, Client, and Server components remain unchanged (see [MCP core concepts](https://modelcontextprotocol.io/docs/learn/architecture#concepts-of-mcp)): -- The host process acts as the container and coordinator of the clients. -- Each client is created by the host and maintains an isolated server connection. -- Servers provide specialized context and capabilities. +- **Host** acts as a container and coordinator for clients. +- Each **Client** is created by the Host and maintains an independent connection with a Server. +- **Server** provides dedicated context and capabilities. -With the exception that the clients and servers communicate with the MQTT broker instead of directly with each other. +The key difference is that Clients and Servers now communicate through the MQTT Broker, instead of directly. With the broker in place, the relationship between Clients and Servers becomes many-to-many rather than one-to-one. -See [Core Components](https://modelcontextprotocol.io/docs/learn/architecture#concepts-of-mcp) for more details. +### Role of the MQTT Broker -### MQTT Broker +The MQTT Broker serves as the centralized message router: -The MQTT broker acts as a centralized message router: -- Facilitates communication between clients and servers. -- Support service discovery and service registration (via retained messages). -- Authenticates and authorizes clients and servers. +- Forwards messages between Clients and Servers. +- Supports service registration and discovery (via retained messages). +- Handles authentication and authorization for Clients and Servers. -## Server Side Load Balancing and Scalability +## Server Scaling and Load Balancing -To achieve MCP server-side load balancing and scalability, an MCP server can start multiple instances (processes), each using a unique `server-id` as the MQTT Client ID to establish an independent MQTT connection. All instances of an MCP server share the same `server-name`. +To achieve scalability and load balancing, an MCP Server can launch multiple instances (processes). Each instance connects to the broker with a unique `server-id` as its MQTT Client ID, while all instances share the same `server-name`. -The client must first subscribe to the service discovery topic to obtain the list of `server-id`s for a specific `server-name`. Then, based on a client-defined server selection strategy (e.g., random selection or round-robin), it initiates an `initialize` request to one of the `server-id`s. After initialization is complete, the MCP client communicates with the selected MCP server instance on a specific RPC topic. +**Client interaction flow:** + +1. The Client subscribes to the service discovery topic to obtain all available `server-id`s under the target `server-name`. +2. The Client selects a Server instance based on a custom policy (e.g., random or round-robin) and sends an `initialize` request. +3. After initialization, the Client communicates with the selected Server instance through a dedicated RPC topic. ```mermaid graph LR @@ -83,8 +90,7 @@ graph LR ``` -This allows us to achieve high availability and scalability on the MCP server side: - -- When scaling up, existing MCP clients remain connected to the old server instances, while new MCP clients have the opportunity to initiate initialization requests to the new server instances. +This approach enables high availability and scalability of MCP servers: -- When scaling down, MCP clients could re-initiate initialization requests to the MCP server, thereby connecting to another server instance. \ No newline at end of file +- **During scaling up**, existing MCP clients remain connected to old server instances, while new clients can initialize with newly added instances. +- **During scaling down**, MCP clients can reinitialize and connect to other available server instances. \ No newline at end of file diff --git a/en_US/emqx-ai/mcp-over-mqtt/overview.md b/en_US/emqx-ai/mcp-over-mqtt/overview.md index d69fb5f06..2439102ab 100644 --- a/en_US/emqx-ai/mcp-over-mqtt/overview.md +++ b/en_US/emqx-ai/mcp-over-mqtt/overview.md @@ -1,21 +1,24 @@ # MCP over MQTT -MCP over MQTT is an implementation of [MCP](https://modelcontextprotocol.io/docs/getting-started/intro) based on the MQTT protocol, designed to provide efficient, low-latency tool invocation and messaging capabilities for AI applications. With MCP over MQTT, developers can easily integrate AI models and services into IoT systems, enabling seamless interaction between devices and AI. +MCP over MQTT is an implementation of the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/docs/getting-started/intro) that runs on top of the MQTT protocol. -## Why Use MQTT for MCP - -MQTT is a lightweight and widely used protocol for IoT and edge computing. It is designed to handle unreliable networks and low bandwidth, making it ideal for communication between edge devices and cloud services. +By leveraging MQTT’s lightweight, efficient, and widely adopted characteristics, it extends MCP’s context and tool invocation capabilities to IoT and edge computing scenarios, enabling low-latency, seamless interaction between devices and AI services. -By using MQTT as the transport layer for MCP, we extend the application scope of MCP to a wider range of scenarios, including edge computing, IoT, and cloud services—anywhere MQTT is needed. - -## Features +## Why Use MQTT for MCP -MCP over MQTT supports all MCP features and adds the following: +MQTT is a lightweight, publish/subscribe messaging protocol that is widely used in IoT and edge computing. It is specifically designed to handle unreliable networks and low-bandwidth environments, making it an ideal choice for communication between edge devices and cloud services. -- **Built-in service registration and discovery**: MCP clients can discover available MCP servers from the MQTT broker. +By using MQTT as the transport layer for MCP, we extend MCP’s applicability to a broader range of scenarios, including edge computing, IoT, and cloud services, wherever MQTT is already in use. -- **Built-in load balancing and scalability**: MCP servers can be horizontally scaled by adding more MCP server instances, while maintaining server-side state. +## Key Features -- **Support for centralized authentication and authorization**: MCP over MQTT can leverage the authentication and authorization mechanisms of the MQTT broker, ensuring that only authorized clients can access MCP services. +In addition to preserving all the standard MCP capabilities, MCP over MQTT enhances and extends them with the following features: -- **Support for service name management and distribution**: On top of MCP, EMQX introduces the concept of MCP service names for identification and classification. Users can centrally design and distribute MCP service names on EMQX, simplifying the management and maintenance of multiple MCP services. +- **Built-in Service Registration and Discovery** + MCP clients can automatically discover available MCP servers via the MQTT broker, simplifying service integration. +- **Load Balancing and Horizontal Scalability** + MCP servers can be scaled horizontally by deploying multiple instances while maintaining state consistency, improving both availability and throughput. +- **Centralized Authentication and Authorization** + By leveraging the MQTT broker’s authentication and access control mechanisms, MCP over MQTT ensures that only authorized clients can access designated MCP services. +- **Service Name Management and Distribution** + On top of the standard MCP protocol, EMQX introduces the concept of **MCP service names** for service identification and classification. Administrators can centrally manage and distribute service names in EMQX, simplifying unified management and maintenance in multi-service environments. diff --git a/en_US/emqx-ai/mcp-over-mqtt/specification.md b/en_US/emqx-ai/mcp-over-mqtt/specification.md index 4691ba1a1..5ba9899ca 100644 --- a/en_US/emqx-ai/mcp-over-mqtt/specification.md +++ b/en_US/emqx-ai/mcp-over-mqtt/specification.md @@ -112,8 +112,9 @@ The Client ID of the MCP client, referred to as `mcp-client-id`, can be any stri | `$mcp-server/presence/+/{server-name-filter}` | The presence topic to receive the presence message of the MCP server. | | `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}` | The RPC topic to receive PRC requests, responses and notifications sent by the MCP server. | -::: info -- The client **MUST** set the **No Local** option for the RPC topic (`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}`) subscription to avoid receiving its own messages. +::: tip Note + +The client **MUST** set the **No Local** option for the RPC topic (`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}`) subscription to avoid receiving its own messages. ::: ### MCP Client Publications @@ -125,8 +126,9 @@ The Client ID of the MCP client, referred to as `mcp-client-id`, can be any stri | `$mcp-client/presence/{mcp-client-id}` | Send disconnected notification for the MCP client. | | `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` | The RPC topic to send RPC requests/responses to a specific server. | -::: info -- When connecting to the MQTT broker, the client **MUST** set `$mcp-client/presence/{mcp-client-id}` as the will topic with a "disconnected" notification as the payload to notify the server in case of an unexpected disconnection. +::: tip Note + +When connecting to the MQTT broker, the client **MUST** set `$mcp-client/presence/{mcp-client-id}` as the will topic with a "disconnected" notification as the payload to notify the server in case of an unexpected disconnection. ::: ## Service Discovery diff --git a/en_US/emqx-ai/sdks/mcp-sdk-paho-c.md b/en_US/emqx-ai/sdks/mcp-sdk-paho-c.md index e69de29bb..f0a99587d 100644 --- a/en_US/emqx-ai/sdks/mcp-sdk-paho-c.md +++ b/en_US/emqx-ai/sdks/mcp-sdk-paho-c.md @@ -0,0 +1,65 @@ +# C SDK with Paho MQTT + +This guide demonstrates how to use the [MCP over MQTT C SDK with Paho MQTT](https://github.com/mqtt-ai/paho-mcp-over-mqtt) to create a simple **MCP over MQTT server**. + Currently, only the MCP server is supported. You can use the Python SDK to create an MCP client for interaction. + +## Create an MCP Server + +Following the instructions in the [C SDK with Paho MQTT README](https://github.com/mqtt-ai/paho-mcp-over-mqtt), after installing the dependencies and SDK, create a file named `demo_mcp_server.c` and add the following code: + +```c +#include "mcp_server.h" + +// Callback function to get temperature +const char* get_temperature_callback(int n_args, property_t *args) { + // Read sensor data + float temp = read_temperature_sensor(); + + // Return JSON formatted result + static char result[64]; + snprintf(result, sizeof(result), "{\"temperature\": %.2f}", temp); + return result; +} + +// Define MCP tools +mcp_tool_t my_tools[] = { + { + .name = "get_temperature", + .description = "Get device temperature", + .property_count = 0, + .properties = NULL, + .call = get_temperature_callback + } +}; + +// Initialize MCP server +mcp_server_t *server = mcp_server_init( + "sensor", // Server name + "Sensor MCP Server", // Description + "mqtt://broker.example.com", // MQTT Broker URI + "client_001", // Client ID + "username", // Username + "password", // Password + NULL // Certificate (optional) +); + +// Register tools +mcp_server_register_tool(server, 1, my_tools); + +// Start server +mcp_server_run(server); +``` + +## Create and Run an MCP Client with Python SDK + +Refer to the [Python SDK](./mcp-sdk-python.md) documentation to create an MCP client that connects to the MCP server created above and invokes the `get_temperature` tool. + +## Build and Compile the MCP Server with CMake + +You can follow the **CMake examples** provided in the [paho-mcp-over-mqtt repository](https://github.com/mqtt-ai/paho-mcp-over-mqtt). + +After building, run the generated executable to start the MCP server: + +```bash +./demo_mcp_server +``` \ No newline at end of file diff --git a/en_US/emqx-ai/sdks/mcp-sdk-python.md b/en_US/emqx-ai/sdks/mcp-sdk-python.md index a850618c6..3795c53d3 100644 --- a/en_US/emqx-ai/sdks/mcp-sdk-python.md +++ b/en_US/emqx-ai/sdks/mcp-sdk-python.md @@ -108,7 +108,7 @@ if __name__ == "__main__": ## Run the Demo -1. First, install the required dependencies: +1. Install the required dependencies: ```bash uv add git+https://github.com/emqx/mcp-python-sdk --branch main diff --git a/en_US/emqx-ai/sdks/mcp-sdk-typescript.md b/en_US/emqx-ai/sdks/mcp-sdk-typescript.md index e69de29bb..dcf18fbd9 100644 --- a/en_US/emqx-ai/sdks/mcp-sdk-typescript.md +++ b/en_US/emqx-ai/sdks/mcp-sdk-typescript.md @@ -0,0 +1,383 @@ +# TypeScript SDK + +This guide demonstrates how to use [@emqx-ai/mcp-mqtt-sdk](https://github.com/emqx/mcp-typescript-sdk) to create an MCP over MQTT server and client. + The SDK supports both browser and Node.js environments and provides complete TypeScript type safety. + +For convenience, this tutorial runs the demo in a Node.js environment. However, you can easily integrate it into a browser environment and use it with frameworks like Vue or React. + +## Create a Demo Project + +First, create a new Node.js project (Node.js >= 18 required): + +```bash +mkdir mcp_typescript_demo +cd mcp_typescript_demo +npm init -y +``` + +## Install Dependencies + +Install the TypeScript MCP SDK: + +```bash +# Using npm +npm install @emqx-ai/mcp-mqtt-sdk +npm install -D typescript @types/node ts-node + +# Or using yarn +yarn add @emqx-ai/mcp-mqtt-sdk +yarn add -D typescript @types/node ts-node + +# Or using pnpm +pnpm add @emqx-ai/mcp-mqtt-sdk +pnpm add -D typescript @types/node ts-node +``` + +## Create a Simple MCP Server + +In the `mcp_typescript_demo` project, create a simple MCP server exposing calculator tools and resources. + Create a file named `demo_mcp_server.ts` and add the following code: + +```typescript +// demo_mcp_server.ts +import { McpMqttServer } from "@emqx-ai/mcp-mqtt-sdk"; + +// Create MCP server +const server = new McpMqttServer({ + host: "mqtt://broker.emqx.io:1883", + serverId: "demo-calculator-server", + serverName: "demo_server/calculator", + name: "Calculator MCP Server", + version: "1.0.0", + description: "A simple calculator MCP server", + capabilities: { + tools: { listChanged: true }, + resources: { listChanged: true }, + }, +}); + +// Add addition tool +server.tool( + "add", + "Add two numbers", + { + type: "object", + properties: { + a: { type: "number", description: "The first number" }, + b: { type: "number", description: "The second number" }, + }, + required: ["a", "b"], + }, + async (params: Record) => { + const { a, b } = params as { a: number; b: number }; + const result = a + b; + return { + content: [ + { + type: "text", + text: `${a} + ${b} = ${result}`, + }, + ], + }; + }, +); + +// Add multiplication tool +server.tool( + "multiply", + "Multiply two numbers", + { + type: "object", + properties: { + a: { type: "number", description: "The first number" }, + b: { type: "number", description: "The second number" }, + }, + required: ["a", "b"], + }, + async (params: Record) => { + const { a, b } = params as { a: number; b: number }; + const result = a * b; + return { + content: [ + { + type: "text", + text: `${a} × ${b} = ${result}`, + }, + ], + }; + }, +); + +// Add personalized greeting resources +const names = ["Alice", "Bob", "Charlie", "Diana", "World"]; +names.forEach((name) => { + server.resource( + `greeting://${name}`, + `Personalized greeting - ${name}`, + async () => { + return { + contents: [ + { + uri: `greeting://${name}`, + mimeType: "text/plain", + text: `Hello, ${name}! Welcome to our calculator service.`, + }, + ], + }; + }, + { + description: `Generate a personalized greeting message for ${name}`, + mimeType: "text/plain", + }, + ); +}); + +// Add server status resource +server.resource( + "status://server", + "Server status", + async () => { + return { + contents: [ + { + uri: "status://server", + mimeType: "application/json", + text: JSON.stringify( + { + name: "Calculator MCP Server", + status: "running", + uptime: process.uptime(), + availableTools: ["add", "multiply"], + timestamp: new Date().toISOString(), + }, + null, + 2, + ), + }, + ], + }; + }, + { + description: "Server runtime status information", + mimeType: "application/json", + }, +); + +// Event handling +server.on("ready", () => { + console.log("Calculator MCP Server started"); +}); + +server.on("error", (error) => { + console.error("Server error:", error); +}); + +// Start server +async function startServer() { + try { + await server.start(); + } catch (error) { + console.error("Failed to start server:", error); + process.exit(1); + } +} + +// Graceful shutdown +process.on("SIGINT", async () => { + console.log("Shutting down server..."); + await server.stop(); + process.exit(0); +}); + +startServer(); +``` + +## Create a Simple MCP Client + +In the same project, create a simple MCP client that connects to the server and lists available tools and resources. +Create a file named `demo_mcp_client.ts` and add the following code: + +```typescript +// demo_mcp_client.ts +import { McpMqttClient } from "@emqx-ai/mcp-mqtt-sdk"; + +// Create MCP client +const client = new McpMqttClient({ + host: "mqtt://broker.emqx.io:1883", + name: "Demo MCP Client", + version: "1.0.0", +}); + +async function onServerDiscovered(server: any) { + console.log(`Discovered server ${server.name}, connecting...`); + await client.initializeServer(server.serverId); +} + +async function onServerConnected(server: any, initResult: any) { + if (!initResult) { + console.error(`Failed to connect to ${server.name}`); + return; + } + + console.log(`Connected to ${server.name}`); + const capabilities = initResult.capabilities; + + // List tools + if (capabilities.tools) { + try { + const tools = await client.listTools(server.serverId); + console.log( + `${server.name} tools:`, + tools.map((t) => t.name), + ); + + // Test addition tool + if (tools.some((t) => t.name === "add")) { + const result = await client.callTool(server.serverId, "add", { + a: 1, + b: 2, + }); + console.log("Result of add(a=1, b=2):", result.content[0]?.text); + } + + // Test multiplication tool + if (tools.some((t) => t.name === "multiply")) { + const result = await client.callTool(server.serverId, "multiply", { + a: 3, + b: 4, + }); + console.log("Result of multiply(a=3, b=4):", result.content[0]?.text); + } + } catch (error) { + console.error("Tool operation error:", error); + } + } + + // List and read resources + if (capabilities.resources) { + try { + const resources = await client.listResources(server.serverId); + console.log( + `${server.name} resources:`, + resources.map((r) => r.uri), + ); + + // Read server status + if (resources.some((r) => r.uri === "status://server")) { + const status = await client.readResource( + server.serverId, + "status://server", + ); + console.log("Server status:", status.contents[0]?.text); + } + + // Read dynamic greeting resource + const greeting = await client.readResource( + server.serverId, + "greeting://Alice", + ); + console.log("Greeting resource:", greeting.contents[0]?.text); + } catch (error) { + console.error("Resource operation error:", error); + } + } +} + +async function onServerDisconnected(serverId: string) { + console.log(`Disconnected from server ${serverId}`); +} + +// Register event handlers +client.on("serverDiscovered", onServerDiscovered); +client.on("serverInitialized", (server) => { + // For demo purposes, mock the initialization result + onServerConnected(server, { capabilities: { tools: true, resources: true } }); +}); +client.on("serverDisconnected", onServerDisconnected); +client.on("error", (error) => { + console.error("Client error:", error); +}); + +// Start client +async function startClient() { + try { + await client.connect(); + console.log("Demo MCP Client started"); + + // Keep running + while (true) { + // Simulate other work while MQTT client runs in the background + await new Promise((resolve) => setTimeout(resolve, 20000)); + } + } catch (error) { + console.error("Failed to start client:", error); + process.exit(1); + } +} + +// Graceful shutdown +process.on("SIGINT", async () => { + console.log("Shutting down client..."); + await client.disconnect(); + process.exit(0); +}); + +startClient(); +``` + +## Configure the Project + +Since the SDK uses ES modules, configure your project to support modern JavaScript module syntax. + +Add module type and scripts to `package.json`: + +```json +{ + "type": "module", + "scripts": { + "start:server": "ts-node --esm demo_mcp_server.ts", + "start:client": "ts-node --esm demo_mcp_client.ts" + } +} +``` + +Create a `tsconfig.json` file: + +```json +{ + "compilerOptions": { + "target": "ES2022", + "module": "ESNext", + "moduleResolution": "node", + "esModuleInterop": true, + "allowSyntheticDefaultImports": true, + "strict": false, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true + }, + "ts-node": { + "esm": true + } +} +``` + +## Run the Demo + +1. Start the client: + +```bash +npm run start:client +``` + +2. Open a new terminal and start the server: + +```bash +npm run start:server +``` + +Even if the client starts before the server, it will discover and connect once the server becomes available. +The client will list available tools, call the `add` tool with parameters `a=1` and `b=2`, and call the `multiply` tool with parameters `a=3` and `b=4`. + +## Conclusion + +With this end-to-end demo, you’ve successfully created a fully functional MCP over MQTT system. Now, large models such as DeepSeek, Claude, GPT, and Gemini can discover and invoke your exposed calculator tools via the MCP protocol, enabling seamless integration and intelligent interaction with external services. \ No newline at end of file diff --git a/en_US/emqx-ai/sdks/mcp-sdks-overview.md b/en_US/emqx-ai/sdks/mcp-sdks-overview.md deleted file mode 100644 index e69de29bb..000000000 diff --git a/en_US/emqx-ai/sdks/overview.md b/en_US/emqx-ai/sdks/overview.md index d260bc7a6..20cb4a703 100644 --- a/en_US/emqx-ai/sdks/overview.md +++ b/en_US/emqx-ai/sdks/overview.md @@ -2,3 +2,20 @@ EMQX provides SDKs for various platforms and programming languages to help developers quickly integrate MCP over MQTT and Multimedia Server features. These SDKs abstract the underlying communication details, allowing developers to focus on implementing business logic. +## Multimedia Service Client Code Examples + +EMQX offers client-side examples based on WebRTC and MQTT, demonstrating how to interact with the multimedia server for audio/video calls, speech recognition (ASR), and text-to-speech (TTS). + +Developers can use these examples as a reference to quickly integrate multimedia AI capabilities into web applications or devices. + +- [TypeScript WebRTC Example](./webrtc-typescript.md) + +## MCP over MQTT SDKs + +EMQX provides SDKs in multiple programming languages, making it easy to integrate MCP over MQTT across different platforms and environments: + +- [Erlang SDK](./mcp-sdk-erlang.md) +- [ESP32 C SDK](./mcp-sdk-esp32-c.md) +- [C SDK with Paho MQTT](./mcp-sdk-paho-c.md) +- [Python SDK](./mcp-sdk-python.md) +- [TypeScript SDK](./mcp-sdk-typescript.md) diff --git a/zh_CN/emqx-ai/sdks/mcp-sdks-overview.md b/zh_CN/emqx-ai/sdks/mcp-sdks-overview.md deleted file mode 100644 index e69de29bb..000000000 diff --git a/zh_CN/emqx-ai/sdks/overview.md b/zh_CN/emqx-ai/sdks/overview.md index bfb490aae..e0be3bdbf 100644 --- a/zh_CN/emqx-ai/sdks/overview.md +++ b/zh_CN/emqx-ai/sdks/overview.md @@ -1,3 +1,21 @@ # MCP over MQTT 和多媒体服务 SDK EMQX 提供了多种平台和编程语言的 SDK，帮助开发者快速集成 MCP over MQTT 和多媒体服务器的功能。这些 SDK 封装了底层的通信细节，使开发者能够专注于业务逻辑的实现。 + +## 多媒体服务客户端代码示例 + +EMQX 提供了基于 WebRTC 和 MQTT 的客户端示例，演示如何与多媒体服务器交互，实现音视频通话、语音识别（ASR）和文本转语音（TTS）等功能。 + +开发者可以直接使用这些示例作为参考，将多媒体 AI 能力快速集成到 Web 应用或设备中。 + +- [TypeScript WebRTC 示例](./webrtc-typescript.md) + +## MCP over MQTT SDKs + +EMQX 提供了多种语言的 SDK，方便开发者在不同平台和环境中集成 MCP over MQTT： + +- [Erlang SDK](./mcp-sdk-erlang.md) +- [ESP32 C SDK](./mcp-sdk-esp32-c.md) +- [C SDK with Paho MQTT](./mcp-sdk-paho-c.md) +- [Python SDK](./mcp-sdk-python.md) +- [TypeScript SDK](./mcp-sdk-typescript.md) From 7bee3af53866d108d3348860cc7b54347c084960 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Thu, 18 Sep 2025 11:37:41 +0800 Subject: [PATCH 43/49] Update architecture.md --- en_US/emqx-ai/multimedia-ai/architecture.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/en_US/emqx-ai/multimedia-ai/architecture.md b/en_US/emqx-ai/multimedia-ai/architecture.md index f41d616e8..98c535bc8 100644 --- a/en_US/emqx-ai/multimedia-ai/architecture.md +++ b/en_US/emqx-ai/multimedia-ai/architecture.md @@ -20,7 +20,7 @@ flowchart LR LLM[LLM] ``` -### Components +## Components - **Device**: Exchanges audio and video data with the Multimedia Server via WebRTC. - **Multimedia Server**: Processes audio and video streams from devices, provides Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) services, and communicates with the AI Agent. From 133cb273ef0edfb92c2ab7ac5e196da86f202da7 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Thu, 18 Sep 2025 11:49:49 +0800 Subject: [PATCH 44/49] Update files in ja_JP folder --- en_US/emqx-ai/sdks/overview.md | 2 +- ja_JP/emqx-ai/mcp-over-mqtt/architecture.md | 96 ++++ ja_JP/emqx-ai/mcp-over-mqtt/overview.md | 24 + ja_JP/emqx-ai/mcp-over-mqtt/specification.md | 498 ++++++++++++++++++ ja_JP/emqx-ai/multimedia-ai/architecture.md | 62 +++ .../emqx-ai/multimedia-ai/message-protocol.md | 378 +++++++++++++ ja_JP/emqx-ai/multimedia-ai/overview.md | 33 ++ ja_JP/emqx-ai/overview.md | 9 + ja_JP/emqx-ai/sdks/mcp-sdk-erlang.md | 189 +++++++ ja_JP/emqx-ai/sdks/mcp-sdk-esp32-c.md | 59 +++ ja_JP/emqx-ai/sdks/mcp-sdk-paho-c.md | 65 +++ ja_JP/emqx-ai/sdks/mcp-sdk-python.md | 130 +++++ ja_JP/emqx-ai/sdks/mcp-sdk-typescript.md | 383 ++++++++++++++ ja_JP/emqx-ai/sdks/mcp-sdks-overview.md | 0 ja_JP/emqx-ai/sdks/multimedia-ai/overview.md | 11 + .../sdks/multimedia-ai/webrtc-typescript.md | 464 ++++++++++++++++ ja_JP/emqx-ai/sdks/overview.md | 21 + zh_CN/emqx-ai/sdks/overview.md | 2 +- 18 files changed, 2424 insertions(+), 2 deletions(-) delete mode 100644 ja_JP/emqx-ai/sdks/mcp-sdks-overview.md diff --git a/en_US/emqx-ai/sdks/overview.md b/en_US/emqx-ai/sdks/overview.md index 20cb4a703..9f3764753 100644 --- a/en_US/emqx-ai/sdks/overview.md +++ b/en_US/emqx-ai/sdks/overview.md @@ -8,7 +8,7 @@ EMQX offers client-side examples based on WebRTC and MQTT, demonstrating how to Developers can use these examples as a reference to quickly integrate multimedia AI capabilities into web applications or devices. -- [TypeScript WebRTC Example](./webrtc-typescript.md) +- [TypeScript WebRTC Example](./multimedia-ai/webrtc-typescript.md) ## MCP over MQTT SDKs diff --git a/ja_JP/emqx-ai/mcp-over-mqtt/architecture.md b/ja_JP/emqx-ai/mcp-over-mqtt/architecture.md index e69de29bb..43772dc89 100644 --- a/ja_JP/emqx-ai/mcp-over-mqtt/architecture.md +++ b/ja_JP/emqx-ai/mcp-over-mqtt/architecture.md @@ -0,0 +1,96 @@ +# MCP over MQTT Architecture + +MCP over MQTT inherits the core concepts of the standard MCP architecture (Host, Client, Server), while introducing a centralized MQTT Broker as the transport layer. The broker enables message routing, service registration and discovery, authentication, and authorization. + +This architecture not only preserves MCP’s original context interaction model but also leverages MQTT’s lightweight and broadly applicable design, providing the foundation for many-to-many communication, load balancing, and scalability in IoT and edge computing scenarios. + +## Core Components of the MQTT Transport + +In the MCP over MQTT architecture, a centralized MQTT Broker is introduced as the message router, while other components (Host, Client, Server) remain consistent with the standard MCP design. + +```mermaid +graph LR + subgraph "Application Host Process" + H[Host] + C1[Client 1] + C2[Client 2] + C3[Client 3] + H --> C1 + H --> C2 + H --> C3 + end + + subgraph "MQTT Broker" + B[Broker] + C1 --> B + C2 --> B + C3 --> B + end + + subgraph "Servers" + S1[Server A
External APIs] + R1[("Remote
Resource A")] + B --> S1 + S1 <--> R1 + end + + subgraph "Servers" + S2[Server B
External APIs] + R2[("Remote
Resource B")] + B --> S2 + S2 <--> R2 + end +``` + +### Host, Client, and Server + +The Host, Client, and Server components remain unchanged (see [MCP core concepts](https://modelcontextprotocol.io/docs/learn/architecture#concepts-of-mcp)): + +- **Host** acts as a container and coordinator for clients. +- Each **Client** is created by the Host and maintains an independent connection with a Server. +- **Server** provides dedicated context and capabilities. + +The key difference is that Clients and Servers now communicate through the MQTT Broker, instead of directly. With the broker in place, the relationship between Clients and Servers becomes many-to-many rather than one-to-one. + +### Role of the MQTT Broker + +The MQTT Broker serves as the centralized message router: + +- Forwards messages between Clients and Servers. +- Supports service registration and discovery (via retained messages). +- Handles authentication and authorization for Clients and Servers. + +## Server Scaling and Load Balancing + +To achieve scalability and load balancing, an MCP Server can launch multiple instances (processes). Each instance connects to the broker with a unique `server-id` as its MQTT Client ID, while all instances share the same `server-name`. + +**Client interaction flow:** + +1. The Client subscribes to the service discovery topic to obtain all available `server-id`s under the target `server-name`. +2. The Client selects a Server instance based on a custom policy (e.g., random or round-robin) and sends an `initialize` request. +3. After initialization, the Client communicates with the selected Server instance through a dedicated RPC topic. + +```mermaid +graph LR + + C1["MCP Client1"] + C2["MCP Client2"] + C3["MCP Client3"] + C4["MCP Client4"] + + subgraph "MCP Server Instances (server-name-a)" + S1[Server Instance 1] + S2[Server Instance 2] + end + + C1 <-- "RPC topic of client-1 and server instance 1" --> S1 + C2 <-- "RPC topic of client-2 and server instance 1" --> S1 + C3 <-- "RPC topic of client-3 and server instance 2" --> S2 + C4 <-- "RPC topic of client-4 and server instance 2" --> S2 + +``` + +This approach enables high availability and scalability of MCP servers: + +- **During scaling up**, existing MCP clients remain connected to old server instances, while new clients can initialize with newly added instances. +- **During scaling down**, MCP clients can reinitialize and connect to other available server instances. \ No newline at end of file diff --git a/ja_JP/emqx-ai/mcp-over-mqtt/overview.md b/ja_JP/emqx-ai/mcp-over-mqtt/overview.md index e69de29bb..2439102ab 100644 --- a/ja_JP/emqx-ai/mcp-over-mqtt/overview.md +++ b/ja_JP/emqx-ai/mcp-over-mqtt/overview.md @@ -0,0 +1,24 @@ +# MCP over MQTT + +MCP over MQTT is an implementation of the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/docs/getting-started/intro) that runs on top of the MQTT protocol. + +By leveraging MQTT’s lightweight, efficient, and widely adopted characteristics, it extends MCP’s context and tool invocation capabilities to IoT and edge computing scenarios, enabling low-latency, seamless interaction between devices and AI services. + +## Why Use MQTT for MCP + +MQTT is a lightweight, publish/subscribe messaging protocol that is widely used in IoT and edge computing. It is specifically designed to handle unreliable networks and low-bandwidth environments, making it an ideal choice for communication between edge devices and cloud services. + +By using MQTT as the transport layer for MCP, we extend MCP’s applicability to a broader range of scenarios, including edge computing, IoT, and cloud services, wherever MQTT is already in use. + +## Key Features + +In addition to preserving all the standard MCP capabilities, MCP over MQTT enhances and extends them with the following features: + +- **Built-in Service Registration and Discovery** + MCP clients can automatically discover available MCP servers via the MQTT broker, simplifying service integration. +- **Load Balancing and Horizontal Scalability** + MCP servers can be scaled horizontally by deploying multiple instances while maintaining state consistency, improving both availability and throughput. +- **Centralized Authentication and Authorization** + By leveraging the MQTT broker’s authentication and access control mechanisms, MCP over MQTT ensures that only authorized clients can access designated MCP services. +- **Service Name Management and Distribution** + On top of the standard MCP protocol, EMQX introduces the concept of **MCP service names** for service identification and classification. Administrators can centrally manage and distribute service names in EMQX, simplifying unified management and maintenance in multi-service environments. diff --git a/ja_JP/emqx-ai/mcp-over-mqtt/specification.md b/ja_JP/emqx-ai/mcp-over-mqtt/specification.md index e69de29bb..5ba9899ca 100644 --- a/ja_JP/emqx-ai/mcp-over-mqtt/specification.md +++ b/ja_JP/emqx-ai/mcp-over-mqtt/specification.md @@ -0,0 +1,498 @@ +# Specification + +This specification defines the MQTT-specific requirements like MQTT topics and client ID formats. It also outlines the lifecycle of the MQTT transport, including service discovery, initialization, capability list changes, resource updates, and shutdown procedures. + +It should be read in conjunction with the [MCP Specification](https://modelcontextprotocol.io/specification/2025-06-18). + +## Terminology + +- **server-name**: The server name is the identifier of a MCP server, which will be included in the topic. + + Multiple connections with the same `server-name` are considered multiple instances of the same MCP server and provide exactly the same service. When the MCP client sends an initialize message, it should select one of them according to a client-side determined strategy. + + Multiple MCP servers with different `server-name`s may still provide similar functions. In this case, when the client sends an initialize message, it should select one of them to establish a connection as needed. The selection criteria can be based on the client's permissions, recommendations from a LLM, or the user's choice. + + After connected to the MQTT broker, the broker may suggest a `server-name` to the MCP server by including a `MCP-SERVER-NAME` user property in the MQTT CONNECT message. If so, the MCP server **MUST** use this `server-name` as its server name. If the broker does not suggest a `server-name`, the MCP server **SHOULD** use a default `server-name` based on the functionality it provides. + + The `server-name` must be a hierarchical topic style separated by `/` so that the client can subscribe to a certain type of MCP server using MQTT topic wildcards, for example: `server-type/sub-type/name`. + + The `server-name` should not `+` and `#` characters. + + The `server-name` should be unique among all MCP servers. + +- **server-name-filter**: The MQTT topic filter to match the `server-name`, it may include `/`, `+` and `#` characters. See descriptions about **server-name** for more details. + + After connected to the MQTT broker, the broker may suggest a `server-name-filter` to the MCP client by including a `MCP-SERVER-NAME-FILTERS` user property in the MQTT CONNACK message. If so, the MCP client **MUST** use this `server-name-filter` to subscribe to the server's presence topic. The value of the `MCP-SERVER-NAME-FILTERS` is a JSON array of strings, each string is a MQTT topic filter. + If the broker does not suggest a `server-name-filter`, the MCP client **SHOULD** use a default `server-name-filter` based on the functionality it provides. + +- **server-id**: The MQTT Client ID of a MCP server instance. Any string except `/`, `+` and `#`. It must be globally unique and will also be included in the topic. + +- **mcp-client-id**: The MQTT Client ID of the client. Any string except `/`, `+` and `#`. It must be globally unique and will be included in the topic. Each time an initialization request is made, a different client-id must be used. + +## Message Topics + +MCP over MQTT transmits messages through MQTT topics. This protocol includes the following message topics: + +| Topic Name | Topic Name | Description | +|----------------------------------|---------------------------------------------------------------------|------------------------------------------------------------------------------------| +| Server's Control Topic | `$mcp-server/{server-id}/{server-name}` | Used for sending and receiving initialization messages and other control messages. | +| Server's Capability Change Topic | `$mcp-server/capability/{server-id}/{server-name}` | Used for sending and receiving server capability list changed or the resource updated notification. | +| Server's Presence Topic | `$mcp-server/presence/{server-id}/{server-name}` | Used for sending and receiving server's online/offline status messages. | +| Client's Presence Topic | `$mcp-client/presence/{mcp-client-id}` | Used for sending and receiving client's online/offline status messages. | +| Client's Capability Change Topic | `$mcp-client/capability/{mcp-client-id}` | Used for sending and receiving client capability list changed notification. | +| RPC Topic | `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` | Used for sending and receiving RPC requests/responses, and notification messages. | + +## MQTT Protocol Version + +The MCP server and client **MUST** use MQTT Protocol version 5.0. + +## User Property + +For `CONNECT` messages, the following user properties **MUST** be set: +- `MCP-COMPONENT-TYPE`: `mcp-client` or `mcp-server`. +- `MCP-META`: A JSON object containing metadata about the MCP component, such as its version, implementation details, and location. This metadata can be used by the broker to suggest a server name to the MCP server or a server name filter to the MCP client. + +For `CONNACK` messages sent by the broker, the following user properties **MAY** be set: +- `MCP-SERVER-NAME`: The broker suggested server name for the MCP server. Only present if it's a MCP server. +- `MCP-RBAC`: A JSON array of server names and its corresponding role names, which can be used by the MCP client to determine the roles it has for the MCP server. Each element in the array is a JSON object with two fields: `server_name` and `role_name`. Only present if it's a MCP client. +- `MCP-SERVER-NAME-FILTERS`: The broker suggested server name filters. It's a JSON array of strings, each string is a MQTT topic filter that the MCP client can use to subscribe to the server's presence topic. This allows the client to filter the servers it is interested in based on its permissions or other criteria. Only present if it's a MCP client. + +For `PUBLISH` messages, the following user properties **MUST** be set: +- `MCP-COMPONENT-TYPE`: `mcp-client` or `mcp-server`. +- `MCP-MQTT-CLIENT-ID`: MQTT client ID of the sender. + +## Session Expiry Interval + +The session expiry interval **MUST** be set to 0, meaning the session will be cleaned up when the client disconnects. + +## MQTT Client ID + +### MCP Server + +The Client ID of the MCP server can be any string except `/`, `+` and `#`, referred to as `server-id`. + +### MCP Client + +The Client ID of the MCP client, referred to as `mcp-client-id`, can be any string except `/`, `+` and `#`, each time an initialization request is made, a different client-id must be used. + +## MQTT Topics and Topic Filters + +### MCP Server Subscriptions + +| Topic Filter | Explanation | +|-------------------------------------------------------|----------------------------------------------------------------------------------------------------------| +| `$mcp-server/{server-id}/{server-name}` | The control topic of the MCP server to receive control messages. | +| `$mcp-client/capability/{mcp-client-id}` | The MCP client’s capability change topic to receive capability list changed notification of the clients. | +| `$mcp-client/presence/{mcp-client-id}` | The MCP client’s presence topic to receive the disconnected notification of the clients. | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` | The RPC topic to receive RPC requests, RPC responses, and notifications from a MCP client. | + +::: info +- The server **MUST** set the **No Local** option for the RPC topic (`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`) subscription to avoid receiving its own messages. +::: + +### MCP Server Publications + +| Topic Name | Messages | +|------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------| +| `$mcp-server/capability/{server-id}/{server-name}` | capability list changed or resource updated notification. | +| `$mcp-server/presence/{server-id}/{server-name}` | Presence messages for the MCP server.
See [ServiceDiscovery](#service-discovery) for more details | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` | RPC requests, responses and notifications. | + +::: info +- The server **MUST** set the **RETAIN** flag to `True` for the topic `$mcp-server/presence/{server-id}/{server-name}` when publishing the server presence message. +- When connecting to the MQTT broker, the server **MUST** set `$mcp-server/presence/{server-id}/{server-name}` as the will topic with an empty payload to clear the retain message in case of an unexpected disconnection. +::: + + +### MCP Client Subscriptions + +| Topic Filter | Explanation | +|--------------------------------------------------------------------|------------------------------------------------------------------------------------------------| +| `$mcp-server/capability/{server-id}/{server-name-filter}` | The capability change topic to receive capability list changed or resource updated notification of the MCP server. | +| `$mcp-server/presence/+/{server-name-filter}` | The presence topic to receive the presence message of the MCP server. | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}` | The RPC topic to receive PRC requests, responses and notifications sent by the MCP server. | + +::: tip Note + +The client **MUST** set the **No Local** option for the RPC topic (`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}`) subscription to avoid receiving its own messages. +::: + +### MCP Client Publications + +| Topic Name | Messages | +|------------------------------------------------------|--------------------------------------------------------------------| +| `$mcp-server/{server-id}/{server-name}` | Send control messages like the initialize request. | +| `$mcp-client/capability/{mcp-client-id}` | Send client capability list changed notification | +| `$mcp-client/presence/{mcp-client-id}` | Send disconnected notification for the MCP client. | +| `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` | The RPC topic to send RPC requests/responses to a specific server. | + +::: tip Note + +When connecting to the MQTT broker, the client **MUST** set `$mcp-client/presence/{mcp-client-id}` as the will topic with a "disconnected" notification as the payload to notify the server in case of an unexpected disconnection. +::: + +## Service Discovery + +### Service Registration + +After the MCP server starts, it registers its service with the MQTT broker. The presence topic for service discovery and registration is: `$mcp-server/presence/{server-id}/{server-name}`. + +The MCP server **MUST** publish a "server/online" notification to the service presence topic when they start, with the **RETAIN** flag set to `True`. + +The "server/online" notification **SHOULD** provide only limited information about the server to avoid the message size being too large. The client can request more detailed information after initialization. + +- A brief description of the MCP server's functionality to help clients determine which MCP servers they need to initialize. +- Some metadata, such as roles and permissions, to help clients understand the access control policies of the MCP server. The `rbac` field in the metadata can include roles, each with a name, description, allowed methods, allowed tools, and allowed resources, which maybe used by the MQTT broker to implement role-based access control (RBAC) for the MCP server. + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/server/online", + "params": { + "server_name": "example/server", + "description": "This is a brief description about the functionalities provided by this MCP server to allow clients to choose as needed. If tools are provided, it explains what tools are available but does not include tool parameters to reduce message size.", + "meta": { + "rbac": { + "roles": [ + { + "name": "admin", + "description": "Administrator role with full access", + "allowed_methods": [ + "notifications/initialized", + "ping", "tools/list", "tools/call", "resources/list", "resources/read", + "resources/subscribe", "resources/unsubscribe" + ], + "allowed_tools": "all", + "allowed_resources": "all" + }, + { + "name": "user", + "description": "User role with limited access", + "allowed_methods": [ + "notifications/initialized", + "ping", "tools/list", "tools/call", "resources/list", "resources/read" + ], + "allowed_tools": [ + "get_vehicle_status", "get_vehicle_location" + ], + "allowed_resources": [ + "file:///vehicle/telemetry.data" + ] + } + ] + } + } + } +} +``` + +More detailed information, such as parameter details of the tools, **SHOULD** only be fetched by the client when needed, by sending `**/list` requests to the server. + +The client can subscribe to the `$mcp-server/presence/+/{server-name-filter}` topic at any time, where `{server-name-filter}` is a filter for the server name. + +For example, if the server name is `{server-type}/{sub-type}/{name}`, and the client determines through its permissions that it can only access MCP servers of type `{server-type}/{sub-type}`, it can subscribe to `$mcp-server/presence/+/{server-type}/{sub-type}/#`, thereby subscribing to the service presence topic for all MCP servers of the `{sub-type}` type at once. + +Although the client can subscribe to `$mcp-server/presence/+/#` to get all types of MCP servers, the administrator might restrict it through ACL (Access Control List) on the MQTT broker to only send and receive messages on RPC topics like `$mcp-rpc/{mcp-client-id}/{server-id}/{server-type}/{sub-type}/#`. Therefore, subscribing to overly broad topics is not useful. By designing the `{server-name-filter}` appropriately, the client can reduce interference from irrelevant information. + +### Service Unregistration + +When connecting to the MQTT broker, the server must set `$mcp-server/presence/{server-id}/{server-name}` as the will topic, with an empty payload will message, to clear the registration information in case of an unexpected disconnection. + +Before actively disconnecting from the MQTT broker, the server **MUST** send an empty payload message to the `$mcp-server/presence/{server-id}/{server-name}` topic to clear the registration information. + +On the `$mcp-server/presence/{server-id}/{server-name}` topic: + +- When the client receives a `server/online` notification, it should record the `{server-id}` as one of the instances of that `{server-name}`. +- When the client receives an empty payload message, it should clear the cached `{server-id}`. As long as any instance of that `{server-name}` is online, the client should consider the MCP server to be online. + +The message flow for service registration and unregistration is as follows: + +```mermaid +sequenceDiagram + participant MCP_Server as MCP Server + participant MQTT_Broker as MQTT Broker + participant MCP_Client as MCP Client + + MCP_Server ->> MQTT_Broker: Register Service
Topic: $mcp-server/presence/{server-id}/{server-name}
Retain: True + Note right of MQTT_Broker: Store Retained Messages + + MCP_Client ->> MQTT_Broker: Subscribe Services
Topic Filter: $mcp-server/presence/+/ {server-name-filter} + + MQTT_Broker ->> MCP_Client: Description of Service
Topic: $mcp-server/presence/{server-id}/{server-name}
Payload: "notifications/server/online" + Note left of MCP_Client: Record the server-id for a server-name. + + MCP_Server ->> MQTT_Broker: Unregister Service
Topic: $mcp-server/presence/{server-id}/{server-name}
Retain: True
Payload: Empty + Note right of MQTT_Broker: Clean Retained Messages + + MQTT_Broker ->> MCP_Client: Description of Service
Topic: $mcp-server/presence/{server-id}/{server-name}
Payload: Empty + Note left of MCP_Client: Remove the server-id +``` + +## Initialization + +This section only describes the MQTT transport specific parts of the initialization phase, please see [Lifecycle](https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle#initialization) for more details. + +The initialization phase **MUST** be the first interaction between client and server. + +The client **MUST** subscribe to the RPC topic (`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`) before sending the initialization request, with the **No Local** subscription option. + +The server **MUST** subscribe to the RPC topic (`$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`) before responding to the initialization response, with the **No Local** subscription option. + +```mermaid +sequenceDiagram + participant MCP_Client as MCP Client + participant MCP_Server as MCP Server + + Note right of MCP_Client: Subscribe the
server's RPC topic + MCP_Client ->> MCP_Server: Initialize Request
Topic: $mcp-server/{server-id}/{server-name} + Note left of MCP_Server: Subscribe the
client's RPC topic + MCP_Server ->> MCP_Client: Initialize Response
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + MCP_Client ->> MCP_Server: Initialized
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + MCP_Client ->> MCP_Server: RPC Req/Resp/Notification
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + MCP_Server ->> MCP_Client: RPC Req/Resp/Notification
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} +``` + +The client **MUST** initiate this phase by sending an `initialize` request to the topic `$mcp-server/{server-id}/{server-name}` containing: + +- Protocol version supported +- Client capabilities +- Client implementation information + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "method": "initialize", + "params": { + "protocolVersion": "2024-11-05", + "capabilities": { + "roots": { + "listChanged": true + }, + "sampling": {} + }, + "clientInfo": { + "name": "ExampleClient", + "version": "1.0.0" + } + } +} +``` + +The server **MUST** respond with its own capabilities to the topic `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` and information: + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "result": { + "protocolVersion": "2024-11-05", + "capabilities": { + "logging": {}, + "prompts": { + "listChanged": true + }, + "resources": { + "subscribe": true, + "listChanged": true + }, + "tools": { + "listChanged": true + } + }, + "serverInfo": { + "name": "ExampleServer", + "version": "1.0.0" + } + } +} +``` + +After successful initialization, the client **MUST** send an initialized notification to indicate it is ready to begin normal operations, to the topic `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}`: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/initialized" +} +``` + +## Capability List Changed + +Before initiating the Initialize request, the MCP client **MUST** subscribe to the MCP server's capability list changed topic: `$mcp-server/capability/{server-id}/{server-name-filter}`, where `{server-name-filter}` is a filter for the server name. + +Before the MCP server responds to the initialization request, it **MUST** first subscribe to the MCP client's capability list changed topic: `$mcp-client/capability/{mcp-client-id}`. + +If there are subsequent capability list updates: + +- The server will send a notification to: `$mcp-server/capability/{server-id}/{server-name}` +- The client will send a notification to: `$mcp-client/capability/{mcp-client-id}` + +The payload of the capability list changed notification depends on the specific capability that has changed. For example "notifications/tools/list_changed" for tools. After receiving a capability list change notification, the client or server needs to retrieve the updated capability list. See the specific capability documentation for details. + +```mermaid + +sequenceDiagram + participant MCP_Client as MCP Client + participant MCP_Server as MCP Server + + Note right of MCP_Client: Client subscribes the server's
capability change topic + MCP_Client ->> MCP_Server: Initialize + + Note left of MCP_Server: Server subscribes the client's
capability change topic + MCP_Server ->> MCP_Client: Initialize Response + MCP_Client ->> MCP_Server: Initialized + + MCP_Server -->> MCP_Client: Capability List Changed
Topic: $mcp-server/capability/{server-id}/{server-name} + + MCP_Client ->> MCP_Server: List Capability
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + + MCP_Server -->> MCP_Client: List Capability Response
$mcp-rpc/{mcp-client-id}/{server-id}/{server-name} +``` + +## Resource Update + +The MCP protocol specifies that the client can subscribe to changes of a specific resource. + +If the server provides the capability to subscribe to resources, the client can subscribe to the resource changes before sending the initialized notification. + +The topic for the client to subscribe to resource changes is: `$mcp-server/capability/{server-id}/{server-name}`. + +When a resource changes, the server **SHOULD** send a notification to `$mcp-server/capability/{server-id}/{server-name}`. + +```mermaid + +sequenceDiagram + participant MCP_Client as MCP Client + participant MCP_Server as MCP Server + + MCP_Client ->> MCP_Server: Initialize + MCP_Server ->> MCP_Client: Initialize Response + Note right of MCP_Client: Client subscribes the server's
resource update topic + MCP_Client ->> MCP_Server: Initialized + + MCP_Client ->> MCP_Server: List Resources
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name} + + MCP_Server -->> MCP_Client: List Resources Response
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name}
URIs: [{resource-uri}, {resource-uri}, ...] + + MCP_Server -->> MCP_Client: Resource Updated
Topic: $mcp-server/capability/{server-id}/{server-name}
URI: {resource-uri} + + MCP_Client ->> MCP_Server: Read Resource
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name}
URI: {resource-uri} + + MCP_Server -->> MCP_Client: Read Resource Response
Topic: $mcp-rpc/{mcp-client-id}/{server-id}/{server-name}
URI: {resource-uri} +``` + +## Shutdown + +### Server Disconnect + +The server **MUST** connect with a will message to notify the client when it disconnects unexpectedly, the will topic is `$mcp-server/presence/{server-id}/{server-name}` and the payload is empty. + +Before a MCP server disconnects from the MQTT broker, the server **MUST** send an empty message to the presence topic `$mcp-server/presence/{server-id}/{server-name}`. + +The MCP server may want to 'de-initialize' with a MCP client but still keep the connection with the MQTT broker, in this case it **MUST** send a "disconnected" notification to the rpc topic `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` and then unsubscribe from the following topics: +- `$mcp-client/capability/{mcp-client-id}` +- `$mcp-client/presence/{mcp-client-id}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` + +The message format for the MCP server's "disconnected" notification is: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/disconnected" +} +``` + +When a MCP client receives an empty payload message on the server's presence topic or a "disconnected" notification on the rpc topic, it **MUST** consider the server to be offline and clear the cached `{server-id}` for that `{server-name}` and unsubscribe from the following topics: +- `$mcp-server/capability/{server-id}/{server-name-filter}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}` + +### Client Disconnect + +The server **MUST** subscribe to the client's presence topic (`$mcp-client/presence/{mcp-client-id}`) before sending the initialization response. + +The client **MUST** connect with a will message to notify the server when it disconnects unexpectedly, the will topic is `$mcp-client/presence/{mcp-client-id}` and the payload is a "disconnected" notification. + +Before the client disconnects from the MQTT broker, it **MUST** send a "disconnected" notification to the topic `$mcp-client/presence/{mcp-client-id}`. + +The client may want to 'de-initialize' with a MCP server but still keep the connection with the MQTT broker, in this case it **MUST** send a "disconnected" notification to the rpc topic `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` and then unsubscribe from the following topics: +- `$mcp-server/capability/{server-id}/{server-name-filter}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name-filter}` + +After the MCP server receives the "disconnected" notification, it **MUST** unsubscribe from the following topics: +- `$mcp-client/capability/{mcp-client-id}` +- `$mcp-client/presence/{mcp-client-id}` +- `$mcp-rpc/{mcp-client-id}/{server-id}/{server-name}` + +The message format for the MCP client's "disconnected" notification is: + +```json +{ + "jsonrpc": "2.0", + "method": "notifications/disconnected" +} +``` + +## Health Checks + +The client or the server **MAY** send `ping` requests to the server at any time to check the health of their counterpart. + +- If the client does not receive a `ping` response from the server within a reasonable time, it **MUST** send a "disconnected" notification to the topic `$mcp-client/presence/{mcp-client-id}` and disconnect itself. +- If the server does not receive a `ping` response from the client within a reasonable time, it **MUST** send any other PRC requests to the client. + +For more information, see [Ping](https://modelcontextprotocol.io/specification/2025-06-18/basic/utilities/ping). + +## Timeouts + +All RPC requests are sent asynchronously via MQTT messages, so timeout issues need to be considered. The timeout duration may vary for different RPC requests, but it should be configurable. + +Below are the recommended default timeout values for each type of RPC request in this protocol: + +- "initialize": 30 seconds +- "ping": 10 seconds +- "roots/list": 30 seconds +- "resources/list": 30 seconds +- "tools/list": 30 seconds +- "prompts/list": 30 seconds +- "prompts/get": 30 seconds +- "sampling/createMessage": 60 seconds +- "resources/read": 30 seconds +- "resources/templates/list": 30 seconds +- "resources/subscribe": 30 seconds +- "tools/call": 60 seconds +- "completion/complete": 60 seconds +- "logging/setLevel": 30 seconds + + + +## Error Handling + +Implementations **SHOULD** be prepared to handle these error cases: + +- Protocol version mismatch +- Failure to negotiate required capabilities +- Initialize request timeout +- Shutdown timeout + +Implementations **SHOULD** implement appropriate timeouts for all requests, to prevent +hung connections and resource exhaustion. + +Example initialization error: + +```json +{ + "jsonrpc": "2.0", + "id": 1, + "error": { + "code": -32602, + "message": "Unsupported protocol version", + "data": { + "supported": ["2025-03-26"], + "requested": "1.0.0" + } + } +} +``` \ No newline at end of file diff --git a/ja_JP/emqx-ai/multimedia-ai/architecture.md b/ja_JP/emqx-ai/multimedia-ai/architecture.md index e69de29bb..98c535bc8 100644 --- a/ja_JP/emqx-ai/multimedia-ai/architecture.md +++ b/ja_JP/emqx-ai/multimedia-ai/architecture.md @@ -0,0 +1,62 @@ +# Multimedia Server Architecture + +The EMQX Multimedia Server uses WebRTC to establish peer-to-peer audio and video data transmission with clients, while MQTT is used to carry general messages and WebRTC signaling. + +Developers can build AI agent programs in Python, which communicate with the Multimedia Server via Standard Input/Output (STDIO), enabling more sophisticated business logic and AI-driven workflows. + +The following diagram illustrates the overall architecture of an AI system built with the Multimedia Server: + +```mermaid +flowchart LR + Device <-- WebRTC --> MediaServer + MediaServer <-- STDIO --> AIAgent + MediaServer <--> ASRTTS + AIAgent <--> LLM + + Device[Device] + MediaServer[Multimedia Server] + AIAgent[AI Agent] + ASRTTS[ASR/TTS] + LLM[LLM] +``` + +## Components + +- **Device**: Exchanges audio and video data with the Multimedia Server via WebRTC. +- **Multimedia Server**: Processes audio and video streams from devices, provides Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) services, and communicates with the AI Agent. +- **AI Agent**: Receives ASR results from the Multimedia Server over STDIO. It encapsulates the core business logic of the AI application, calls the Large Language Model (LLM) for natural language understanding, and uses Multimedia Server APIs to send text or audio streams back to devices. +- **ASR/TTS**: External AI service providers that offer speech recognition and text-to-speech capabilities. +- **LLM**: Large Language Model, responsible for natural language processing and text generation. + +## Workflow + +The diagram below shows how the components interact in a typical workflow: + +```mermaid +sequenceDiagram + actor Device + participant Media as Multimedia Server + participant Agent as AI Agent + participant LLM + + Device ->>+ Media: WebRTC Audio + Media ->> Media: Run ASR + Media ->> Agent: ASR Result + Agent ->> LLM: Process Input with MCP Tools + LLM ->> Agent: LLM Output + Agent ->> Agent: Apply Business Logic + Agent ->> Media: TTS Request + Media ->> Media: Run TTS + Media ->> Device: WebRTC Audio + Device ->> Media: WebRTC Video + Agent ->> Media: Image Analysis Request + Media ->> Media: Run Image Analysis + Media ->> Agent: Image Analysis Result + Agent ->> LLM: Summarize Analysis Result + LLM ->> Agent: Summary + Agent ->> Media: TTS Request + Media ->> Device: WebRTC Audio + Agent ->> Agent: Additional Processing + Agent ->> Media: Send Control Message + Media ->> Device: MQTT Message +``` diff --git a/ja_JP/emqx-ai/multimedia-ai/message-protocol.md b/ja_JP/emqx-ai/multimedia-ai/message-protocol.md index e69de29bb..457d6e407 100644 --- a/ja_JP/emqx-ai/multimedia-ai/message-protocol.md +++ b/ja_JP/emqx-ai/multimedia-ai/message-protocol.md @@ -0,0 +1,378 @@ +# Multimedia AI Messaging Protocol + +This document describes the message protocol used for interaction between the multimedia server, clients (devices), and AI agents. + +## WebRTC Signaling via MQTT + +After establishing the MQTT connection, the client needs to use the following MQTT topic to set up the WebRTC connection: + +- `$webrtc//multimedia_proxy`: The MQTT topic for signaling messages between the client and the multimedia proxy for WebRTC connection setup. The client should subscribe to this topic to receive signaling messages from the multimedia proxy. + +- `$webrtc/`: The MQTT topic for the device to receive signaling messages. + +The client should send `offer` and `candidate` messages to the `$webrtc//multimedia_proxy` topic and wait for `answer` and `candidate` messages from the multimedia proxy on the `$webrtc/` topic to establish the WebRTC connection. + +The format of the signaling messages for setting up the WebRTC connections: + +```json +{ + "type": "sdp_offer", + "data": { + "sdp": , + "type": "offer" + } +} +``` + +```json +{ + "type": "sdp_answer", + "data": { + "sdp": , + "type": "answer" + } +} +``` + +```json +{ + "type": "ice_candidate", + "data": { + "candidate": , + "sdpMid": , + "sdpMLineIndex": , + "usernameFragment": + } +} +``` + +The `data` field above can be generated using the [RTCPeerConnection API](https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection) in the browser, for example: + +```javascript +// Create an offer +const offer = await pc.createOffer(); +await pc.setLocalDescription(offer); +const message = { + type: "sdp_offer", + data: offer +}; +// Send the message to the multimedia proxy via MQTT +mqttClient.publish(`$webrtc/${deviceId}/multimedia_proxy`, JSON.stringify(message)); +``` + +```javascript +// Handle the answer from the multimedia proxy +mqttClient.on('message', (topic, message) => { + const msg = JSON.parse(message.toString()); + if (msg.type === 'sdp_answer') { + const answer = msg.data; + pc.setRemoteDescription(new RTCSessionDescription(answer)); + } else if (msg.type === 'ice_candidate') { + const candidate = new RTCIceCandidate(msg.data); + pc.addIceCandidate(candidate); + } +}); +``` + +The multimedia proxy will send the `webrtc_terminated` message to the client when the WebRTC connection is terminated: + +```json +{ + "type": "webrtc_terminated", + "reason": "reason for termination" +} +``` + + +## Send General Messages via MQTT + +The multimedia server and devices exchange general messages through the following MQTT topics: + +- **`$message/`**: Topic for the multimedia server to send general messages to a device. +- **`$message//multimedia_proxy`**: Topic for a device to send arbitrary messages to the multimedia server. These messages are forwarded to the AI Agent via the `message_from_device` method. + +### Messages Sent from Multimedia Server to Devices + +The multimedia server can publish the following message types on the `$message/` topic: + +A `asr_response` message is sent when ASR results are available: + +```json +{ + "type": "asr_response", + "format": "merged" | "raw", + "results": +} +``` + +A `tts_begin` message is sent when a TTS task is started: + +```json +{ + "type": "tts_begin", + "task_id": "task_id" +} +``` + +A `tts_text` message is sent when a text is converted to speech and the text should also be sent to the device: + +```json +{ + "type": "tts_text", + "task_id": "task_id", + "text": "text" +} +``` + +A `tts_complete` message is sent when the TTS task is completed: + +```json +{ + "type": "tts_complete", + "task_id": "task_id" +} +``` + +A `tts_terminate` message is sent when the TTS task is finished or terminated: + +```json +{ + "type": "tts_terminate", + "task_id": "task_id" +} +``` + +A `message` message is sent to device when the agent sends an arbitrary message to the device (by the `message_to_device` method): + +```json +{ + "type": "message", + "payload": +} +``` + +### Messages Sent from Devices to the Multimedia Server + +Devices can publish arbitrary messages to the **`$message//multimedia_proxy`** topic. + The multimedia server will forward these messages to the AI Agent: + +```json +{ + "type": "message", + "payload": +} +``` + + +## Interaction Protocol between Multimedia Server and AI Agent + +Using AI agents can can extend the capabilities of the Multimedia Server, for example by processing ASR results according to business logic or sending custom messages to devices. + +The multimedia proxy interacts with AI agents using a simple JSON RPC 2.0 based protocol. The messages are sent over Standard Input/Output (STDIO). Messages are delimited by newlines (`\n`), and MUST NOT contain embedded newlines. + +- **Initialization**: + After the STDIO connection is established, the agent must send an initialization message to the multimedia proxy, to negotiate the protocol version and configuration: + + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "method": "init", + "params": { + "protocol_version": "1.0", + "configs": { + "asr": { + // If enabled, multimedia proxy will send merged ASR text (based on the timestamps of the sentences) every time a new ASR result is available, otherwise it is the agent's responsibility to merge the ASR results. + "auto_merge": false + } + } + } + } + ``` + + The multimedia proxy will respond with an acknowledgment: + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "result": "ok" + } + ``` + +- **ASR Result**: + The multimedia proxy sends the ASR results as notifications to the AI agents in the following format: + ```json + { + "jsonrpc": "2.0", + "method": "asr_result", + "params": { + // The current device ID + "device_id": "device_id", + "text": "Recognized text" + } + } + ``` + +- **TTS and Send**: + + The AI Agent can request the Multimedia Server to perform **TTS** and send the audio to a target device. + + 1. The agent sends a `tts_and_send_start` message to initiate the task. + 2. The agent sends one or more `tts_and_send` messages to provide the text to be synthesized. + - Multiple texts for the same task can be sent in batches or sequentially, but must use the same `task_id`. + 3. Finally, the agent sends a `tts_and_send_finish` message to signal the end of the task. + + The start message: + ```json + { + "jsonrpc": "2.0", + "id": "3", + "method": "tts_and_send_start", + "params": { + // The deivce ID to send the audio to + "device_id": "device_id", + // + "task_id": "aaa", + "text": "Text to be converted to speech" + } + } + ``` + + The texts to be converted to speech can be sent in one batch: + ```json + [ + { + "jsonrpc": "2.0", + "id": "4", + "method": "tts_and_send", + "params": { + // The deivce ID to send the audio to + "device_id": "device_id", + // + "task_id": "aaa", + "text": "Text to be converted to speech" + } + }, + { + "jsonrpc": "2.0", + "id": "5", + "method": "tts_and_send", + "params": { + // The deivce ID to send the audio to + "device_id": "device_id", + // + "task_id": "aaa", + "text": ", and more text can be send in one batch" + } + }, + { + "jsonrpc": "2.0", + "id": "6", + "method": "tts_and_send_finish", + "params": { + // The device ID to send the audio to + "device_id": "device_id", + // The task ID of the TTS task + "task_id": "aaa" + } + } + ] + ``` + + The `tts_and_send_start` and `tts_and_send_finish` messages can be sent either in the same batch as `tts_and_send` messages or separately. + + The Multimedia Server confirms each message with `"ok"` or returns an error: + + ```json + [ + { + "jsonrpc": "2.0", + "id": "4", + "result": "ok" + }, + { + "jsonrpc": "2.0", + "id": "5", + "result": "ok" + }, + { + "jsonrpc": "2.0", + "id": "6", + "result": "ok" + } + ] + ``` + +- **Image Analysis**: + The AI agents can request the multimedia proxy to perform image analysis: + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "method": "image_analysis", + "params": { + // The ID of the device to capture images from + "device_id": "device_id", + // The count of images to capture and analyze + "image_count": 2, + "capture_interval": 1000, // Interval in milliseconds between captures + "image_format": "jpeg", // Format of the captured images + "user_prompt": "Analyze the images and provide insights" + } + } + ``` + + The multimedia proxy will respond with the analysis results: + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "result": { + "analysis_result": "Analysis result" + } + } + ``` + +- **Forward Messages Received from Device**: + The Multimedia Server forwards messages received on the `$message//multimedia_proxy` topic to the AI Agent using the `message_from_device` method: + + ```json + { + "jsonrpc": "2.0", + "method": "message_from_device", + "params": { + // The ID of the device that sent the message + "device_id": "device_id", + "payload": "payload" + } + } + ``` + +- **Send Message to Device**: + The AI Agent can send arbitrary messages to devices through the Multimedia Server: + + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "method": "message_to_device", + "params": { + // The ID of the device to send the message to + // The message will be sent to the device via the `$message/` MQTT topic + "device_id": "device_id", + // Or you can specify the topic manually to send message to any device + // If specified, the `device_id` field will be ignored + "topic": "topic/to/device", + "payload": "payload" + } + } + ``` + + The Multimedia Server responds with a confirmation: + ```json + { + "jsonrpc": "2.0", + "id": "unique_id", + "result": "ok" + } + ``` diff --git a/ja_JP/emqx-ai/multimedia-ai/overview.md b/ja_JP/emqx-ai/multimedia-ai/overview.md index e69de29bb..d543bea2d 100644 --- a/ja_JP/emqx-ai/multimedia-ai/overview.md +++ b/ja_JP/emqx-ai/multimedia-ai/overview.md @@ -0,0 +1,33 @@ +# EMQX Multimedia Server + +The EMQX Multimedia Server is a high-performance audio and video processing platform built on WebRTC technology. It can receive RTP/SRTP audio and video streams from clients and integrates multiple AI capabilities, including Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and Image Understanding. By leveraging large language models (LLMs), the EMQX Multimedia Server enables advanced voice interactions and tool invocation, providing robust technical support for AI applications that require audio and video capabilities. + +## Key Features + +- **Real-time Audio and Video Processing** + Supports high-quality audio and video streaming with low latency and high reliability. +- **Automatic Speech Recognition (ASR)** + Converts speech to text with high accuracy, suitable for voice assistants, intelligent customer service, and more. +- **Text-to-Speech (TTS)** + Generates natural-sounding speech in multiple languages and voice styles, enhancing interactive experiences. +- **Image Understanding** + Integrates image recognition and analysis to support diverse visual processing tasks such as object detection and scene analysis. +- **LLM Integration** + Harnesses large model capabilities to enable complex voice conversations and tool invocation, meeting diverse business needs. +- **Flexible Architecture** + Designed for scalability and customization, supporting horizontal expansion. Developers can flexibly integrate services from different providers, including TTS, ASR, image processing, and LLMs. +- **High Reliability** + Built on a distributed architecture to ensure high availability and system stability under large-scale workloads. +- **Low Latency** + Optimized network transmission and model processing pipelines deliver smooth real-time interactions. + +## Use Cases + +The EMQX Multimedia Server can be applied to a wide range of AI-driven scenarios, including: + +- **Emotional Companionship** + Provides personalized companion services through conversational AI and emotion recognition technologies. +- **Intelligent Customer Service** + Enables efficient voice-based interactions with ASR and TTS, improving customer service quality. +- **Smart Device Control** + Facilitates intuitive device control through voice commands and image recognition. \ No newline at end of file diff --git a/ja_JP/emqx-ai/overview.md b/ja_JP/emqx-ai/overview.md index e69de29bb..45587a24a 100644 --- a/ja_JP/emqx-ai/overview.md +++ b/ja_JP/emqx-ai/overview.md @@ -0,0 +1,9 @@ +# EMQX AI + +EMQX AI is EMQX's innovative practice in the field of artificial intelligence, dedicated to providing efficient and reliable communication infrastructure for AI applications. It consists of the following two aspects: + +- **Message Communication:** Through the MCP over MQTT protocol, EMQX offers lightweight, low-latency tool invocation and message transmission for AI applications, meeting communication needs in scenarios such as model inference, real-time feedback, intelligent data collection, and intelligent control. See [MCP over MQTT](./mcp-over-mqtt/overview.md) for details. + +- **Multimedia Streaming:** Based on a WebRTC-powered multimedia server, EMQX provides real-time audio and video streaming capabilities for AI applications, suitable for scenarios like intelligent voice, video analysis, image understanding, and remote collaboration. See [Multimedia AI](./multimedia-ai/overview.md) for details. + +By combining these two solutions, EMQX builds a solid communication foundation for AI applications, helping developers quickly create intelligent and real-time AI solutions. diff --git a/ja_JP/emqx-ai/sdks/mcp-sdk-erlang.md b/ja_JP/emqx-ai/sdks/mcp-sdk-erlang.md index e69de29bb..06048b30c 100644 --- a/ja_JP/emqx-ai/sdks/mcp-sdk-erlang.md +++ b/ja_JP/emqx-ai/sdks/mcp-sdk-erlang.md @@ -0,0 +1,189 @@ +# Erlang SDK + +This document demonstrates how to use the [MCP over MQTT Erlang SDK](https://github.com/emqx/mcp-mqtt-erl) to create a simple MCP over MQTT server and client. + +## Example + +### Create a Simple MCP Client + +```erlang +-module(mcp_mqtt_erl_client_demo). + +-behaviour(mcp_mqtt_erl_client_session). + +-export([ + client_name/0, + client_version/0, + client_capabilities/0, + received_non_mcp_message/3 +]). +-export([start_link/0]). + +%% The client name, version, and capabilities. These details will be sent to the server during MCP initialization. +client_name() -> + <<"emqx_tools/cli_demo">>. + +client_version() -> + <<"1.0">>. + +client_capabilities() -> #{}. + +%% Callback for non-MCP messages +received_non_mcp_message(MqttClient, Msg, State) -> + io:format("~p Received non-MCP message: ~p~n", [MqttClient, Msg]), + State. + +%% Start the MCP over MQTT client +start_link() -> + mcp_mqtt_erl_client:start_link( + #{ + server_name_filter => <<"#">>, + callback_mod => ?MODULE, + broker_address => {"127.0.0.1", 1883}, + mqtt_options => #{ + clientid => <<"emqx_tools_cli_demo">>, + username => <<"emqx">>, + password => <<"public">> + } + }). +``` + +Here, `server_name_filter` is used to subscribe to the MQTT topic filter for MCP servers, and `mqtt_options` are options passed to the underlying MQTT client. + +### Create a Simple MCP Server + +Below is a simple MCP server implementation that supports two tools: `tool1` and `tool2`. + +```erlang +-module(mcp_mqtt_erl_server_demo). + +-behaviour(mcp_mqtt_erl_server_session). + +-include_lib("mcp_mqtt_erl/include/mcp_mqtt_erl_types.hrl"). +-include_lib("mcp_mqtt_erl/include/emqx_mcp_tools.hrl"). + +-export([ + start_link/2 +]). + +-export([ + server_name/0, + server_id/2, + server_version/0, + server_capabilities/0, + server_instructions/0, + server_meta/0 +]). + +-export([ + initialize/2, + list_resources/1, + read_resource/2, + call_tool/3, + list_tools/1 +]). + +-type loop_data() :: #{ + server_id => binary(), + client_info => map(), + client_capabilities => map(), + mcp_client_id => binary(), + _ => any() +}. + +-spec start_link(integer(), mcp_mqtt_erl_server:config()) -> gen_statem:start_ret(). +start_link(Idx, Conf) -> + mcp_mqtt_erl_server:start_link(Idx, Conf). + +server_version() -> + <>. + +server_name() -> + <<"emqx_tools/info_apis">>. + +server_id(ClientIdPrefix, Idx) -> + Idx1 = integer_to_binary(Idx), + Node = atom_to_binary(node()), + <>. + +server_capabilities() -> + #{ + resources => #{ + subscribe => true, + listChanged => true + }, + tools => #{ + listChanged => true + } + }. + +server_instructions() -> + <<"">>. + +server_meta() -> + #{ + authorization => #{ + roles => [<<"admin">>, <<"user">>] + } + }. + +-spec initialize(binary(), client_params()) -> {ok, loop_data()}. +initialize(ServerId, #{client_info := ClientInfo, client_capabilities := Capabilities, mcp_client_id := McpClientId}) -> + io:format("initialize --- server_id: ~p, client_info: ~p, client_capabilities: ~p, mcp_client_id: ~p~n", [ServerId, ClientInfo, Capabilities, McpClientId]), + {ok, #{ + server_id => ServerId, + client_info => ClientInfo, + client_capabilities => Capabilities, + mcp_client_id => McpClientId + }}. + +-spec call_tool(binary(), map(), loop_data()) -> {ok, call_tool_result() | [call_tool_result()], loop_data()}. +call_tool(ToolName, Args, LoopData) -> + io:format("call_tool --- tool_name: ~p, args: ~p~n", [ToolName, Args]), + Result = #{ + type => text, + text => <<"This is the result of the tool call">> + }, + {ok, Result, LoopData}. + +-spec list_tools(loop_data()) -> {ok, [tool_def()], loop_data()}. +list_tools(LoopData) -> + io:format("list_tools --- ~n", []), + Tools = [ + #{ + name => <<"tool1">>, + description => <<"This is tool 1">>, + inputSchema => #{ + type => <<"object">>, + properties => #{ + arg1 => #{ + type => <<"string">>, + description => <<"Argument 1">> + }, + arg2 => #{ + type => <<"integer">>, + description => <<"Argument 2">> + } + } + } + }, + #{ + name => <<"tool2">>, + description => <<"This is tool 2">>, + inputSchema => #{ + type => <<"object">>, + properties => #{ + arg1 => #{ + type => <<"string">>, + description => <<"Argument 1">> + }, + arg2 => #{ + type => <<"boolean">>, + description => <<"Argument 2">> + } + } + } + } + ], + {ok, Tools, LoopData}. +``` diff --git a/ja_JP/emqx-ai/sdks/mcp-sdk-esp32-c.md b/ja_JP/emqx-ai/sdks/mcp-sdk-esp32-c.md index e69de29bb..b16694429 100644 --- a/ja_JP/emqx-ai/sdks/mcp-sdk-esp32-c.md +++ b/ja_JP/emqx-ai/sdks/mcp-sdk-esp32-c.md @@ -0,0 +1,59 @@ +# ESP32 C SDK + +This guide demonstrates how to use the [MCP over MQTT C SDK for ESP32](https://github.com/mqtt-ai/esp-mcp-over-mqtt) to create a simple MCP over MQTT server. + Currently, only the MCP server is supported. You can create an MCP client using the Python SDK for interaction. + +The SDK uses the MQTT library included in ESP-IDF, making it suitable for ESP32 devices. Therefore, it must be used within the ESP-IDF environment. + +## Create an MCP Server + +Following the instructions in the [ESP32 C SDK README](https://github.com/mqtt-ai/esp-mcp-over-mqtt), create a new file named `mcp_server_example.c` in your ESP-IDF project and add the following code: + +```c +#include "mcp_server.h" + +const char* get_temperature_callback(int n_args, property_t *args) { + // Read sensor data + float temp = read_temperature_sensor(); + + // Return JSON formatted result + static char result[64]; + snprintf(result, sizeof(result), "{\"temperature\": %.2f}", temp); + return result; +} + +// Define MCP tools +mcp_tool_t my_tools[] = { + { + .name = "get_temperature", + .description = "Get device temperature", + .property_count = 0, + .properties = NULL, + .call = get_temperature_callback + } +}; + +// Initialize MCP server +mcp_server_t *server = mcp_server_init( + "esp32_sensor", // Server name + "ESP32 Sensor MCP Server", // Description + "mqtt://broker.example.com",// MQTT Broker URI + "esp32_client_001", // Client ID + "username", // Username + "password", // Password + NULL // Certificate (optional) +); + +// Register tools +mcp_server_register_tool(server, 1, my_tools); + +// Start server +mcp_server_run(server); +``` + +## Use MCP Server in an ESP-IDF Project + +For detailed usage, see the [ESP32 MCP Demo](https://github.com/mqtt-ai/esp32-mcp-mqtt-tutorial/tree/main/samples/blog_3) project. + This example demonstrates how to integrate the MCP over MQTT C SDK for ESP32 into an ESP-IDF project, set up an MCP server, and interact with it using an MCP client implemented in the Python SDK. + +After building the project with ESP-IDF and flashing it onto the ESP32 device, the MCP server will start automatically. \ No newline at end of file diff --git a/ja_JP/emqx-ai/sdks/mcp-sdk-paho-c.md b/ja_JP/emqx-ai/sdks/mcp-sdk-paho-c.md index e69de29bb..f0a99587d 100644 --- a/ja_JP/emqx-ai/sdks/mcp-sdk-paho-c.md +++ b/ja_JP/emqx-ai/sdks/mcp-sdk-paho-c.md @@ -0,0 +1,65 @@ +# C SDK with Paho MQTT + +This guide demonstrates how to use the [MCP over MQTT C SDK with Paho MQTT](https://github.com/mqtt-ai/paho-mcp-over-mqtt) to create a simple **MCP over MQTT server**. + Currently, only the MCP server is supported. You can use the Python SDK to create an MCP client for interaction. + +## Create an MCP Server + +Following the instructions in the [C SDK with Paho MQTT README](https://github.com/mqtt-ai/paho-mcp-over-mqtt), after installing the dependencies and SDK, create a file named `demo_mcp_server.c` and add the following code: + +```c +#include "mcp_server.h" + +// Callback function to get temperature +const char* get_temperature_callback(int n_args, property_t *args) { + // Read sensor data + float temp = read_temperature_sensor(); + + // Return JSON formatted result + static char result[64]; + snprintf(result, sizeof(result), "{\"temperature\": %.2f}", temp); + return result; +} + +// Define MCP tools +mcp_tool_t my_tools[] = { + { + .name = "get_temperature", + .description = "Get device temperature", + .property_count = 0, + .properties = NULL, + .call = get_temperature_callback + } +}; + +// Initialize MCP server +mcp_server_t *server = mcp_server_init( + "sensor", // Server name + "Sensor MCP Server", // Description + "mqtt://broker.example.com", // MQTT Broker URI + "client_001", // Client ID + "username", // Username + "password", // Password + NULL // Certificate (optional) +); + +// Register tools +mcp_server_register_tool(server, 1, my_tools); + +// Start server +mcp_server_run(server); +``` + +## Create and Run an MCP Client with Python SDK + +Refer to the [Python SDK](./mcp-sdk-python.md) documentation to create an MCP client that connects to the MCP server created above and invokes the `get_temperature` tool. + +## Build and Compile the MCP Server with CMake + +You can follow the **CMake examples** provided in the [paho-mcp-over-mqtt repository](https://github.com/mqtt-ai/paho-mcp-over-mqtt). + +After building, run the generated executable to start the MCP server: + +```bash +./demo_mcp_server +``` \ No newline at end of file diff --git a/ja_JP/emqx-ai/sdks/mcp-sdk-python.md b/ja_JP/emqx-ai/sdks/mcp-sdk-python.md index e69de29bb..3795c53d3 100644 --- a/ja_JP/emqx-ai/sdks/mcp-sdk-python.md +++ b/ja_JP/emqx-ai/sdks/mcp-sdk-python.md @@ -0,0 +1,130 @@ +# Python SDK + +This guide demonstrates how to use the [MCP over MQTT Python SDK](https://github.com/emqx/mcp-python-sdk) to create a simple MCP over MQTT server and client. + +## Create a Demo Project + +Let's use [uv](https://docs.astral.sh/uv/) to create a demo project: + +```bash +uv init mcp_over_mqtt_demo +cd mcp_over_mqtt_demo +``` + +## Create a Simple MCP Server + +In the `mcp_over_mqtt_demo` project, create a simple MCP server that exposes a calculator tool and some resources. Create a file named `demo_mcp_server.py` and add the following code: + +```python +# demo_mcp_server.py +from mcp.server.fastmcp import FastMCP + +# Create an MCP server +mcp = FastMCP( + "demo_mcp_server/calculator", + log_level="DEBUG", + mqtt_server_description="A simple FastMCP server that exposes a calculator tool", + mqtt_options={ + "host": "broker.emqx.io", + }, +) + +# Add an addition tool +@mcp.tool() +def add(a: int, b: int) -> int: + """Add two numbers""" + return a + b + +# Add a dynamic greeting resource +@mcp.resource("greeting://{name}") +def get_greeting(name: str) -> str: + """Get a personalized greeting""" + return f"Hello, {name}!" +``` + +## Create a Simple MCP Client + +In the same project, create a simple MCP client that connects to the server and lists available tools and resources. Create a file named `demo_mcp_client.py` and add the following code: + +```python +# demo_mcp_client.py +import logging +import anyio +import mcp.client.mqtt as mcp_mqtt +from mcp.shared.mqtt import configure_logging + +configure_logging(level="INFO") +logger = logging.getLogger(__name__) + +async def on_mcp_server_discovered(client, server_name): + logger.info(f"Discovered {server_name}, connecting ...") + await client.initialize_mcp_server(server_name) + +async def on_mcp_connect(client, server_name, connect_result): + success, init_result = connect_result + if success == 'error': + logger.error(f"Failed to connect to {server_name}: {init_result}") + return + logger.info(f"Connected to {server_name}, success={success}, init_result={init_result}") + capabilities = init_result.capabilities + if capabilities.prompts: + prompts = await client.list_prompts(server_name) + logger.info(f"Prompts of {server_name}: {prompts}") + if capabilities.resources: + resources = await client.list_resources(server_name) + logger.info(f"Resources of {server_name}: {resources}") + resource_templates = await client.list_resource_templates(server_name) + logger.info(f"Resources templates of {server_name}: {resource_templates}") + if capabilities.tools: + toolsResult = await client.list_tools(server_name) + tools = toolsResult.tools + logger.info(f"Tools of {server_name}: {tools}") + if tools[0].name == "add": + result = await client.call_tool(server_name, name = tools[0].name, arguments={"a": 1, "b": 2}) + logger.info(f"Calling the tool as add(a=1, b=2), result: {result}") + +async def on_mcp_disconnect(client, server_name): + logger.info(f"Disconnected from {server_name}") + +async def main(): + async with mcp_mqtt.MqttTransportClient( + "test_client", + auto_connect_to_mcp_server = True, + on_mcp_server_discovered = on_mcp_server_discovered, + on_mcp_connect = on_mcp_connect, + on_mcp_disconnect = on_mcp_disconnect, + mqtt_options = mcp_mqtt.MqttOptions( + host="broker.emqx.io", + ) + ) as client: + client.start() + while True: + ## Simulate other works while the MQTT transport client is running in the background... + await anyio.sleep(20) + +if __name__ == "__main__": + anyio.run(main) +``` + +## Run the Demo + +1. Install the required dependencies: + +```bash +uv add git+https://github.com/emqx/mcp-python-sdk --branch main +uv add "mcp[cli]" +``` + +2. Now run the client: + +```bash +uv run demo_mcp_client.py +``` + +3. Open a new terminal and run the server: + +```bash +uv run mcp run --transport mqtt ./demo_mcp_server.py +``` + +Even if the client starts before the server, it will discover the server and connect to it. The client will list available tools and resources, and call the `add` tool with parameters `a=1` and `b=2`. diff --git a/ja_JP/emqx-ai/sdks/mcp-sdk-typescript.md b/ja_JP/emqx-ai/sdks/mcp-sdk-typescript.md index e69de29bb..dcf18fbd9 100644 --- a/ja_JP/emqx-ai/sdks/mcp-sdk-typescript.md +++ b/ja_JP/emqx-ai/sdks/mcp-sdk-typescript.md @@ -0,0 +1,383 @@ +# TypeScript SDK + +This guide demonstrates how to use [@emqx-ai/mcp-mqtt-sdk](https://github.com/emqx/mcp-typescript-sdk) to create an MCP over MQTT server and client. + The SDK supports both browser and Node.js environments and provides complete TypeScript type safety. + +For convenience, this tutorial runs the demo in a Node.js environment. However, you can easily integrate it into a browser environment and use it with frameworks like Vue or React. + +## Create a Demo Project + +First, create a new Node.js project (Node.js >= 18 required): + +```bash +mkdir mcp_typescript_demo +cd mcp_typescript_demo +npm init -y +``` + +## Install Dependencies + +Install the TypeScript MCP SDK: + +```bash +# Using npm +npm install @emqx-ai/mcp-mqtt-sdk +npm install -D typescript @types/node ts-node + +# Or using yarn +yarn add @emqx-ai/mcp-mqtt-sdk +yarn add -D typescript @types/node ts-node + +# Or using pnpm +pnpm add @emqx-ai/mcp-mqtt-sdk +pnpm add -D typescript @types/node ts-node +``` + +## Create a Simple MCP Server + +In the `mcp_typescript_demo` project, create a simple MCP server exposing calculator tools and resources. + Create a file named `demo_mcp_server.ts` and add the following code: + +```typescript +// demo_mcp_server.ts +import { McpMqttServer } from "@emqx-ai/mcp-mqtt-sdk"; + +// Create MCP server +const server = new McpMqttServer({ + host: "mqtt://broker.emqx.io:1883", + serverId: "demo-calculator-server", + serverName: "demo_server/calculator", + name: "Calculator MCP Server", + version: "1.0.0", + description: "A simple calculator MCP server", + capabilities: { + tools: { listChanged: true }, + resources: { listChanged: true }, + }, +}); + +// Add addition tool +server.tool( + "add", + "Add two numbers", + { + type: "object", + properties: { + a: { type: "number", description: "The first number" }, + b: { type: "number", description: "The second number" }, + }, + required: ["a", "b"], + }, + async (params: Record) => { + const { a, b } = params as { a: number; b: number }; + const result = a + b; + return { + content: [ + { + type: "text", + text: `${a} + ${b} = ${result}`, + }, + ], + }; + }, +); + +// Add multiplication tool +server.tool( + "multiply", + "Multiply two numbers", + { + type: "object", + properties: { + a: { type: "number", description: "The first number" }, + b: { type: "number", description: "The second number" }, + }, + required: ["a", "b"], + }, + async (params: Record) => { + const { a, b } = params as { a: number; b: number }; + const result = a * b; + return { + content: [ + { + type: "text", + text: `${a} × ${b} = ${result}`, + }, + ], + }; + }, +); + +// Add personalized greeting resources +const names = ["Alice", "Bob", "Charlie", "Diana", "World"]; +names.forEach((name) => { + server.resource( + `greeting://${name}`, + `Personalized greeting - ${name}`, + async () => { + return { + contents: [ + { + uri: `greeting://${name}`, + mimeType: "text/plain", + text: `Hello, ${name}! Welcome to our calculator service.`, + }, + ], + }; + }, + { + description: `Generate a personalized greeting message for ${name}`, + mimeType: "text/plain", + }, + ); +}); + +// Add server status resource +server.resource( + "status://server", + "Server status", + async () => { + return { + contents: [ + { + uri: "status://server", + mimeType: "application/json", + text: JSON.stringify( + { + name: "Calculator MCP Server", + status: "running", + uptime: process.uptime(), + availableTools: ["add", "multiply"], + timestamp: new Date().toISOString(), + }, + null, + 2, + ), + }, + ], + }; + }, + { + description: "Server runtime status information", + mimeType: "application/json", + }, +); + +// Event handling +server.on("ready", () => { + console.log("Calculator MCP Server started"); +}); + +server.on("error", (error) => { + console.error("Server error:", error); +}); + +// Start server +async function startServer() { + try { + await server.start(); + } catch (error) { + console.error("Failed to start server:", error); + process.exit(1); + } +} + +// Graceful shutdown +process.on("SIGINT", async () => { + console.log("Shutting down server..."); + await server.stop(); + process.exit(0); +}); + +startServer(); +``` + +## Create a Simple MCP Client + +In the same project, create a simple MCP client that connects to the server and lists available tools and resources. +Create a file named `demo_mcp_client.ts` and add the following code: + +```typescript +// demo_mcp_client.ts +import { McpMqttClient } from "@emqx-ai/mcp-mqtt-sdk"; + +// Create MCP client +const client = new McpMqttClient({ + host: "mqtt://broker.emqx.io:1883", + name: "Demo MCP Client", + version: "1.0.0", +}); + +async function onServerDiscovered(server: any) { + console.log(`Discovered server ${server.name}, connecting...`); + await client.initializeServer(server.serverId); +} + +async function onServerConnected(server: any, initResult: any) { + if (!initResult) { + console.error(`Failed to connect to ${server.name}`); + return; + } + + console.log(`Connected to ${server.name}`); + const capabilities = initResult.capabilities; + + // List tools + if (capabilities.tools) { + try { + const tools = await client.listTools(server.serverId); + console.log( + `${server.name} tools:`, + tools.map((t) => t.name), + ); + + // Test addition tool + if (tools.some((t) => t.name === "add")) { + const result = await client.callTool(server.serverId, "add", { + a: 1, + b: 2, + }); + console.log("Result of add(a=1, b=2):", result.content[0]?.text); + } + + // Test multiplication tool + if (tools.some((t) => t.name === "multiply")) { + const result = await client.callTool(server.serverId, "multiply", { + a: 3, + b: 4, + }); + console.log("Result of multiply(a=3, b=4):", result.content[0]?.text); + } + } catch (error) { + console.error("Tool operation error:", error); + } + } + + // List and read resources + if (capabilities.resources) { + try { + const resources = await client.listResources(server.serverId); + console.log( + `${server.name} resources:`, + resources.map((r) => r.uri), + ); + + // Read server status + if (resources.some((r) => r.uri === "status://server")) { + const status = await client.readResource( + server.serverId, + "status://server", + ); + console.log("Server status:", status.contents[0]?.text); + } + + // Read dynamic greeting resource + const greeting = await client.readResource( + server.serverId, + "greeting://Alice", + ); + console.log("Greeting resource:", greeting.contents[0]?.text); + } catch (error) { + console.error("Resource operation error:", error); + } + } +} + +async function onServerDisconnected(serverId: string) { + console.log(`Disconnected from server ${serverId}`); +} + +// Register event handlers +client.on("serverDiscovered", onServerDiscovered); +client.on("serverInitialized", (server) => { + // For demo purposes, mock the initialization result + onServerConnected(server, { capabilities: { tools: true, resources: true } }); +}); +client.on("serverDisconnected", onServerDisconnected); +client.on("error", (error) => { + console.error("Client error:", error); +}); + +// Start client +async function startClient() { + try { + await client.connect(); + console.log("Demo MCP Client started"); + + // Keep running + while (true) { + // Simulate other work while MQTT client runs in the background + await new Promise((resolve) => setTimeout(resolve, 20000)); + } + } catch (error) { + console.error("Failed to start client:", error); + process.exit(1); + } +} + +// Graceful shutdown +process.on("SIGINT", async () => { + console.log("Shutting down client..."); + await client.disconnect(); + process.exit(0); +}); + +startClient(); +``` + +## Configure the Project + +Since the SDK uses ES modules, configure your project to support modern JavaScript module syntax. + +Add module type and scripts to `package.json`: + +```json +{ + "type": "module", + "scripts": { + "start:server": "ts-node --esm demo_mcp_server.ts", + "start:client": "ts-node --esm demo_mcp_client.ts" + } +} +``` + +Create a `tsconfig.json` file: + +```json +{ + "compilerOptions": { + "target": "ES2022", + "module": "ESNext", + "moduleResolution": "node", + "esModuleInterop": true, + "allowSyntheticDefaultImports": true, + "strict": false, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true + }, + "ts-node": { + "esm": true + } +} +``` + +## Run the Demo + +1. Start the client: + +```bash +npm run start:client +``` + +2. Open a new terminal and start the server: + +```bash +npm run start:server +``` + +Even if the client starts before the server, it will discover and connect once the server becomes available. +The client will list available tools, call the `add` tool with parameters `a=1` and `b=2`, and call the `multiply` tool with parameters `a=3` and `b=4`. + +## Conclusion + +With this end-to-end demo, you’ve successfully created a fully functional MCP over MQTT system. Now, large models such as DeepSeek, Claude, GPT, and Gemini can discover and invoke your exposed calculator tools via the MCP protocol, enabling seamless integration and intelligent interaction with external services. \ No newline at end of file diff --git a/ja_JP/emqx-ai/sdks/mcp-sdks-overview.md b/ja_JP/emqx-ai/sdks/mcp-sdks-overview.md deleted file mode 100644 index e69de29bb..000000000 diff --git a/ja_JP/emqx-ai/sdks/multimedia-ai/overview.md b/ja_JP/emqx-ai/sdks/multimedia-ai/overview.md index e69de29bb..9b9adde02 100644 --- a/ja_JP/emqx-ai/sdks/multimedia-ai/overview.md +++ b/ja_JP/emqx-ai/sdks/multimedia-ai/overview.md @@ -0,0 +1,11 @@ +# Clients Compatible with Multimedia Services + +The Multimedia AI Server uses WebRTC for audio and video streaming, and MQTT for general messages and WebRTC signaling. In practice, clients do not need a dedicated SDK. Any client that supports WebRTC and MQTT protocols can interact with the Multimedia AI Service. Common client types include: + +- **Web Browsers**: Modern browsers such as Chrome, Firefox, Edge, and Safari support WebRTC natively and can directly access the Multimedia AI Service. +- **Mobile Applications**: Mobile apps can integrate a WebRTC SDK (e.g., [Pion](https://pion.ly)) to communicate with the Multimedia AI Service. +- **Embedded Devices**: IoT devices can integrate WebRTC libraries adapted for their platforms, such as [ESP32 WebRTC](https://github.com/espressif/esp-webrtc-solution), to connect to the Multimedia AI Service. + +For details on WebRTC signaling and MQTT message formats, see the [Multimedia AI Messaging Protocol](../../multimedia-ai/message-protocol.md). + +We also provide a browser-based client code example demonstrating how to interact with the Multimedia Service: [TypeScript WebRTC Example](./webrtc-typescript.md). \ No newline at end of file diff --git a/ja_JP/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md b/ja_JP/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md index e69de29bb..00ed5cd08 100644 --- a/ja_JP/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md +++ b/ja_JP/emqx-ai/sdks/multimedia-ai/webrtc-typescript.md @@ -0,0 +1,464 @@ +# TypeScript WebRTC Example + +This document demonstrates how to use TypeScript to interact with the EMQX Multimedia Service via MQTT signaling for WebRTC, enabling audio/video calls, speech recognition (ASR), and text-to-speech (TTS). + +The EMQX Multimedia Service uses the MQTT protocol to exchange WebRTC signaling. Clients can easily integrate real-time audio/video communication and leverage AI-powered services such as ASR and TTS. + +## Core Architecture + +```shell +TypeScript Client ←→ MQTT Signaling ←→ EMQX Multimedia Service + ↓ + WebRTC P2P Connection +``` + +**Key MQTT Topics**: + +- `$webrtc/{device_id}` — receive SDP answers and ICE candidates from the server. +- `$webrtc/{device_id}/multimedia_proxy` — send SDP offers and ICE candidates. +- `$message/{device_id}` — receive ASR, TTS, and general messages. + +## Core Implementation + +### WebRTC Signaling Class + +`MqttWebRTCSignaling` is the core signaling management class. It handles MQTT message exchange, WebRTC connection setup, and multimedia service integration. The class abstracts complex signaling logic into a simple API for developers. + +```typescript +import type { MqttClient } from 'mqtt' + +interface SignalingMessage { + type: 'sdp_offer' | 'sdp_answer' | 'ice_candidate' | 'webrtc_terminated' + | 'asr_response' | 'tts_begin' | 'tts_text' | 'tts_complete' | 'tts_terminate' + | 'chat' | 'message' + data?: any + payload?: any + reason?: string + results?: string + text?: string + task_id?: string +} + +interface WebRTCCallbacks { + onLocalStream?: (stream: MediaStream) => void + onRemoteStream?: (stream: MediaStream) => void + onConnectionStateChange?: (state: string) => void + onASRResponse?: (text: string) => void + onTTSText?: (text: string) => void + onMessage?: (message: string) => void + onError?: (error: Error) => void +} + +export class MqttWebRTCSignaling { + private mqttClient: MqttClient + private pc: RTCPeerConnection | null = null + private localStream: MediaStream | null = null + private remoteStream: MediaStream | null = null + private clientId: string + private callbacks: WebRTCCallbacks + private messageHandler: ((topic: string, message: Buffer) => void) | null = null + private ttsText: string = '' + + constructor(options: { + mqttClient: MqttClient + clientId: string + callbacks?: WebRTCCallbacks + }) { + this.mqttClient = options.mqttClient + this.clientId = options.clientId + this.callbacks = options.callbacks || {} + } + + async connect(): Promise { + if (!this.mqttClient.connected) { + throw new Error('MQTT client is not connected') + } + + // Subscribe to topics + await this.subscribeToTopics() + + // Set up message handler + this.setupMessageHandler() + + // Initialize WebRTC + await this.setupWebRTC() + } + + private async subscribeToTopics(): Promise { + const topics = [ + `$webrtc/${this.clientId}`, + `$message/${this.clientId}` + ] + + return new Promise((resolve, reject) => { + this.mqttClient.subscribe(topics, { qos: 0 }, (err) => { + if (err) reject(err) + else resolve() + }) + }) + } + + private setupMessageHandler(): void { + this.messageHandler = (topic: string, message: Buffer) => { + if (topic === `$webrtc/${this.clientId}` || topic === `$message/${this.clientId}`) { + try { + const payload = JSON.parse(message.toString()) as SignalingMessage + this.handleSignalingMessage(topic, payload) + } catch (error) { + console.error('Failed to parse message:', error) + } + } + } + + this.mqttClient.on('message', this.messageHandler) + } + + private async handleSignalingMessage(topic: string, message: SignalingMessage): Promise { + if (topic === `$webrtc/${this.clientId}`) { + // WebRTC signaling handling + switch (message.type) { + case 'sdp_answer': + if (this.pc && message.data) { + const answer = new RTCSessionDescription({ + type: 'answer', + sdp: message.data.sdp || message.data + }) + await this.pc.setRemoteDescription(answer) + } + break + + case 'ice_candidate': + if (this.pc && message.data) { + const candidate = new RTCIceCandidate(message.data) + await this.pc.addIceCandidate(candidate) + } + break + + case 'webrtc_terminated': + this.callbacks.onError?.(new Error(`WebRTC terminated: ${message.reason}`)) + this.disconnect() + break + } + } else if (topic === `$message/${this.clientId}`) { + // Multimedia service message handling + switch (message.type) { + case 'asr_response': + this.callbacks.onASRResponse?.(message.results || '') + break + + case 'tts_begin': + this.ttsText = '' + break + + case 'tts_text': + this.ttsText += message.text || '' + this.callbacks.onTTSText?.(this.ttsText) + break + + case 'tts_complete': + console.log('TTS complete:', message.task_id) + break + + case 'tts_terminate': + console.log('TTS terminated:', message.task_id) + break + + case 'message': + this.callbacks.onMessage?.(message.payload || '') + break + } + } + } + + private async setupWebRTC(): Promise { + // Get user media + this.localStream = await navigator.mediaDevices.getUserMedia({ + video: true, + audio: { + echoCancellation: true, + noiseSuppression: true, + autoGainControl: true, + sampleRate: 48000 + } + }) + + this.callbacks.onLocalStream?.(this.localStream) + + // Create RTCPeerConnection + this.pc = new RTCPeerConnection({ + iceServers: [{ urls: 'stun:stun.l.google.com:19302' }] + }) + this.remoteStream = new MediaStream() + + // Add local stream + this.localStream.getTracks().forEach(track => { + this.pc!.addTrack(track, this.localStream!) + }) + + // Set event handlers + this.setupPeerConnectionHandlers() + + // Create and send offer + const offer = await this.pc.createOffer() + await this.pc.setLocalDescription(offer) + this.sendSignal('sdp_offer', offer) + } + + private setupPeerConnectionHandlers(): void { + if (!this.pc) return + + // ICE candidate handling + this.pc.onicecandidate = (event) => { + if (event.candidate) { + this.sendSignal('ice_candidate', event.candidate) + } + } + + // Connection state changes + this.pc.onconnectionstatechange = () => { + if (this.pc) { + this.callbacks.onConnectionStateChange?.(this.pc.connectionState) + + if (this.pc.connectionState === 'failed') { + this.callbacks.onError?.(new Error('Connection failed')) + } + } + } + + // Remote stream handling + this.pc.ontrack = (event) => { + if (this.remoteStream) { + this.remoteStream.addTrack(event.track) + this.callbacks.onRemoteStream?.(this.remoteStream) + } + } + } + + private sendSignal(type: string, data: any): void { + const message = JSON.stringify({ type, data }) + const topic = `$webrtc/${this.clientId}/multimedia_proxy` + + this.mqttClient.publish(topic, message, { qos: 0 }, (err) => { + if (err) { + console.error(`Failed to send ${type}:`, err) + } + }) + } + + // Audio/video control + toggleAudio(enabled?: boolean): void { + if (!this.localStream) return + + this.localStream.getAudioTracks().forEach(track => { + track.enabled = enabled !== undefined ? enabled : !track.enabled + }) + } + + toggleVideo(enabled?: boolean): void { + if (!this.localStream) return + + this.localStream.getVideoTracks().forEach(track => { + track.enabled = enabled !== undefined ? enabled : !track.enabled + }) + } + + // Send chat message + sendChatMessage(message: string): void { + this.sendSignal('chat', message) + } + + disconnect(): void { + // Stop media streams + if (this.localStream) { + this.localStream.getTracks().forEach(track => track.stop()) + this.localStream = null + } + + if (this.remoteStream) { + this.remoteStream.getTracks().forEach(track => track.stop()) + this.remoteStream = null + } + + // Close peer connection + if (this.pc) { + this.pc.close() + this.pc = null + } + + // Clean up message handler + if (this.mqttClient && this.messageHandler) { + this.mqttClient.removeListener('message', this.messageHandler) + this.messageHandler = null + } + + // Unsubscribe + const topics = [`$webrtc/${this.clientId}`, `$message/${this.clientId}`] + this.mqttClient.unsubscribe(topics) + + this.callbacks.onConnectionStateChange?.('disconnected') + } +} +``` + +## Usage Example + +The following example shows basic usage of the API. The WebRTC signaling class can be used in any frontend environment, including Vue, React, or directly in vanilla JavaScript/HTML. + +For a complete working demo, see the [MCP AI Companion Demo](https://github.com/emqx/mcp-ai-companion-demo) (web client). + +### Basic Connection and Callback Setup + +To establish a WebRTC connection: + +1. First connect to MQTT. +2. Then configure callbacks for handling audio/video streams and AI service responses. + +```typescript +import mqtt from 'mqtt' +import { MqttWebRTCSignaling } from './mqtt-webrtc-signaling' + +// Connect to MQTT +const mqttClient = mqtt.connect('ws://broker.emqx.io:8083/mqtt', { + clientId: 'device_123', + username: 'your-username', + password: 'your-password' +}) + +// Create WebRTC signaling +const signaling = new MqttWebRTCSignaling({ + mqttClient, + clientId: 'device_123', + callbacks: { + onLocalStream: (stream) => { + document.getElementById('localVideo').srcObject = stream + }, + onRemoteStream: (stream) => { + document.getElementById('remoteVideo').srcObject = stream + }, + onASRResponse: (text) => { + console.log('Speech recognition:', text) + }, + onTTSText: (text) => { + console.log('AI reply:', text) + }, + onConnectionStateChange: (state) => { + console.log('Connection state:', state) + } + } +}) + +// Start connection +await signaling.connect() + +``` + +### Multimedia Control + +Once connected, you can easily control audio/video devices, send messages, and manage connection state: + +```typescript +// Audio and video control +signaling.toggleAudio(false) // Mute +signaling.toggleVideo(true) // Enable camera + +// Send a chat message +signaling.sendChatMessage('Hello AI!') + +// Disconnect +signaling.disconnect() +``` + +## Key Interactions + +### MQTT Signaling Workflow + +WebRTC connections use MQTT topics for signaling, following the standard offer/answer model: + +``` +1. Client creates offer → sends to $webrtc/{device_id}/multimedia_proxy +2. Server processes offer → returns answer on $webrtc/{device_id} +3. Both sides exchange ICE candidates via these topics +4. P2P connection established → audio/video streaming begins +``` + +### Multimedia AI Services + +The EMQX Multimedia Service provides three core AI features via the `$message/{device_id}` topic: + +- **ASR (Automatic Speech Recognition)**: real-time speech-to-text, supports streaming recognition. +- **TTS (Text-to-Speech)**: converts text to natural speech, supports incremental text streaming. +- **Intelligent Dialogue**: combines ASR and TTS to enable end-to-end voice interaction. + +## Error Handling and Best Practices + +In real-world applications, you must handle potential errors gracefully: + +```typescript +// Common error handling +callbacks: { + onError: (error) => { + if (error.message.includes('getUserMedia')) { + alert('Unable to access camera/microphone. Please check permissions.') + } else if (error.message.includes('Connection failed')) { + // Auto-reconnect logic + setTimeout(() => signaling?.connect(), 2000) + } + } +} +``` + +**Common solutions**: + +- **Media permission issues**: Ensure camera and microphone permissions are granted. +- **Network instability**: Implement auto-reconnect to handle disruptions. +- **Signaling timeouts**: Configure connection timeouts and show user-friendly error messages. + +## Advanced Configuration + +### ICE Server Configuration + +To ensure connectivity across different network environments, configure STUN/TURN servers: + +```typescript +const iceServers = [ + { urls: 'stun:stun.l.google.com:19302' }, // Public STUN server + { + urls: 'turn:your-turn-server.com:3478', // Private TURN server (recommended for production) + username: 'turnuser', + credential: 'turnpass' + } +] +``` + +### Media Constraints Optimization + +Adjust audio/video quality to balance performance and bandwidth usage: + +```typescript +const mediaConstraints = { + video: { + width: { ideal: 1280 }, // Video width + height: { ideal: 720 }, // Video height + frameRate: { ideal: 30 } // Frame rate + }, + audio: { + echoCancellation: true, // Echo cancellation + noiseSuppression: true, // Noise suppression + autoGainControl: true, // Automatic gain control + sampleRate: 48000, // Sampling rate + sampleSize: 16, // Bit depth + channelCount: 2 // Number of channels + } +} +``` + +## Summary + +By integrating TypeScript WebRTC with the EMQX Multimedia Service, you can easily enable: + +- **Real-time Audio/Video Calls**: high-quality, low-latency communication. +- **AI Speech Recognition**: real-time multilingual speech-to-text. +- **AI Speech Synthesis**: natural text-to-speech conversion. +- **Cross-Platform Compatibility**: works with major frontend frameworks and native JavaScript. + +This provides a complete foundation for building modern, AI-powered communication applications. diff --git a/ja_JP/emqx-ai/sdks/overview.md b/ja_JP/emqx-ai/sdks/overview.md index e69de29bb..9f3764753 100644 --- a/ja_JP/emqx-ai/sdks/overview.md +++ b/ja_JP/emqx-ai/sdks/overview.md @@ -0,0 +1,21 @@ +# MCP over MQTT and Multimedia Service SDKs + +EMQX provides SDKs for various platforms and programming languages to help developers quickly integrate MCP over MQTT and Multimedia Server features. These SDKs abstract the underlying communication details, allowing developers to focus on implementing business logic. + +## Multimedia Service Client Code Examples + +EMQX offers client-side examples based on WebRTC and MQTT, demonstrating how to interact with the multimedia server for audio/video calls, speech recognition (ASR), and text-to-speech (TTS). + +Developers can use these examples as a reference to quickly integrate multimedia AI capabilities into web applications or devices. + +- [TypeScript WebRTC Example](./multimedia-ai/webrtc-typescript.md) + +## MCP over MQTT SDKs + +EMQX provides SDKs in multiple programming languages, making it easy to integrate MCP over MQTT across different platforms and environments: + +- [Erlang SDK](./mcp-sdk-erlang.md) +- [ESP32 C SDK](./mcp-sdk-esp32-c.md) +- [C SDK with Paho MQTT](./mcp-sdk-paho-c.md) +- [Python SDK](./mcp-sdk-python.md) +- [TypeScript SDK](./mcp-sdk-typescript.md) diff --git a/zh_CN/emqx-ai/sdks/overview.md b/zh_CN/emqx-ai/sdks/overview.md index e0be3bdbf..5fb984c41 100644 --- a/zh_CN/emqx-ai/sdks/overview.md +++ b/zh_CN/emqx-ai/sdks/overview.md @@ -8,7 +8,7 @@ EMQX 提供了基于 WebRTC 和 MQTT 的客户端示例，演示如何与多媒 开发者可以直接使用这些示例作为参考，将多媒体 AI 能力快速集成到 Web 应用或设备中。 -- [TypeScript WebRTC 示例](./webrtc-typescript.md) +- [TypeScript WebRTC 示例](./multimedia-ai/webrtc-typescript.md) ## MCP over MQTT SDKs From 090d05cfcf00cc40eb8557d343792034722f99b9 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Thu, 18 Sep 2025 12:00:36 +0800 Subject: [PATCH 45/49] Update dir.yaml --- dir.yaml | 45 --------------------------------------------- 1 file changed, 45 deletions(-) diff --git a/dir.yaml b/dir.yaml index bafbf7abc..730f5d2b2 100644 --- a/dir.yaml +++ b/dir.yaml @@ -643,51 +643,6 @@ # children: # - data-integration/rule-configs -- title_en: EMQX AI - title_cn: EMQX AI - title_ja: EMQX AI - path: emqx-ai/overview - collapsed: true - children: - - title_en: MCP over MQTT - title_cn: MCP over MQTT - title_ja: MCP over MQTT - path: emqx-ai/mcp-over-mqtt/overview - children: - - emqx-ai/mcp-over-mqtt/architecture - - emqx-ai/mcp-over-mqtt/specification - - title_en: Multimedia Server - title_cn: 多媒体服务器 - title_ja: マルチメディアサーバー - path: emqx-ai/multimedia-ai/overview - children: - - emqx-ai/multimedia-ai/architecture - - emqx-ai/multimedia-ai/message-protocol - - title_en: SDKs and Examples - title_cn: SDK 和示例 - title_ja: SDK と例 - path: emqx-ai/sdks/overview - collapsed: true - children: - - title_en: MCP SDK - title_cn: MCP SDK - title_ja: MCP SDK - path: emqx-ai/sdks/mcp-sdks-overview - collapsed: true - children: - - emqx-ai/sdks/mcp-sdk-esp32-c - - emqx-ai/sdks/mcp-sdk-paho-c - - emqx-ai/sdks/mcp-sdk-python - - emqx-ai/sdks/mcp-sdk-typescript - - emqx-ai/sdks/mcp-sdk-erlang - - title_en: Multimedia SDK - title_cn: 多媒体 SDK - title_ja: マルチメディア SDK - path: emqx-ai/sdks/multimedia-ai/overview - collapsed: true - children: - - emqx-ai/sdks/multimedia-ai/webrtc-typescript - - title_en: Administration Guide title_cn: 管理员指南 title_ja: 管理者ガイド From 9d4c2742611356807d04cf4de0311819e03a1c83 Mon Sep 17 00:00:00 2001 From: Shawn <506895667@qq.com> Date: Thu, 18 Sep 2025 12:18:57 +0800 Subject: [PATCH 46/49] remove the support for uv --- .python-version | 1 - pyproject.toml | 9 --------- 2 files changed, 10 deletions(-) delete mode 100644 .python-version delete mode 100644 pyproject.toml diff --git a/.python-version b/.python-version deleted file mode 100644 index 2c0733315..000000000 --- a/.python-version +++ /dev/null @@ -1 +0,0 @@ -3.11 diff --git a/pyproject.toml b/pyproject.toml deleted file mode 100644 index dd813ee59..000000000 --- a/pyproject.toml +++ /dev/null @@ -1,9 +0,0 @@ -[project] -name = "emqx-docs" -version = "0.1.0" -description = "Add your description here" -readme = "README.md" -requires-python = ">=3.11" -dependencies = [ - "pyyaml>=6.0.2", -] From 8f141867e896621462a2632de5d074b81b183ef3 Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Thu, 18 Sep 2025 17:58:58 +0800 Subject: [PATCH 47/49] Update dates --- en_US/changes/all-changes-ee.md | 2 +- en_US/changes/changes-ee-v5.md | 2 +- zh_CN/changes/all-changes-ee.md | 2 +- zh_CN/changes/changes-ee-v5.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/en_US/changes/all-changes-ee.md b/en_US/changes/all-changes-ee.md index ecc5e9b02..72f1dcded 100644 --- a/en_US/changes/all-changes-ee.md +++ b/en_US/changes/all-changes-ee.md @@ -4,7 +4,7 @@ The release notes page for EMQX Enterprise provides a comprehensive and detailed ## v5.10 -- [5.10.1](./changes-ee-v5.md#_5-10-1): 2025-09-17 +- [5.10.1](./changes-ee-v5.md#_5-10-1): 2025-09-18 - [5.10.0](./changes-ee-v5.md#_5-10-0): 2025-06-10 ## v5.9 diff --git a/en_US/changes/changes-ee-v5.md b/en_US/changes/changes-ee-v5.md index fada9fb75..9b5c8123c 100644 --- a/en_US/changes/changes-ee-v5.md +++ b/en_US/changes/changes-ee-v5.md @@ -2,7 +2,7 @@ ## 5.10.1 -*Release Date: 2025-09-17* +*Release Date: 2025-09-18* Make sure to check the breaking changes and known issues before upgrading to EMQX 5.10.1. diff --git a/zh_CN/changes/all-changes-ee.md b/zh_CN/changes/all-changes-ee.md index f839e9b09..4cef42957 100644 --- a/zh_CN/changes/all-changes-ee.md +++ b/zh_CN/changes/all-changes-ee.md @@ -4,7 +4,7 @@ EMQX 企业版版本发布页面全面详细地记录了 EMQX 企业版每个版 ## v5.10 -- [5.10.1](./changes-ee-v5.md#_5-10-1): 2025-09-17 +- [5.10.1](./changes-ee-v5.md#_5-10-1): 2025-09-18 - [5.10.0](./changes-ee-v5.md#_5-10-0): 2025-06-10 ## v5.9 diff --git a/zh_CN/changes/changes-ee-v5.md b/zh_CN/changes/changes-ee-v5.md index a56222063..575e53103 100644 --- a/zh_CN/changes/changes-ee-v5.md +++ b/zh_CN/changes/changes-ee-v5.md @@ -2,7 +2,7 @@ ## 5.10.1 -*发布日期：2025-09-17* +*发布日期：2025-09-18* 升级前请查看已知问题列表和不兼容变更列表。 From c428aa27438fa41759a6b2266610b06ca04d73fe Mon Sep 17 00:00:00 2001 From: Ivan Dyachkov Date: Thu, 18 Sep 2025 12:24:49 +0200 Subject: [PATCH 48/49] chore: add missing changelog entries to 5.10.1, update release date --- en_US/changes/changes-ee-v5.md | 50 +++++++++++++++++++++++++++------- 1 file changed, 40 insertions(+), 10 deletions(-) diff --git a/en_US/changes/changes-ee-v5.md b/en_US/changes/changes-ee-v5.md index 9b5c8123c..b2c185508 100644 --- a/en_US/changes/changes-ee-v5.md +++ b/en_US/changes/changes-ee-v5.md @@ -8,10 +8,23 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ ### Enhancements +#### Performance + +- [#15899](https://github.com/emqx/emqx/pull/15899) Authorization (authz) cache is now cleared immediately when a client disconnects, reducing unnecessary memory consumption. + +- [#15907](https://github.com/emqx/emqx/pull/15907) Improve system memory usage. Fields such as client ID, username, password, and topic are copied into new binaries (when more than 64 bytes) instead of being slices from the raw packet to reduce 'binary' part of memory usage in Erlang VM. + +#### Observability + +- [#15499](https://github.com/emqx/emqx/pull/15499) Added a force deactivate alarm API endpoint to allow administrators to forcibly deactivate active alarms. + +- [#15364](https://github.com/emqx/emqx/pull/15364) Added HTTP header configuration items to the OpenTelemetry integration to adapt to collectors with HTTP authentication. + #### Access Control - [#15294](https://github.com/emqx/emqx/pull/15294) Enhanced LDAP authentication and authorization. LDAP authorization now supports an extended ACL rule format using JSON, in addition to the existing simple topic list. ACL rules can also be fetched from LDAP during authentication based on client information, and are cached in the client’s metadata to avoid repeated LDAP queries during authorization. + - [#15349](https://github.com/emqx/emqx/pull/15349) Optimized external resource management for authentication and authorization. Previously, EMQX could remain connected to a resource configured for a disabled authentication or authorization provider. #### Data Integration @@ -27,15 +40,25 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ Also introduced a new `resource_opts.health_check_interval_jitter`, which adds a uniform random delay to `resource_opts.health_check_interval` to reduce the chance of multiple Actions under the same Connector running health checks at the same time. - [#15542](https://github.com/emqx/emqx/pull/15542) Upgraded our `erlcloud` library to `3.8.3.0`. This allows one to setup a S3 Connector without specifying Access Key Id and Secret Access Key, so long as the EC2 instance EMQX is running in has the correct IAM permissions to read/write to the configured bucket(s). + - [#15845](https://github.com/emqx/emqx/pull/15845) The `static_clientids` configuration for the MQTT Connector now supports specifying a username and password for each client ID. This is particularly useful for scenarios like connecting to Azure IoT Hub, where each device (client ID) requires a unique set of credentials. This enhancement helps ensure successful connections across multiple nodes in a clustered environment. + - [#15911](https://github.com/emqx/emqx/pull/15911) The HTTP request timeout for the HTTP Action is now configurable via the `resource_opts.request_ttl` setting. Previously, this timeout was fixed at 30 seconds and could not be adjusted. + - [#15371](https://github.com/emqx/emqx/pull/15371) Added `tags` fields to the responses of `GET /actions_summary` and `GET /sources_summary` endpoints, and to the fallback actions returned by the `GET /actions/:id` endpoint. #### Observability - [#15499](https://github.com/emqx/emqx/pull/15499) Added a force deactivate alarm API endpoint to allow administrators to forcibly deactivate active alarms. + - [#15364](https://github.com/emqx/emqx/pull/15364) Added HTTP header configuration items to the OpenTelemetry integration to adapt to collectors with HTTP authentication. +- [#15944](https://github.com/emqx/emqx/pull/15944) Improved the information returned when a resource is marked as `disconnected` for the following Connectors: LDAP, Syskeeper, IoTDB, Snowflake (aggregated), JWKS Authentication. + +- [#15911](https://github.com/emqx/emqx/pull/15911) Now, for the HTTP Action, the HTTP request timeout is taken to be the same as `resource_opts.request_ttl`. Previously, it was a fixed, non-configurable value of 30 seconds. + +- [#15371](https://github.com/emqx/emqx/pull/15371) Added `tags` fields to the return of `GET /actions_summary` and `GET /sources_summary`, and to the fallback actions returned in `GET /actions/:id`. + #### CLI - [#15399](https://github.com/emqx/emqx/pull/15399) The `node_dump` tool now exports the current system configuration in HOCON format, with sensitive information (such as passwords and secrets) automatically redacted for security. @@ -44,9 +67,10 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ #### Core MQTT Functionalities - - [#15361](https://github.com/emqx/emqx/pull/15361) Fixed a `function_clause` error when parsing a malformed `User-Property` pair with invalid (too short) length. + - [#15396](https://github.com/emqx/emqx/pull/15396) Removed redundant cleanup operations for shared subscriptions of disconnected clients. These operations were prone to crashes under high disconnect volumes and could lead to inconsistencies in the global broker state. + - [#15416](https://github.com/emqx/emqx/pull/15416) Fixed occasional warning-level log events and crashes during session expiration of WebSocket connections. This issue was introduced by recent WebSocket performance improvements. If did not affect broker capacity, but produced log entries like the following: * `error: {function_clause,[{gen_tcp,send,[closed,[]],[{file,“gen_tcp.erl”},{line,966}]},{cowboy_websocket_linger,commands,3,[{file,“cowboy_websocket_linger.erl”},{line,665}]},...` * `message: {tcp,#Port<0.364>,<<136,130,...>>}, msg: emqx_session_mem_unknown_message` @@ -57,7 +81,6 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ #### Access Control - - [#15818](https://github.com/emqx/emqx/pull/15818) Corrected handling of `{allow|deny, all}` ACL rules. Previously, these rules were internally translated to match `#`, which incorrectly failed to match topics prefixed with `$` (e.g. `$testtopic/1`) due to MQTT spec restrictions. @@ -75,10 +98,8 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ #### Deployment - - [#15553](https://github.com/emqx/emqx/pull/15553) Fixed an issue in the Helm chart where deploying EMQX with default values started multiple replicas and caused all nodes except one to crash. The chart now defaults to a single replica, since clustered deployments require an Commercial License. - - [#15712](https://github.com/emqx/emqx/pull/15712) Fix node boot-up failure during rolling upgrade from older versions (before 5.9) In previous EMQX versions (before 5.9), a bug in the ZIP timestamp encoder could store an invalid “seconds” value in archive entries (values corresponding to the 30th or 31st 2-second slot in DOS time format). @@ -87,17 +108,14 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ #### Clustering - - [#15788](https://github.com/emqx/emqx/pull/15788) Fixed etcd cluster discovery issue. Resolved an issue where EMQX nodes from different clusters could mistakenly join each other when using a shared etcd server. This was caused by a bug in the etcd client library. #### Smart Data Hub - - [#15810](https://github.com/emqx/emqx/pull/15810) Introduced `spb_{en,de}code` functions to correct handling of `bytes_value` Metrics. Fixed an issue with the original `sparkplug_{en,de}code` functions, which did not base64 encode/decode `bytes_value` metric values as required by the [Protobuf specification](https://protobuf.dev/programming-guides/json/). To address this, new `spb_{en,de}code` functions have been introduced for correct encoding/decoding of such fields. The old `sparkplug_{en,de}code` functions are now deprecated to maintain backward compatibility. #### Data Integration - - [#15394](https://github.com/emqx/emqx/pull/15394) Fixed a rare race condition where Action metrics could become inconsistent due to unexpected asynchronous replies. - [#15603](https://github.com/emqx/emqx/pull/15603) Fixed an issue in the MQTT bridge where a stale connection could be shown as `Connected` and would not automatically reconnect. @@ -110,11 +128,17 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ - [#15836](https://github.com/emqx/emqx/pull/15836) Enriched the returned information when a Kafka Consumer Source fails to be added, for example, due to denied topic ACLs -- [#15866](https://github.com/emqx/emqx/pull/15866) Upgrade Kafka producer lib wollf to 4.0.12 to improve handling of temporarily missing partitions in Kafka metadata responses. +- [#15866](https://github.com/emqx/emqx/pull/15866) Upgrade Kafka producer lib wollf to `4.0.12` to improve handling of temporarily missing partitions in Kafka metadata responses. In rare race conditions, Kafka may return an incomplete partition list. Previously, this was only handled when a topic was recreated with fewer partitions, but not when partitions were temporarily missing. This gap could cause the partition producer to stall and block shutdown indefinitely. -- [#15906](https://github.com/emqx/emqx/pull/15906) Upgraded Kafka producer library Wolff from 4.0.12 to 4.0.13, which adds handling for the `record_list_too_large` error in `ProduceResponse`. +- [#15906](https://github.com/emqx/emqx/pull/15906) Upgraded Kafka producer library Wolff from `4.0.12` to `4.0.13`, which adds handling for the `record_list_too_large` error in `ProduceResponse`. + +- [#15902](https://github.com/emqx/emqx/pull/15902) Upgrade MQTT client library to 1.13.8 + + This improves MQTT bridge connectivity with: + - Connector will automatically reconnect when peer broker does not reply PINGRESP. + - Bridge over TLS failure is more promptly handled if connection breaks while waiting for CONNACK. - [#15910](https://github.com/emqx/emqx/pull/15910) Fixed an issue with Connectors where a pool of workers could fail to recover from a failure if multiple workers crashed simultaneously in large worker pools. @@ -127,15 +151,21 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ - TDEngine - Cassandra - Dynamo + - HTTP + - Couchbase + - GCP PubSub + - Snowflake + + Upgraded `gun` and related dependencies to 2.1.0. #### API - [#15547](https://github.com/emqx/emqx/pull/15547) Resolved an issue where EMQX would fail to process HTTP requests with large bodies (e.g., 10MB) in the REST API. + - [#15797](https://github.com/emqx/emqx/pull/15797) To improve compatibility with EMQX 4.x, the `encoding` parameter has been reintroduced in the batch publish HTTP API (`/api/v5/publish/bulk`) as an alias for `payload_encoding`. This change addresses migration issues for users relying on the original `encoding` parameter, and ensures existing integrations using EMQX v4 APIs can continue working without requiring software-level changes. #### Rate Limit - - [#15794](https://github.com/emqx/emqx/pull/15794) Improved the behavior of connection rate limit updates to ensure that changes (e.g., to burst rate or rate thresholds) are applied immediately after the listener configuration is updated. Previously, parts of the internal limiter state were not refreshed correctly, which could result in rate limits appearing stricter than configured. #### Observability From 9cb66542931977622aa5ffbe2887c0feb5df836e Mon Sep 17 00:00:00 2001 From: Meggielqk <126552073+Meggielqk@users.noreply.github.com> Date: Fri, 19 Sep 2025 15:26:02 +0800 Subject: [PATCH 49/49] Update zh file --- en_US/changes/changes-ee-v5.md | 24 +++++------------------ zh_CN/changes/changes-ee-v5.md | 36 +++++++++++++++++++++++++++++----- 2 files changed, 36 insertions(+), 24 deletions(-) diff --git a/en_US/changes/changes-ee-v5.md b/en_US/changes/changes-ee-v5.md index b2c185508..e87ca9191 100644 --- a/en_US/changes/changes-ee-v5.md +++ b/en_US/changes/changes-ee-v5.md @@ -12,13 +12,7 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ - [#15899](https://github.com/emqx/emqx/pull/15899) Authorization (authz) cache is now cleared immediately when a client disconnects, reducing unnecessary memory consumption. -- [#15907](https://github.com/emqx/emqx/pull/15907) Improve system memory usage. Fields such as client ID, username, password, and topic are copied into new binaries (when more than 64 bytes) instead of being slices from the raw packet to reduce 'binary' part of memory usage in Erlang VM. - -#### Observability - -- [#15499](https://github.com/emqx/emqx/pull/15499) Added a force deactivate alarm API endpoint to allow administrators to forcibly deactivate active alarms. - -- [#15364](https://github.com/emqx/emqx/pull/15364) Added HTTP header configuration items to the OpenTelemetry integration to adapt to collectors with HTTP authentication. +- [#15907](https://github.com/emqx/emqx/pull/15907) Improved system memory usage. Fields such as client ID, username, password, and topic are copied into new binaries (when more than 64 bytes) instead of being slices from the raw packet to reduce 'binary' part of memory usage in Erlang VM. #### Access Control @@ -45,19 +39,12 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ - [#15911](https://github.com/emqx/emqx/pull/15911) The HTTP request timeout for the HTTP Action is now configurable via the `resource_opts.request_ttl` setting. Previously, this timeout was fixed at 30 seconds and could not be adjusted. -- [#15371](https://github.com/emqx/emqx/pull/15371) Added `tags` fields to the responses of `GET /actions_summary` and `GET /sources_summary` endpoints, and to the fallback actions returned by the `GET /actions/:id` endpoint. - #### Observability - [#15499](https://github.com/emqx/emqx/pull/15499) Added a force deactivate alarm API endpoint to allow administrators to forcibly deactivate active alarms. - - [#15364](https://github.com/emqx/emqx/pull/15364) Added HTTP header configuration items to the OpenTelemetry integration to adapt to collectors with HTTP authentication. - - [#15944](https://github.com/emqx/emqx/pull/15944) Improved the information returned when a resource is marked as `disconnected` for the following Connectors: LDAP, Syskeeper, IoTDB, Snowflake (aggregated), JWKS Authentication. - -- [#15911](https://github.com/emqx/emqx/pull/15911) Now, for the HTTP Action, the HTTP request timeout is taken to be the same as `resource_opts.request_ttl`. Previously, it was a fixed, non-configurable value of 30 seconds. - -- [#15371](https://github.com/emqx/emqx/pull/15371) Added `tags` fields to the return of `GET /actions_summary` and `GET /sources_summary`, and to the fallback actions returned in `GET /actions/:id`. +- [#15371](https://github.com/emqx/emqx/pull/15371) Added `tags` fields to the responses of `GET /actions_summary` and `GET /sources_summary` endpoints, and to the fallback actions returned by the `GET /actions/:id` endpoint. #### CLI @@ -128,18 +115,17 @@ Make sure to check the breaking changes and known issues before upgrading to EMQ - [#15836](https://github.com/emqx/emqx/pull/15836) Enriched the returned information when a Kafka Consumer Source fails to be added, for example, due to denied topic ACLs -- [#15866](https://github.com/emqx/emqx/pull/15866) Upgrade Kafka producer lib wollf to `4.0.12` to improve handling of temporarily missing partitions in Kafka metadata responses. +- [#15866](https://github.com/emqx/emqx/pull/15866) Upgraded Kafka producer lib wollf to `4.0.12` to improve handling of temporarily missing partitions in Kafka metadata responses. In rare race conditions, Kafka may return an incomplete partition list. Previously, this was only handled when a topic was recreated with fewer partitions, but not when partitions were temporarily missing. This gap could cause the partition producer to stall and block shutdown indefinitely. - [#15906](https://github.com/emqx/emqx/pull/15906) Upgraded Kafka producer library Wolff from `4.0.12` to `4.0.13`, which adds handling for the `record_list_too_large` error in `ProduceResponse`. -- [#15902](https://github.com/emqx/emqx/pull/15902) Upgrade MQTT client library to 1.13.8 +- [#15902](https://github.com/emqx/emqx/pull/15902) Upgraded MQTT client library to 1.13.8. This improves MQTT bridge connectivity with: - This improves MQTT bridge connectivity with: - Connector will automatically reconnect when peer broker does not reply PINGRESP. - Bridge over TLS failure is more promptly handled if connection breaks while waiting for CONNACK. - + - [#15910](https://github.com/emqx/emqx/pull/15910) Fixed an issue with Connectors where a pool of workers could fail to recover from a failure if multiple workers crashed simultaneously in large worker pools. Connectors affected and fixed: diff --git a/zh_CN/changes/changes-ee-v5.md b/zh_CN/changes/changes-ee-v5.md index 575e53103..b2673f578 100644 --- a/zh_CN/changes/changes-ee-v5.md +++ b/zh_CN/changes/changes-ee-v5.md @@ -8,6 +8,11 @@ ### 增强 +#### 性能 + +- [#15899](https://github.com/emqx/emqx/pull/15899) 通过确保在客户端断开时立即清除授权（authz）缓存来改进内存管理，减少不必要的内存消耗。 +- [#15907](https://github.com/emqx/emqx/pull/15907) 优化了系统内存使用。当客户端 ID、用户名、密码和主题等字段长度超过 64 字节时，这些字段将被复制为新的二进制数据，而不再是原始报文的切片，以减少 Erlang 虚拟机中 “binary” 类型内存的占用。 + #### 访问控制 - [#15294](https://github.com/emqx/emqx/pull/15294) 增强了 LDAP 认证和授权功能。LDAP 授权现在支持使用 JSON 格式的扩展 ACL 规则，除了现有的简单主题列表外，还可以在认证过程中基于客户端信息从 LDAP 获取 ACL 规则，并将其缓存在客户端的元数据中，以避免在授权过程中重复进行 LDAP 查询。 @@ -24,12 +29,13 @@ - [#15542](https://github.com/emqx/emqx/pull/15542) 将 `erlcloud` 库升级到 `3.8.3.0`。升级后，如果 EMQX 运行的 EC2 实例具有正确的 IAM 权限来读写配置的 S3 存储桶，就可以在不指定访问密钥 ID 和私有访问密钥的情况下配置 S3 连接器。 - [#15845](https://github.com/emqx/emqx/pull/15845) MQTT 连接器的 `static_clientids` 配置项现支持为每个客户端 ID 分别指定用户名和密码，适用于如 Azure IoT Hub 等要求每个设备使用唯一凭证的场景。此增强提升了在集群部署中多节点连接的兼容性与稳定性。 - [#15911](https://github.com/emqx/emqx/pull/15911) HTTP 动作的 HTTP 请求超时时间现在可以通过 `resource_opts.request_ttl` 设置进行配置。此前，此超时时间固定为 30 秒且不可调整。 -- [#15371](https://github.com/emqx/emqx/pull/15371) 为 `GET /actions_summary` 和 `GET /sources_summary` 接口的响应以及 `GET /actions/:id` 接口返回的备选动作添加了 `tags` 字段。 #### 可观测性 - [#15499](https://github.com/emqx/emqx/pull/15499) 添加了强制停用告警的 API 接口，允许管理员强制停用活动告警。 - [#15364](https://github.com/emqx/emqx/pull/15364) 为 OpenTelemetry 集成添加了 HTTP 头配置项，以适应带有 HTTP 认证的 collector。 +- [#15944](https://github.com/emqx/emqx/pull/15944) 改进了以下连接器在资源被标记为 `disconnected` 状态时返回的信息：LDAP、Syskeeper、IoTDB、Snowflake（聚合模式）、JWKS 认证。 +- [#15371](https://github.com/emqx/emqx/pull/15371) 为 `GET /actions_summary` 和 `GET /sources_summary` 接口的响应以及 `GET /actions/:id` 接口返回的备选动作添加了 `tags` 字段。 #### CLI @@ -64,8 +70,6 @@ mria:transaction(emqx_authn_shard, fun() -> mnesia:delete(emqx_authn_mnesia, {'mqtt:global',<<>>}, write) end). ``` -- [#15899](https://github.com/emqx/emqx/pull/15899) 通过确保在客户端断开时立即清除授权（authz）缓存来改进内存管理，减少不必要的内存消耗。 - #### 集群 - [#15788](https://github.com/emqx/emqx/pull/15788) 修复了 etcd 集群发现问题。解决了使用共享 etcd 服务器时，EMQX 节点可能错误地加入到不同集群中的问题。此问题是由 etcd 客户端库中的 bug 引起的。 @@ -77,21 +81,43 @@ #### 数据集成 - [#15394](https://github.com/emqx/emqx/pull/15394) 修复了一个罕见的竞争条件，导致动作指标因意外的异步回复而变得不一致。 + - [#15603](https://github.com/emqx/emqx/pull/15603) 修复了 MQTT 桥接中的一个问题：过期的连接可能仍显示为 `Connected` 状态，且不会自动重连。 + - [#15826](https://github.com/emqx/emqx/pull/15826) 改进了 Kafka 消费者连接器健康检查行为，尤其是在 ACL 限制的情况下。此前，若配置的用户缺少访问内部 `____emqx_consumer_probe` 消费者组的权限，则 Kafka 消费者连接器的健康检查可能会失败。通过此修复，如果 Kafka broker 返回 "ACL denied" 响应，EMQX 将视该连接为健康连接。 + - [#15827](https://github.com/emqx/emqx/pull/15827) 修复了 GreptimeDB 驱动中的原子泄漏和进程泄漏问题。同时修复了在 GreptimeDB 动作中使用某些错误的写入语法时可能出现的 `function_clause` 错误。 + - [#15836](https://github.com/emqx/emqx/pull/15836) 丰富了 Kafka 消费者源添加失败时的返回信息，例如因主题 ACL 被拒导致的失败。 + - [#15866](https://github.com/emqx/emqx/pull/15866) 升级 Kafka 生产者库 `wolff` 到 4.0.12，以改善 Kafka 元数据响应中临时缺失分区的处理。 在罕见的竞争条件下，Kafka 可能返回不完整的分区列表。此前，这种情况仅在主题被重新创建且分区较少时得到处理，而在分区临时缺失时未被处理。此修复解决了此问题，防止分区生产者挂起并无限期阻止关闭。 -- [#15906](https://github.com/emqx/emqx/pull/15906) 将 Kafka 生产者库 Wolff 从 4.0.12 升级到 4.0.13，新增了处理 `ProduceResponse` 中 `record_list_too_large` 错误的功能。 -- [#15910](https://github.com/emqx/emqx/pull/15910) 修复了连接器在多个工作线程同时崩溃时，无法从失败中恢复的问题。受影响并已修复的连接器包括： + +- [#15906](https://github.com/emqx/emqx/pull/15906) 将 Kafka 生产者库 Wolff 从 `4.0.12` 升级到 `4.0.13`，新增了处理 `ProduceResponse` 中 `record_list_too_large` 错误的功能。 + +- [#15902](https://github.com/emqx/emqx/pull/15902) 将 MQTT 客户端库升级至 1.13.8，提升了 MQTT 桥接的连接稳定性： + + - 当对端 Broker 未响应 PINGRESP 时，连接器将自动重连。 + - 若在等待 CONNACK 期间连接中断，基于 TLS 的桥接失败将更及时地被处理。 + +- [#15910](https://github.com/emqx/emqx/pull/15910) 修复了连接器中的一个问题：在较大的工作线程池中，若多个工作线程同时崩溃，可能导致连接器无法正常恢复。 + + 受影响并已修复的连接器包括： + - MySQL + - PostgreSQL - Oracle - SQLServer - TDEngine - Cassandra - Dynamo + - HTTP + - Couchbase + - GCP PubSub + - Snowflake + + 同时将 `gun` 及相关依赖升级至 2.1.0。 #### API