Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,11 @@ DIFY_PLUGIN_SERVERLESS_CONNECTOR_URL=http://127.0.0.1:5004
DIFY_PLUGIN_SERVERLESS_CONNECTOR_API_KEY=HeRFb6yrzAy5vUSlJWK2lUl36mpkaRycv4witbQpucXacgXg7G9a8gVL
# maximum serialized request payload sent to a serverless plugin runtime
MAX_SERVERLESS_REQUEST_BYTES=5242880
# activation preflight: wake a scaled-to-zero plugin via the connector before invoking it
DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_ENABLED=false
# seconds to wait for the plugin to become ready during the activation preflight;
# if it is not woken up in time the invocation is treated as failed
DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_TIMEOUT=60

# python interpreter, if you are using local runtime, you should set this path to your python interpreter path
# otherwise, it should be /usr/bin/python3
Expand Down
38 changes: 38 additions & 0 deletions docs/runtime/sri.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ The daemon is configured using the following environment variables:
| `DIFY_PLUGIN_SERVERLESS_CONNECTOR_URL` | Base URL of the remote runtime environment, e.g., `https://example.com` |
| `DIFY_PLUGIN_SERVERLESS_CONNECTOR_API_KEY` | Authentication token for accessing SRI, passed in the `Authorization` request header |
| `MAX_SERVERLESS_REQUEST_BYTES` | Maximum serialized request payload size sent to a serverless plugin runtime (in bytes). Default is 5242880 (5 MB). This limit accounts for Lambda Function URL's 6 MB request size limit, with a safety margin for headers and metadata. |
| `DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_ENABLED` | Whether to run the activation preflight before dispatching an invocation. When enabled, the daemon calls `POST /v1/activation/activate` to wake a scaled-to-zero plugin and waits until it is ready. Default is `false`. |
| `DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_TIMEOUT` | Seconds the daemon waits for the plugin to become ready during the activation preflight. If the plugin is not woken up in time, the invocation is treated as failed. Default is `60`. |

---

Expand Down Expand Up @@ -130,6 +132,42 @@ endpoint=http://...,name=...,id=...

---

### `POST /v1/activation/activate`

Optional activation preflight. When `DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_ENABLED` is `true`, the daemon calls this endpoint before dispatching an invocation to wake a plugin that may have been scaled to zero, and blocks until the plugin is ready. It is safe to call on every invocation; the runtime is expected to throttle any underlying activity/lease writes.

**Request**

```http
POST /v1/activation/activate
Authorization: <API_KEY>
Content-Type: application/json

{
"instance_id": "string"
}
```

- `instance_id` (required): the connector function name of the target plugin (the `Name` returned by `/v1/runner/instances` and `/v1/launch`).

**Response**

```json
{
"ready": true,
"endpoint": "string"
}
```

- `200 OK` with `ready = true`: the plugin is ready and the daemon proceeds with the invocation.
- `504 Gateway Timeout` (or `ready = false`): the plugin did not become ready in time.

**Error Handling**

- The daemon bounds the wait with `DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_TIMEOUT`. On timeout or any non-`200` response, the invocation is aborted and reported as a failure.

---

## 🔁 Communication Sequence (ASCII)

```text
Expand Down
38 changes: 38 additions & 0 deletions docs/runtime/sri_cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ daemon 通过如下环境变量进行配置:
| `DIFY_PLUGIN_SERVERLESS_CONNECTOR_URL` | 指定远程运行环境的 Base URL,例如 `https://example.com` |
| `DIFY_PLUGIN_SERVERLESS_CONNECTOR_API_KEY` | 用于访问 SRI 的鉴权 token,将被加入请求 Header 中的 `Authorization` 字段 |
| `MAX_SERVERLESS_REQUEST_BYTES` | 发送到 serverless 插件运行时的最大序列化请求负载大小(以字节为单位)。默认值为 5242880(5 MB)。此限制考虑了 Lambda Function URL 的 6 MB 请求大小限制,并为 headers 和元数据留有安全余量。 |
| `DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_ENABLED` | 是否在下发调用前执行激活(activation)预检。启用后,daemon 会调用 `POST /v1/activation/activate` 唤醒被缩容至零的插件,并等待其就绪。默认值为 `false`。 |
| `DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_TIMEOUT` | 激活预检中,daemon 等待插件就绪的超时时间(秒)。若插件未在超时时间内被唤醒,则本次调用按失败处理。默认值为 `60`。 |

---

Expand Down Expand Up @@ -130,6 +132,42 @@ endpoint=http://...,name=...,id=...

---

### `POST /v1/activation/activate`

可选的激活预检。当 `DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_ENABLED` 为 `true` 时,daemon 会在下发调用前请求该接口,用于唤醒可能已被缩容至零的插件,并阻塞等待插件就绪。该接口可在每次调用前安全调用;运行时应对底层的活跃度/租约写入做节流。

**请求**

```http
POST /v1/activation/activate
Authorization: <API_KEY>
Content-Type: application/json

{
"instance_id": "string"
}
```

- `instance_id`(必填):目标插件在 connector 侧的 function name(即 `/v1/runner/instances` 与 `/v1/launch` 返回的 `Name`)。

**响应**

```json
{
"ready": true,
"endpoint": "string"
}
```

- `200 OK` 且 `ready = true`:插件已就绪,daemon 继续下发调用。
- `504 Gateway Timeout`(或 `ready = false`):插件未能在规定时间内就绪。

**错误处理**

- daemon 使用 `DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_TIMEOUT` 限制等待时长。一旦超时或收到非 `200` 响应,本次调用将被中断并按失败处理。

---

## 🔁 通信时序图(ASCII)

```text
Expand Down
83 changes: 83 additions & 0 deletions internal/core/serverless_connector/activation.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
package serverless

import (
"context"
"fmt"
"io"
"net/http"
"net/url"
"time"

"github.com/langgenius/dify-plugin-daemon/pkg/utils/http_requests"
"github.com/langgenius/dify-plugin-daemon/pkg/utils/parser"
)

type activationRequest struct {
InstanceID string `json:"instance_id"`
}

type activationResponse struct {
Ready bool `json:"ready"`
Endpoint string `json:"endpoint,omitempty"`
}

// ErrActivationTimeout indicates the plugin was not woken up and ready before the
// activation deadline elapsed. Callers should treat it as a failed invocation.
var ErrActivationTimeout = fmt.Errorf("timed out waiting for plugin to become ready")

// Activate performs the activation preflight against the serverless connector.
// It wakes a scaled-to-zero plugin (renewing its activity lease) and blocks until
// the plugin reports ready. instanceID is the connector function name of the
// target plugin. timeout bounds how long the daemon waits for readiness; the
// connector applies its own backstop as well.
//
// It returns nil once the plugin is ready, ErrActivationTimeout when the plugin
// is not ready within the window, or a wrapped error for any other failure.
func Activate(ctx context.Context, instanceID string, timeout time.Duration) error {
activateURL, err := url.JoinPath(baseurl.String(), "/v1/activation/activate")
if err != nil {
return err
}

if timeout > 0 {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, timeout)
defer cancel()
}

response, err := http_requests.Request(
client,
activateURL,
"POST",
http_requests.HttpContext(ctx),
http_requests.HttpHeader(map[string]string{
"Authorization": SERVERLESS_CONNECTOR_API_KEY,
}),
http_requests.HttpPayloadJson(activationRequest{InstanceID: instanceID}),
)
if err != nil {
if ctx.Err() != nil {
return ErrActivationTimeout
}
return fmt.Errorf("failed to request serverless connector activation: %w", err)
}
Comment thread
wylswz marked this conversation as resolved.
defer response.Body.Close()

body, _ := io.ReadAll(io.LimitReader(response.Body, 4*1024))
Comment thread
wylswz marked this conversation as resolved.
Outdated

switch response.StatusCode {
case http.StatusOK:
parsed, err := parser.UnmarshalJsonBytes[activationResponse](body)
if err != nil {
return fmt.Errorf("failed to parse activation response: %w", err)
}
if !parsed.Ready {
return ErrActivationTimeout
}
return nil
case http.StatusGatewayTimeout:
return ErrActivationTimeout
default:
return fmt.Errorf("unexpected response from serverless connector activation: status=%d body=%s", response.StatusCode, string(body))
}
}
26 changes: 26 additions & 0 deletions internal/core/serverless_runtime/io.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import (
"time"

"github.com/langgenius/dify-plugin-daemon/internal/core/io_tunnel/access_types"
serverless "github.com/langgenius/dify-plugin-daemon/internal/core/serverless_connector"
"github.com/langgenius/dify-plugin-daemon/pkg/entities"
"github.com/langgenius/dify-plugin-daemon/pkg/entities/plugin_entities"
routinepkg "github.com/langgenius/dify-plugin-daemon/pkg/routine"
Expand Down Expand Up @@ -363,6 +364,31 @@ func (r *ServerlessPluginRuntime) WriteContext(
l.Close()
}()

if r.ActivationEnabled {
if err := serverless.Activate(
requestCtx,
r.LambdaName,
time.Duration(r.ActivationTimeout)*time.Second,
); err != nil {
if ctx.Err() != nil {
sendEnd = false
return
}
log.Error(
"serverless runtime activation preflight failed",
"session_id", sessionId,
"action", action,
"lambda_name", r.LambdaName,
"error", err,
)
sendError(plugin_entities.ErrorResponse{
ErrorType: "PluginDaemonInnerError",
Message: fmt.Sprintf("Failed to activate plugin before invocation: %v", err),
})
return
}
}

url += "?action=" + string(action)
response, err := r.invokeServerlessWithRetry(requestCtx, url, sessionId, data, action)
if err != nil {
Expand Down
9 changes: 9 additions & 0 deletions internal/core/serverless_runtime/type.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,13 @@ type ServerlessPluginRuntime struct {
MaxRetryTimes int // maximum retry times for serverless invocation
MaxRequestBytes int // maximum serialized request payload size

// ActivationEnabled turns on the activation preflight: before each invocation
// the runtime asks the connector to wake the plugin and waits until ready.
ActivationEnabled bool
// ActivationTimeout bounds, in seconds, how long the preflight waits for the
// plugin to become ready before treating the invocation as failed.
ActivationTimeout int

RuntimeBufferSize int
RuntimeMaxBufferSize int
}
Expand Down Expand Up @@ -65,6 +72,8 @@ func ConstructServerlessPluginRuntime(
PluginMaxExecutionTimeout: config.PluginMaxExecutionTimeout,
MaxRetryTimes: config.MaxServerlessRetryTimes,
MaxRequestBytes: config.MaxServerlessRequestBytes,
ActivationEnabled: config.DifyPluginServerlessConnectorActivationEnabled,
ActivationTimeout: config.DifyPluginServerlessConnectorActivationTimeout,
RuntimeBufferSize: config.PluginRuntimeBufferSize,
RuntimeMaxBufferSize: config.PluginRuntimeMaxBufferSize,

Expand Down
15 changes: 15 additions & 0 deletions internal/types/app/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,17 @@ type Config struct {
DifyPluginServerlessConnectorAPIKey *string `envconfig:"DIFY_PLUGIN_SERVERLESS_CONNECTOR_API_KEY"`
DifyPluginServerlessConnectorLaunchTimeout int `envconfig:"DIFY_PLUGIN_SERVERLESS_CONNECTOR_LAUNCH_TIMEOUT"`

// Activation preflight: before dispatching an invocation to a serverless plugin,
// call the connector's activate endpoint to wake a scaled-to-zero plugin and
// block until it is ready. Disabled by default so it has no effect until
// explicitly enabled.
DifyPluginServerlessConnectorActivationEnabled bool `envconfig:"DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_ENABLED" default:"false"`
// DifyPluginServerlessConnectorActivationTimeout bounds, in seconds, how long
// the daemon waits for the plugin to become ready during the activation
// preflight. If the plugin is not woken up within this window the invocation
// is treated as failed.
DifyPluginServerlessConnectorActivationTimeout int `envconfig:"DIFY_PLUGIN_SERVERLESS_CONNECTOR_ACTIVATION_TIMEOUT"`

MaxServerlessRetryTimes int `envconfig:"MAX_SERVERLESS_RETRY_TIMES" default:"3"`
MaxServerlessRequestBytes int `envconfig:"MAX_SERVERLESS_REQUEST_BYTES" default:"5242880"`
MaxPluginPackageSize int64 `envconfig:"MAX_PLUGIN_PACKAGE_SIZE" validate:"required"`
Expand Down Expand Up @@ -320,6 +331,10 @@ func (c *Config) Validate() error {
if c.MaxServerlessTransactionTimeout == 0 {
return fmt.Errorf("max serverless transaction timeout is empty")
}

if c.DifyPluginServerlessConnectorActivationEnabled && c.DifyPluginServerlessConnectorActivationTimeout <= 0 {
return fmt.Errorf("dify plugin serverless connector activation timeout must be greater than zero when activation is enabled")
}
Comment on lines +335 to +337
case PLATFORM_LOCAL:
if c.PluginWorkingPath == "" {
return fmt.Errorf("plugin working path is empty")
Expand Down
1 change: 1 addition & 0 deletions internal/types/app/default.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ func (config *Config) SetDefault() {
setDefaultInt(&config.PluginMediaCacheSize, 1024)
setDefaultInt(&config.PluginAssetCacheSize, 256)
setDefaultInt(&config.DifyPluginServerlessConnectorLaunchTimeout, 240)
setDefaultInt(&config.DifyPluginServerlessConnectorActivationTimeout, 60)
setDefaultInt(&config.PluginRemoteInstallingMaxSingleTenantConn, 5)
setDefaultString(&config.DBSslMode, "disable")
setDefaultString(&config.PluginStorageLocalRoot, "storage")
Expand Down
Loading