[Feature Request] Extract cross-component shared Go libraries into a top-level libs/ directory

[fslongjin]

Problem / Motivation

There are multiple compile-time dependencies between sibling components in the Go modules of this monorepo, implemented via go.mod replace directives pointing at sibling directories. As a result, "deployable components" and "shared libraries" are indistinguishable in the directory layout, which leads to a series of downstream costs in builds, CI, and releases.

Current dependency graph

(The dashed box marks pure shared libraries — no cmd/main, not deployable.)

graph TD
    subgraph components ["Deployable components"]
        CubeMaster["CubeMaster"]
        CubeOps["CubeOps"]
        Cubelet["Cubelet"]
        ExampleRPC["examples/volume/cos/rpc"]
    end

    subgraph libs ["Pure shared libraries (no cmd/main, not deployable)"]
        CubeDB["CubeDB"]
        cubelog["cubelog"]
        cubevs["CubeNet/cubevs"]
    end

    CubeMaster --> CubeDB
    CubeMaster --> cubelog
    CubeMaster --> Cubelet
    CubeMaster -.-> cubevs

    CubeOps --> CubeDB
    CubeOps --> cubelog

    Cubelet --> cubevs
    Cubelet --> cubelog

    ExampleRPC --> Cubelet

    classDef lib fill:#f6f8fa,stroke:#999,stroke-dasharray: 5 5;
    class CubeDB,cubelog,cubevs lib;

Loading

How each dependency is actually used (all via go.mod replace pointing at sibling directories):

Dependent	Depends on	Actual usage	Evidence
CubeMaster	CubeDB (entire module)	The whole data layer: dao / driver / migrate, 7 files	CubeMaster/pkg/base/db/db.go:21 etc.
CubeMaster	cubelog	Logging / tracing, 60 files	—
CubeMaster	Cubelet (entire module)	Only 1 file uses 2 lightweight packages: volumeplugin pb + grpctarget	CubeMaster/pkg/volume/plugin/rpc/driver.go:19-20
CubeMaster	CubeNet/cubevs	Dead config: replaced but never required	CubeMaster/go.mod:191
CubeOps	CubeDB (entire module)	5 files importing CubeDB/dao etc.	CubeOps/internal/store/db.go:10-13
CubeOps	cubelog	Logging	CubeOps/go.mod:18
Cubelet	CubeNet/cubevs	eBPF datapath contract, 8 files	Cubelet/network/runtime/cubevs_adapter.go:13
Cubelet	cubelog	Logging / tracing, 98 files	Cubelet/go.mod:210
examples/volume/cos/rpc	Cubelet (entire module)	The exact same 2 packages as CubeMaster	examples/volume/cos/rpc/go.mod:23

The problems boil down to two points:

1. Shared libraries are mixed in as top-level components

• CubeDB and cubelog are pure libraries: no cmd/main, not deployable
• Yet they sit at the top level alongside real components like CubeMaster and Cubelet
• The directory layout gives no way to tell "components" from "libraries"

2. Components depend on components

• CubeMaster only uses 2 small packages inside Cubelet (the volumeplugin proto contract and one utility function)
• Yet it replaces the entire Cubelet module for them
• Inheriting Cubelet's full heavy dependency set (k8s, etc.)

Downstream costs this creates:

• Docker build contexts must be the repository root, with Dockerfiles explicitly COPY-ing sibling directories (CubeMaster/docker/Dockerfile:20-29; the comment at Cubelet/Dockerfile:6 says "Build context MUST be the repository root").
• Release builds are forced to serialize: Cubelet's make proto rewrites .pb.go files in place while the CubeMaster compiler is reading them (deploy/one-click/build-release-bundle-builder.sh:85-98: "can tear a .pb.go while the cubemaster compiler is reading it").
• CI cross-component mappings are maintained by hand and have gaps (.github/workflows/unit-test-check.yml:170-200): CubeDB changes re-run CubeOps tests, but CubeMaster — which equally depends on CubeDB/Cubelet/cubelog — is not covered.
• Dead config accumulates: the cubevs replace in CubeMaster/go.mod has no matching require; migration-check.yml has a path trigger pointing at the no-longer-existing CubeMaster/pkg/base/dao/migrate/**.

Proposed Solution

Introduce a top-level libs/ directory to gather shared libraries, with the target layout:

.
├── CubeMaster/            # deployable components
├── Cubelet/
├── CubeOps/
├── ...
└── libs/                  # cross-component shared libraries: used by ≥2 components, no deployable artifacts
    ├── cubelog/           # ← cubelog/
    ├── cubedb/            # ← CubeDB/
    └── volumeplugin/      # ← Cubelet/api/services/volumeplugin/v1
                           #   + Cubelet/plugins/volume/grpctarget

Dependencies after the change:

graph TD
    subgraph components ["Deployable components"]
        CubeMaster["CubeMaster"]
        CubeOps["CubeOps"]
        Cubelet["Cubelet"]
        ExampleRPC["examples/volume/cos/rpc"]
    end

    subgraph libs ["libs/"]
        cubedb["cubedb"]
        cubelog["cubelog"]
        volumeplugin["volumeplugin"]
    end

    cubevs["CubeNet/cubevs (unchanged)"]

    CubeMaster --> cubedb
    CubeMaster --> cubelog
    CubeMaster --> volumeplugin
    CubeOps --> cubedb
    CubeOps --> cubelog
    Cubelet --> cubelog
    Cubelet --> cubevs
    ExampleRPC --> volumeplugin

    classDef lib fill:#f6f8fa,stroke:#999,stroke-dasharray: 5 5;
    class cubedb,cubelog,volumeplugin lib;

Loading

Key change: CubeMaster no longer replaces Cubelet, and shared dependencies are consolidated under libs/. cubevs is the eBPF datapath contract between Cubelet and CubeNet and is intentionally left out of this proposal.

Alternatives Considered

• Keep as-is + document it: lowest cost, but the root build context, serialized releases, and CI mapping gaps remain, and the component-vs-library ambiguity persists.
• go.work only, no directory changes: simplifies local development, but does not solve CubeMaster depending on the entire Cubelet module, nor CubeDB/cubelog sitting at the top level.
• Publish versioned modules to a remote: full decoupling, but a heavy release process and slower iteration; a local libs/ is a prerequisite step toward it, not a conflict.

Additional Context

• Extraction is low-risk: the two Cubelet packages CubeMaster actually imports are both leaf packages (generated pb code + a utility package depending only on strings), with no transitive dependency burden.
• Naming: libs/ vs pkgs/ — open for discussion.

[cubesandboxbot]

This issue might overlap with existing issues. Please check:

• CubeMaster go.mod: transitive dep CubeNet/cubevs not resolvable — replace doesn't propagate from Cubelet, go list -m all fails #1349: CubeMaster go.mod: transitive dep CubeNet/cubevs not resolvable — replace doesn't propagate from Cubelet, go list -m all fails (closed bug). This is a concrete failure caused by the same CubeMaster ↔ Cubelet ↔ cubevs replace-based dependency structure that this proposal aims to eliminate at the root (see the "Dead config accumulates" point about CubeMaster/go.mod).
• [Feature Request] Refactor control plane API architecture: unify protocol definitions, eliminate redundant types, centralize validation #1023: [Feature Request] Refactor control plane API architecture: unify protocol definitions, eliminate redundant types, centralize validation (open). Both proposals involve extracting cross-component shared code into a shared Go module consumed via require + replace (an api/ module there vs a libs/ directory here) — worth coordinating so the two restructuring efforts don't conflict on module layout.

These are related rather than exact duplicates; flagging them so maintainers can cross-link.