cubemaster: fix integration tests panicking on unopened dao - #1383
cubemaster: fix integration tests panicking on unopened dao#1383aptacc2421 wants to merge 2 commits into
Conversation
| // mocktest_daoConfig mirrors the DBConfig -> dao.Config mapping that | ||
| // app.initDatabaseSchema performs at startup, so mock_db can establish the | ||
| // shared dao handle before db.Init (a dao.Default() wrapper) needs it. | ||
| func mocktest_daoConfig() dao.Config { |
There was a problem hiding this comment.
This is a field-for-field copy of the DBConfig → dao.Config mapping in app.initDatabaseSchema (cmd/cubemaster/app/main.go:242-255). The fix's correctness depends on dao.Open's identity-dedup (CubeDB/dao/dao.go:76-84), so these two copies must never drift: if a field is added to config.DBConfig/dao.Config and updated in only one place, the second dao.Open from app.Run stops being a no-op and instead fails with dao: already opened with ... (requested ...) — an opaque test-only failure far from the real cause.
Consider extracting the mapping into a single shared helper (e.g. dao.ConfigFromDBConfig(*config.DBConfig), or a ToDAOConfig() method on config.DBConfig) and calling it from both initDatabaseSchema and mock_db, so the idempotence guarantee holds by construction rather than by coincidence.
Review: cubemaster: fix integration tests panicking on unopened dao (#1383)SummaryThis PR fixes #1372. The fix:
VerdictNo blocking issues. The fix is correct and addresses the reported panic in both affected paths (integration tests and mock-debug mode). Verified against the base tree:
Minor observations (non-blocking)
Tests
This review was generated by an AI assistant. |
18157c4 to
a11655f
Compare
|
Addressed the review feedback: the DBConfig → dao.Config mapping is now extracted into db.ConfigFromDBConfig in pkg/base/db and used by both app startup (initDatabaseSchema) and the integration/mock-debug bootstrap (mock_db), so dao.Open's identity-dedup cannot drift between the two call sites. Added a focused unit test for the mapping; TestDemo still passes against a reachable MySQL/Redis. |
| // through this helper — keeping the two mappings from drifting. | ||
| func ConfigFromDBConfig(src *config.DBConfig) (dao.Config, error) { | ||
| if src == nil { | ||
| return dao.Config{}, errors.New("instance_db_config is not set") |
There was a problem hiding this comment.
The error text instance_db_config is not set is specific to the app's config shape, but this helper lives in the generic pkg/base/db package and takes an arbitrary *config.DBConfig. Accurate for the two current call sites (both pass the instance DB config), but a neutral message such as errors.New("db config is nil") would keep the helper reusable without leaking a caller-specific field name. Non-blocking.
a11655f to
1ebb73d
Compare
| MigrationLockTimeoutSeconds: src.MigrationLockTimeoutSeconds, | ||
| daoCfg, err := db.ConfigFromDBConfig(cfg.InstanceDBConfig) | ||
| if err != nil { | ||
| return fmt.Errorf("dao: %w", err) |
There was a problem hiding this comment.
Non-blocking: this refactor drops the actionable error text for a missing DB config. The old path reported dao: instance_db_config is not set — it names the exact yaml key an operator must add. The new path surfaces db config is nil, and because coreInit wraps initDatabaseSchema errors as dao migrate: %w, the final log reads dao migrate: dao: db config is nil — "dao" appears twice and the config key is gone.
No caller matches on the old string, and in the app this path is unreachable with a nil config because coreInit guards cfg.InstanceDBConfig == nil first. But mock-debug mode reaches ConfigFromDBConfig with whatever config.GetDbConfig() returns, so the message matters there. Keeping the field name in ConfigFromDBConfig's error (e.g. instance_db_config is not set / db config is nil) would preserve operator diagnostics.
| // path that the app bootstrap runs before tests. | ||
| daoCfg, err := db.ConfigFromDBConfig(config.GetDbConfig()) | ||
| if err != nil { | ||
| stdlog.Fatalf("integration: dao config fail: %v", err) |
There was a problem hiding this comment.
Non-blocking robustness note: this fix rests on dao.Open being idempotent for this config identity, and dao.Open keys identity on the full config — including Pwd, Addr, and every timeout/pool field (identityFromConfig in CubeDB/dao/dao.go). Both call sites read the same cfg.InstanceDBConfig global today, so they agree. But config.Init() starts a hotswap watcher (hotswap/file.go) that atomically swaps cfg on a config-file write; a write landing in the window between MockInit() and initDatabaseSchema would make the app's second dao.Open fail startup with dao: already opened with .... The window is tiny and unlikely in practice, but a one-line comment here (and in initDatabaseSchema) noting that the two opens must be built from a single config snapshot would make the coupling explicit and cheaper to debug.
PR TencentCloud#984 turned pkg/base/db.Init into a thin wrapper over dao.Default(), which panics with "dao: Open has not been called yet" until the shared handle is established by dao.Open at app startup. Both the integration test bootstrap (TestMain) and mock-debug mode (cmd/cubemaster/main.go) call MockInit() before app.Run(), so mock_db() reached dao.Default() before any Open. Open the shared dao handle in mock_db() from the loaded DB config first. dao.Open is idempotent for the same identity, so the later open in app.Run()'s initDatabaseSchema is a no-op and schema migration still runs there. The DBConfig -> dao.Config mapping is extracted into db.ConfigFromDBConfig and shared with app startup, so dao.Open's identity-dedup cannot drift between the two call sites. Verified: TestRound passes against a reachable MySQL; without a database the package fails with a clean connection error instead of panicking. Fixes TencentCloud#1372 Assisted-by: DeepSeek Harness:deepseek-v4-flash Signed-off-by: Xinhong Hu <tp5092@foxmail.com>
1ebb73d to
da743e6
Compare
|
Addressed both new non-blocking notes:
Build and the ConfigFromDBConfig unit test pass. |
| // mock-debug bootstrap: config hotswap could otherwise change the | ||
| // identity between the two dao.Open calls and the second one would | ||
| // fail with "dao: already opened with ... (requested ...)". | ||
| daoCfg, err := db.ConfigFromDBConfig(cfg.InstanceDBConfig) |
There was a problem hiding this comment.
Minor precision nit on the comment: the two call sites don't actually read the same snapshot. mock_db() reads the live global via config.GetDbConfig(), while this reads cfg.InstanceDBConfig from the cfg captured at Run() start. What ConfigFromDBConfig guarantees is the same mapping, not the same snapshot — if a config hot-reload landed in the window between MockInit() and app.Run(), the two identities would still differ and this second dao.Open would fail with already opened with ... (requested ...). The comment in mock_init.go already acknowledges that window honestly; consider rewording to "same mapping as" so the claim here doesn't overstate the protection.
| // identity, so every call site that opens the handle before app startup | ||
| // (schema init, integration/mock-debug bootstrap) must build the dao config | ||
| // through this helper — keeping the two mappings from drifting. | ||
| func ConfigFromDBConfig(src *config.DBConfig) (dao.Config, error) { |
There was a problem hiding this comment.
Trivial: the helper is generic over *config.DBConfig, but the nil error message hardcodes the instance-DB field name (instance_db_config is not set), and the wrap in initDatabaseSchema doubles the prefix (dao: db config is nil: instance_db_config is not set). A neutral message such as db config is nil would keep the helper accurate if it is ever reused for another DB config. Note this also changes the operator-visible error text from the previous dao: instance_db_config is not set.
90cd67a to
1dbbd4d
Compare
| // "dao: already opened with ... (requested ...)". | ||
| daoCfg, err := db.ConfigFromDBConfig(cfg.InstanceDBConfig) | ||
| if err != nil { | ||
| return fmt.Errorf("dao: %w", err) |
There was a problem hiding this comment.
Minor diagnostic regression: the removed fmt.Errorf("dao: instance_db_config is not set") named the exact YAML key an operator must set, while this now surfaces the vaguer "dao: db config is nil" from ConfigFromDBConfig(nil). Consider preserving the key-specific wording — e.g., have ConfigFromDBConfig return an error that mentions instance_db_config, or keep a src == nil guard here so the message stays actionable.
| // make the second dao.Open fail with "dao: already opened with ...". | ||
| daoCfg, err := db.ConfigFromDBConfig(config.GetDbConfig()) | ||
| if err != nil { | ||
| stdlog.Fatalf("integration: dao config fail: %v", err) |
There was a problem hiding this comment.
mocktest_Ctx is the mock-lifecycle context, cancelled at the end of TestMain. Opening the process-lifetime shared dao handle with it semantically ties the global handle to the mock's teardown; it's benign today because both drivers ignore ctx in Open, but context.Background() would express the handle's actual lifetime more accurately.
| // db.Init is a thin wrapper that returns dao.Default(), which panics | ||
| // with "dao: Open has not been called yet" until the shared handle is | ||
| // established by dao.Open. Open it here from the loaded DB config; | ||
| // dao.Open is idempotent for the same identity, so the later | ||
| // dao.Open in app.Run's startup path (initDatabaseSchema) is a no-op. | ||
| // Schema (including t_cube_host_type) is owned by the dao.Migrate | ||
| // path that the integration test bootstrap runs before tests. | ||
| // path that the app bootstrap runs before tests. The dao config must | ||
| // be built from the same snapshot as initDatabaseSchema: config | ||
| // hotswap between MockInit and app.Run could change the identity and | ||
| // make the second dao.Open fail with "dao: already opened with ...". |
There was a problem hiding this comment.
Thanks for the contribution! The overall behavioral change looks good to me. However, I think we can clean up the comments here a bit. We should focus on describing what the code does, rather than explaining the rationale behind this panic. Otherwise, it might confuse future readers and comes across as noise.
- initDatabaseSchema comment overstated the protection: the two
dao.Open call sites share the same DBConfig -> dao.Config mapping
via ConfigFromDBConfig, but read separate config snapshots (mock_db
reads the live global via config.GetDbConfig(), app startup reads
the cfg captured at Run() start). A hotswap landing between MockInit
and app.Run would still change the identity and fail the second
dao.Open; reword the comment to claim only the shared mapping.
- ConfigFromDBConfig's nil error hardcoded the instance-DB field name
and doubled the prefix after wrapping ("dao: db config is nil:
instance_db_config is not set"). Use a neutral "db config is nil" so
the helper stays accurate if reused for another DB config. This
changes the operator-visible error text from the previous
"dao: instance_db_config is not set".
Assisted-by: CodeBuddy:kimi-k3-1
Signed-off-by: Xinhong Hu <tp5092@foxmail.com>
1dbbd4d to
ce9de8e
Compare
| } | ||
| if _, err := dao.Open(mocktest_Ctx, daoCfg); err != nil { | ||
| stdlog.Fatalf("dao open fail:%v", err) | ||
| } |
There was a problem hiding this comment.
The fix is correct for the two known bootstrap paths, but it leaves db.Init's contract fragile: Init still ignores its cfg argument and dao.Default() panics with ErrNotOpened if any future bootstrap path calls Init before dao.Open. The pre-condition is documented on Init, but nothing enforces it, so a third pre-dao.Open caller (a new integration harness, a test helper) would hit the same panic this PR fixes. Worth considering whether Init should fall back to opening the dao from the cfg it currently ignores when the handle is unopened — that would close the whole class of init-order bugs rather than this one site (balanced against the existing "fail loudly" intent).
| // identity, so every call site that opens the handle before app startup | ||
| // (schema init, integration/mock-debug bootstrap) must build the dao config | ||
| // through this helper — keeping the two mappings from drifting. | ||
| func ConfigFromDBConfig(src *config.DBConfig) (dao.Config, error) { |
There was a problem hiding this comment.
This helper is a second copy of the field list that dao.Config / identityFromConfig already owns. If dao.Config ever gains a field (e.g. the existing Extra), this mapping will silently omit it and the two bootstrap dao.Open identities could drift — the exact problem this PR sets out to prevent. Consider a round-trip test (config.DBConfig → dao.Config → identity) or moving the mapping next to dao.Config so the field list lives in one place.
Fixes #1372
Summary
CubeMaster's integration test package panics at TestMain on current master: pkg/base/db.Init is now a thin wrapper over dao.Default(), which panics with "dao: Open has not been called yet" until the shared handle is established by dao.Open at app startup. Both the integration test bootstrap (TestMain) and mock-debug mode (cmd/cubemaster/main.go) call MockInit() before app.Run(), so mock_db() reached dao.Default() before any Open.
What Changed
Open the shared dao handle in mock_db() from the loaded DB config first. dao.Open is idempotent for the same identity, so the later open in app.Run()'s initDatabaseSchema is a no-op and schema migration still runs there.
Validation
Assisted-by: DeepSeek Harness:deepseek-v4-flash