Skip to content

Feat optimize api/worker #258

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Feat optimize api/worker #258

wants to merge 10 commits into from

Conversation

freehere107
Copy link
Collaborator

No description provided.

Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR optimizes background workers and API handling by introducing a periodic cron task, batching DAO operations, extending pagination, and updating API definitions and dependencies.

  • Add a cron job in the observer to refresh script metadata every 3 minutes.
  • Refactor DAO methods for batch inserts and cursor-based pagination on events/extrinsics.
  • Extend API docs with after_id, hidden_params, transaction_id and bump key module versions.

Reviewed Changes

Copilot reviewed 64 out of 64 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
internal/observer/observer.go Import cron, add RunCron function and schedule it alongside Subscribe.
internal/observer/go-worker.go Record worker processing time metrics and parse indices via model.ParseExtrinsicOrEventIndex.
internal/dao/runtimeVersion_test.go Update test to match new CreateRuntimeVersion signature.
internal/dao/runtimeVersion.go Change CreateRuntimeVersion to use context, block number and return bool.
internal/dao/db.go Inject Dao into DbStorage and add GetBlocksByNums/GetCurrentBlockNum.
internal/dao/dao_test.go Update DAO test calls for events, extrinsics, and runtime version.
internal/dao/dao.go Pass DAO reference into DbStorage.
internal/dao/chainExtrinsic_test.go Adapt tests to new CreateExtrinsic and GetExtrinsicList signatures.
internal/dao/chainExtrinsic.go Batch insert extrinsics and support afterId/fixedTableIndex paging.
internal/dao/chainEvent_test.go Adapt tests to new CreateEvent and GetEventList signatures.
internal/dao/chainEvent.go Batch insert events and support afterId/fixedTableIndex paging.
internal/dao/chainBlock.go Conditionally set ENGINE=InnoDB and add GetBlocksByNums.
internal/dao/IDao.go Update interface signatures for event/extrinsic/runtime version methods.
go.mod Bump versions for scale.go, subscan-plugin, cron, cli, gorm and others.
docs/api/swagger.yaml Add transaction_id, after_id, and hidden_params to schemas.
docs/api/swagger.json Mirror new schema fields from YAML in JSON.
docs/api/docs.go Mirror new schema fields in Go docs template.
cmd/main.go Start Prometheus metrics server on Before and shutdown on After.
cmd/commands.go Add plugin subcommand, set network context.
.github/workflows/docker.yaml Include feat/* branches in Docker CI triggers.
Comments suppressed due to low confidence (1)

internal/dao/IDao.go:36

  • [nitpick] The parameter name u is ambiguous. Rename it to something more descriptive, e.g., signedExtrinsicCount, to clarify its purpose.
CreateExtrinsic(c context.Context, txn *GormDB, extrinsic []model.ChainExtrinsic, u int) error

perrornet and others added 2 commits July 19, 2025 10:46
Fixed an issue where if a block had other system events besides ExtrinsicSuccess/Failed it would always result failed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants