Skip to content

fix(server): prevent graph clear API from clearing meta table in MySQL backend storage #2888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
LYD031106 wants to merge 8 commits into
base: release-1.5.0
Choose a base branch
from
Open

fix(server): prevent graph clear API from clearing meta table in MySQL backend storage #2888

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
e376b6c
fix(server): ensure graph clear API does not clear meta table
LYD031106 Oct 24, 2025
45d95fe
test(server): add MySQL integration tests for server module
LYD031106 Nov 16, 2025
22b2407
fix(server): ensure graph clear API does not clear system table
LYD031106 Nov 19, 2025
d7f35f1
fix(server): Remove invalid references
LYD031106 Nov 20, 2025
4ad56cb
fix(server): ensure graph clear API does not clear meta table
LYD031106 Nov 30, 2025
40e0314
fix(server): ensure graph clear API does not clear meta table
LYD031106 Dec 6, 2025
ea75f05
fix(server): ensure graph clear API does not clear meta table
LYD031106 Dec 6, 2025
e14eb0f
feat: CI/CD流程调整
LYD031106 Jan 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions ...er/hugegraph-mysql/src/main/java/org/apache/hugegraph/backend/store/mysql/MysqlStore.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@ public void truncate() {
this.checkOpened();

this.truncateTables();
this.init();
Copy link
Member

@imbajin imbajin Oct 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Important: Check consistency with other backend stores

The PR description mentions that "HugeGraph only supports hstore, rocksdb, hbase, and memory" but this fix is only applied to MySQL backend.

Questions to verify:

  1. Do RocksDB, HBase, and Memory backends have the same issue?
  2. From my quick check, HBase's truncate() method doesn't call init() after truncating tables - why is MySQL different?
  3. Is there a fundamental difference in how MySQL handles the meta table vs other backends?

Recommendation:

  • Document WHY MySQL requires this init() call while other backends don't
  • Verify other supported backends (RocksDB, HBase) don't have similar issues
  • Consider adding a comment explaining the MySQL-specific requirement:
this.truncateTables();
// MySQL-specific: Re-initialize tables to restore meta table
// Other backends handle meta table separately during truncation
this.init();

Copy link
Author

@LYD031106 LYD031106 Oct 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Root Cause
The issue stems from the fact that backend stores such as MySQL, HBase, and RocksDB persist metadata (e.g., driverVersion) to the backend storage. When the truncate() operation is executed to clear all table data, this metadata is also deleted. As a result, subsequent operations fail due to missing metadata or state conflicts.

Analysis by Backend

  • RocksDB:
    Although affected by the same problem, its truncate() method explicitly calls this.init() after clearing data to reinitialize the metadata, thereby avoiding errors.
    @Override
    public synchronized void truncate() {
        Lock writeLock = this.storeLock.writeLock();
        writeLock.lock();
        try {
            this.checkOpened();

            this.clear(false);
            this.init();
            // Clear write-batch
            this.dbs.values().forEach(BackendSessionPool::forceResetSessions);
            LOG.debug("Store truncated: {}", this.store);
        } finally {
            writeLock.unlock();
        }
    }
  • HBase:
    Confirmed to have the same issue as MySQL.
    Action Required: A follow-up PR is needed to fix the HBase backend accordingly.
  • Memory / HStore:
    These backends are not affected because they do not persist critical version information or similar metadata to the backend storage.

Copy link
Member

@imbajin imbajin Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Root Cause The issue stems from the fact that backend stores such as MySQL, HBase, and RocksDB persist metadata (e.g., driverVersion) to the backend storage. When the truncate() operation is executed to clear all table data, this metadata is also deleted. As a result, subsequent operations fail due to missing metadata or state conflicts.

Analysis by Backend

  • RocksDB:
    Although affected by the same problem, its truncate() method explicitly calls this.init() after clearing data to reinitialize the metadata, thereby avoiding errors.
    @Override
    public synchronized void truncate() {
        Lock writeLock = this.storeLock.writeLock();
        writeLock.lock();
        try {
            this.checkOpened();

            this.clear(false);
            this.init();
            // Clear write-batch
            this.dbs.values().forEach(BackendSessionPool::forceResetSessions);
            LOG.debug("Store truncated: {}", this.store);
        } finally {
            writeLock.unlock();
        }
    }
  • HBase:
    Confirmed to have the same issue as MySQL.
    Action Required: A follow-up PR is needed to fix the HBase backend accordingly.
  • Memory / HStore:
    These backends are not affected because they do not persist critical version information or similar metadata to the backend storage.

@LYD031106 LGTM, we could add a basic test & comment for the above issue (and we could merge this PR soon)

BTW, this week we prepare to release the new version (1.7.0)

Copy link
Author

@LYD031106 LYD031106 Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have submitted a MySQL compatibility fix to the release-1.5.0 branch. Since this is a legacy compatibility patch, I will submit a separate PR to the master branch to address the HBase storage issues.
Regarding the addition of test cases: I’ve noticed that hugegraph-test currently lacks backend-specific tests for HBase and MySQL. I plan to add a minimal set of regression tests. However, I’m not very familiar with how to build such backend-specific test cases. Could you please provide some guidance or reference examples?

Copy link
Member

@imbajin imbajin Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have submitted a MySQL compatibility fix to the release-1.5.0 branch. Since this is a legacy compatibility patch, I will submit a separate PR to the master branch to address the HBase storage issues. Regarding the addition of test cases: I’ve noticed that hugegraph-test currently lacks backend-specific tests for HBase and MySQL. I plan to add a minimal set of regression tests. However, I’m not very familiar with how to build such backend-specific test cases. Could you please provide some guidance or reference examples?

refer ci steps: https://github.com/apache/incubator-hugegraph/actions/runs/18783126996/workflow?pr=2888#L49
image

Copy link
Member

@imbajin imbajin Oct 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Minor: Consider side effects of calling init()

The init() method does more than just initTables():

  • It calls checkClusterConnected()
  • Creates database if not exists (sessions.createDatabase())
  • Opens a new session
  • Then calls checkOpened() again

Observation:
Most of these operations are already done and may be redundant when called from truncate() (which already calls checkOpened() first).

Potential issue:
If there are any idempotency concerns with these operations being called multiple times, this could introduce subtle bugs.

Suggestion:
Review whether all operations in init() are safe to call multiple times during the store lifecycle, or extract just the table initialization logic into a separate method.

Copy link
Member

@imbajin imbajin Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Important: Verify init() is necessary after truncate

Adding this.init() after truncation appears to reinitialize the meta table. However:

  1. Does init() have any side effects beyond restoring meta data?
  2. Is this call idempotent and safe to call multiple times?
  3. Should this also be added to clearTables() method for consistency?

Please verify this is the correct approach and consider adding a comment explaining why re-initialization is needed after truncate.

LOG.debug("Store truncated: {}", this.store);
}

Expand Down
Loading