Skip to content
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@ public void truncate() {
this.checkOpened();

this.truncateTables();
this.init();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Important: Check consistency with other backend stores

The PR description mentions that "HugeGraph only supports hstore, rocksdb, hbase, and memory" but this fix is only applied to MySQL backend.

Questions to verify:

  1. Do RocksDB, HBase, and Memory backends have the same issue?
  2. From my quick check, HBase's truncate() method doesn't call init() after truncating tables - why is MySQL different?
  3. Is there a fundamental difference in how MySQL handles the meta table vs other backends?

Recommendation:

  • Document WHY MySQL requires this init() call while other backends don't
  • Verify other supported backends (RocksDB, HBase) don't have similar issues
  • Consider adding a comment explaining the MySQL-specific requirement:
this.truncateTables();
// MySQL-specific: Re-initialize tables to restore meta table
// Other backends handle meta table separately during truncation
this.init();

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Root Cause
The issue stems from the fact that backend stores such as MySQL, HBase, and RocksDB persist metadata (e.g., driverVersion) to the backend storage. When the truncate() operation is executed to clear all table data, this metadata is also deleted. As a result, subsequent operations fail due to missing metadata or state conflicts.

Analysis by Backend

  • RocksDB:
    Although affected by the same problem, its truncate() method explicitly calls this.init() after clearing data to reinitialize the metadata, thereby avoiding errors.
    @Override
    public synchronized void truncate() {
        Lock writeLock = this.storeLock.writeLock();
        writeLock.lock();
        try {
            this.checkOpened();

            this.clear(false);
            this.init();
            // Clear write-batch
            this.dbs.values().forEach(BackendSessionPool::forceResetSessions);
            LOG.debug("Store truncated: {}", this.store);
        } finally {
            writeLock.unlock();
        }
    }
  • HBase:
    Confirmed to have the same issue as MySQL.
    Action Required: A follow-up PR is needed to fix the HBase backend accordingly.
  • Memory / HStore:
    These backends are not affected because they do not persist critical version information or similar metadata to the backend storage.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Root Cause The issue stems from the fact that backend stores such as MySQL, HBase, and RocksDB persist metadata (e.g., driverVersion) to the backend storage. When the truncate() operation is executed to clear all table data, this metadata is also deleted. As a result, subsequent operations fail due to missing metadata or state conflicts.

Analysis by Backend

  • RocksDB:
    Although affected by the same problem, its truncate() method explicitly calls this.init() after clearing data to reinitialize the metadata, thereby avoiding errors.
    @Override
    public synchronized void truncate() {
        Lock writeLock = this.storeLock.writeLock();
        writeLock.lock();
        try {
            this.checkOpened();

            this.clear(false);
            this.init();
            // Clear write-batch
            this.dbs.values().forEach(BackendSessionPool::forceResetSessions);
            LOG.debug("Store truncated: {}", this.store);
        } finally {
            writeLock.unlock();
        }
    }
  • HBase:
    Confirmed to have the same issue as MySQL.
    Action Required: A follow-up PR is needed to fix the HBase backend accordingly.
  • Memory / HStore:
    These backends are not affected because they do not persist critical version information or similar metadata to the backend storage.

@LYD031106 LGTM, we could add a basic test & comment for the above issue (and we could merge this PR soon)

BTW, this week we prepare to release the new version (1.7.0)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have submitted a MySQL compatibility fix to the release-1.5.0 branch. Since this is a legacy compatibility patch, I will submit a separate PR to the master branch to address the HBase storage issues.
Regarding the addition of test cases: I’ve noticed that hugegraph-test currently lacks backend-specific tests for HBase and MySQL. I plan to add a minimal set of regression tests. However, I’m not very familiar with how to build such backend-specific test cases. Could you please provide some guidance or reference examples?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have submitted a MySQL compatibility fix to the release-1.5.0 branch. Since this is a legacy compatibility patch, I will submit a separate PR to the master branch to address the HBase storage issues. Regarding the addition of test cases: I’ve noticed that hugegraph-test currently lacks backend-specific tests for HBase and MySQL. I plan to add a minimal set of regression tests. However, I’m not very familiar with how to build such backend-specific test cases. Could you please provide some guidance or reference examples?

refer ci steps: https://github.com/apache/incubator-hugegraph/actions/runs/18783126996/workflow?pr=2888#L49
image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Minor: Consider side effects of calling init()

The init() method does more than just initTables():

  • It calls checkClusterConnected()
  • Creates database if not exists (sessions.createDatabase())
  • Opens a new session
  • Then calls checkOpened() again

Observation:
Most of these operations are already done and may be redundant when called from truncate() (which already calls checkOpened() first).

Potential issue:
If there are any idempotency concerns with these operations being called multiple times, this could introduce subtle bugs.

Suggestion:
Review whether all operations in init() are safe to call multiple times during the store lifecycle, or extract just the table initialization logic into a separate method.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Important: Verify init() is necessary after truncate

Adding this.init() after truncation appears to reinitialize the meta table. However:

  1. Does init() have any side effects beyond restoring meta data?
  2. Is this call idempotent and safe to call multiple times?
  3. Should this also be added to clearTables() method for consistency?

Please verify this is the correct approach and consider adding a comment explaining why re-initialization is needed after truncate.

LOG.debug("Store truncated: {}", this.store);
}

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
/*
*
* * Licensed to the Apache Software Foundation (ASF) under one or more
* * contributor license agreements. See the NOTICE file distributed with
* * this work for additional information regarding copyright ownership.
* * The ASF licenses this file to You under the Apache License, Version 2.0
* * (the "License"); you may not use this file except in compliance with
* * the License. You may obtain a copy of the License at
* *
* * http://www.apache.org/licenses/LICENSE-2.0
* *
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS,
* * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* * See the License for the specific language governing permissions and
* * limitations under the License.
*
*/

package org.apache.hugegraph.unit.mysql;

import org.apache.hugegraph.backend.store.BackendStore;
import org.apache.hugegraph.backend.store.hbase.HbaseStoreProvider;
import org.apache.hugegraph.backend.store.mysql.MysqlStoreProvider;
import org.apache.hugegraph.config.HugeConfig;
import org.apache.hugegraph.unit.BaseUnitTest;
import org.apache.hugegraph.unit.FakeObjects;
import org.junit.After;
import org.junit.Before;

public class BaseMysqlUnitTest extends BaseUnitTest {
protected BackendStore store;

protected HugeConfig config;
protected MysqlStoreProvider provider;

@Before
public void setup() {
this.config = FakeObjects.newConfig();

this.provider = new MysqlStoreProvider();

this.store = this.provider.loadSystemStore(config);
}

@After
public void down(){
if (this.store != null) {
try {
this.store.close();
} catch (Exception e) {
// pass
}
}
if (this.provider != null) {
try {
this.provider.close();
} catch (Exception e) {
// pass
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/*
*
* * Licensed to the Apache Software Foundation (ASF) under one or more
* * contributor license agreements. See the NOTICE file distributed with
* * this work for additional information regarding copyright ownership.
* * The ASF licenses this file to You under the Apache License, Version 2.0
* * (the "License"); you may not use this file except in compliance with
* * the License. You may obtain a copy of the License at
* *
* * http://www.apache.org/licenses/LICENSE-2.0
* *
* * Unless required by applicable law or agreed to in writing, software
* * distributed under the License is distributed on an "AS IS" BASIS,
* * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* * See the License for the specific language governing permissions and
* * limitations under the License.
*
*/

package org.apache.hugegraph.unit.mysql;

import org.apache.hugegraph.testutil.Assert;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class MysqlTest extends BaseMysqlUnitTest{
private static final String GRAPH_NAME = "test_graph";

@Before
public void setUp(){
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Important: Inconsistent test lifecycle annotations

The setUp() method in MysqlTest calls super.setup(), but the base class method setup() doesn't have @Before annotation. This means:

  1. The base class setup runs for every test in MysqlTest (due to @before here)
  2. But may not run correctly for other test classes extending BaseMysqlUnitTest
  3. The naming inconsistency (setup vs setUp) is confusing
Suggested change
public void setUp(){
// Either add @Before to the base class setup() method, or rename for consistency:
@Before
public void setup() {
super.setup();
}

this.provider.open(GRAPH_NAME);
this.provider.init();
}

@After
public void teardown(){
if (this.store != null) {
this.store.close();
}
if (this.provider != null) {
this.provider.close();
}
}

@Test
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Important: Incomplete test coverage

The test only verifies that the version is preserved, but doesn't verify:

  1. That data in non-meta tables IS actually cleared - The bug could still exist if truncate silently fails
  2. That the Counters table content is preserved (if that's intentional)
  3. That schema data is cleared - The fix might have side effects on other stores

Suggested additions:

@Test
public void testMysqlTruncateClearsDataButPreservesMeta() {
    // 1. Insert some test data into graph store
    // 2. Record meta version before truncate
    // 3. Call truncate
    // 4. Verify meta version unchanged
    // 5. Verify test data was actually cleared
    // 6. Verify Counters table state (if needed)
}

public void testMysqlMetaVersion(){
// init store
this.store.init();
String beforeVersion = this.store.storedVersion();
this.store.truncate();
String afterInitVersion = this.store.storedVersion();
Assert.assertNotNull(beforeVersion);
Assert.assertNotNull(afterInitVersion);
Assert.assertEquals(beforeVersion, afterInitVersion);
}
}
Loading