Cannot create knowledge base without tiflash

[634750802]

2024-12-02 13:28:27 2024-12-02 05:28:27,735 - app.api.admin_routes.knowledge_base.routes:90 - ERROR - (pymysql.err.OperationalError) (1105, 'the tiflash replica count: 1 should be less than the total tiflash server count: 0')
2024-12-02 13:28:27 [SQL: ALTER TABLE chunks_1 SET TIFLASH REPLICA 1]
2024-12-02 13:28:27 (Background on this error at: https://sqlalche.me/e/20/e3q8)
2024-12-02 13:28:27 Traceback (most recent call last):
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
2024-12-02 13:28:27     self.dialect.do_execute(
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 924, in do_execute
2024-12-02 13:28:27     cursor.execute(statement, parameters)
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
2024-12-02 13:28:27     result = self._query(query)
2024-12-02 13:28:27              ^^^^^^^^^^^^^^^^^^
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
2024-12-02 13:28:27     conn.query(q)
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 563, in query
2024-12-02 13:28:27     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2024-12-02 13:28:27                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 825, in _read_query_result
2024-12-02 13:28:27     result.read()
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 1199, in read
2024-12-02 13:28:27     first_packet = self.connection._read_packet()
2024-12-02 13:28:27                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 775, in _read_packet
2024-12-02 13:28:27     packet.raise_for_error()
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/protocol.py", line 219, in raise_for_error
2024-12-02 13:28:27     err.raise_mysql_exception(self._data)
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/err.py", line 150, in raise_mysql_exception
2024-12-02 13:28:27     raise errorclass(errno, errval)
2024-12-02 13:28:27 pymysql.err.OperationalError: (1105, 'the tiflash replica count: 1 should be less than the total tiflash server count: 0')
2024-12-02 13:28:27 
2024-12-02 13:28:27 The above exception was the direct cause of the following exception:
2024-12-02 13:28:27 
2024-12-02 13:28:27 Traceback (most recent call last):
2024-12-02 13:28:27   File "/app/app/api/admin_routes/knowledge_base/routes.py", line 80, in create_knowledge_base
2024-12-02 13:28:27     init_kb_tidb_vector_store(session, knowledge_base)
2024-12-02 13:28:27   File "/app/app/rag/knowledge_base/index_store.py", line 25, in init_kb_tidb_vector_store
2024-12-02 13:28:27     vector_store.ensure_table_schema()
2024-12-02 13:28:27   File "/app/app/rag/vector_store/tidb_vector_store.py", line 76, in ensure_table_schema
2024-12-02 13:28:27     VectorAdaptor(engine).create_vector_index(
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/tidb_vector/sqlalchemy/adaptor.py", line 95, in create_vector_index
2024-12-02 13:28:27     conn.execute(query)
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1418, in execute
2024-12-02 13:28:27     return meth(
2024-12-02 13:28:27            ^^^^^
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 515, in _execute_on_connection
2024-12-02 13:28:27     return connection._execute_clauseelement(
2024-12-02 13:28:27            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1640, in _execute_clauseelement
2024-12-02 13:28:27     ret = self._execute_context(
2024-12-02 13:28:27           ^^^^^^^^^^^^^^^^^^^^^^
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
2024-12-02 13:28:27     return self._exec_single_context(
2024-12-02 13:28:27            ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
2024-12-02 13:28:27     self._handle_dbapi_exception(
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2353, in _handle_dbapi_exception
2024-12-02 13:28:27     raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
2024-12-02 13:28:27     self.dialect.do_execute(
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 924, in do_execute
2024-12-02 13:28:27     cursor.execute(statement, parameters)
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/cursors.py", line 153, in execute
2024-12-02 13:28:27     result = self._query(query)
2024-12-02 13:28:27              ^^^^^^^^^^^^^^^^^^
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/cursors.py", line 322, in _query
2024-12-02 13:28:27     conn.query(q)
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 563, in query
2024-12-02 13:28:27     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
2024-12-02 13:28:27                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 825, in _read_query_result
2024-12-02 13:28:27     result.read()
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 1199, in read
2024-12-02 13:28:27     first_packet = self.connection._read_packet()
2024-12-02 13:28:27                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/connections.py", line 775, in _read_packet
2024-12-02 13:28:27     packet.raise_for_error()
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/protocol.py", line 219, in raise_for_error
2024-12-02 13:28:27     err.raise_mysql_exception(self._data)
2024-12-02 13:28:27   File "/usr/local/lib/python3.11/site-packages/pymysql/err.py", line 150, in raise_mysql_exception
2024-12-02 13:28:27     raise errorclass(errno, errval)
2024-12-02 13:28:27 sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1105, 'the tiflash replica count: 1 should be less than the total tiflash server count: 0')
2024-12-02 13:28:27 [SQL: ALTER TABLE chunks_1 SET TIFLASH REPLICA 1]

[sykp241095]

@634750802 Do you mean we 'd better raise an exception if there is no tiflash node?

And add some prompt to renmind user to add some tiflash node.

[634750802]

@634750802 Do you mean we 'd better raise an exception if there is no tiflash node?

And add some prompt to renmind user to add some tiflash node.

If we support cluster with TiFlash only, we could add a new bootstrap status require_tiflash.

And add some description to deployment docs:

• Why we use TiFlash & What is TiFlash (link to official documents?)
• User need to add --tiflash 1 flag if it uses tiup playground.
• User should enable TiFlash nodes on self-hosted TiDB cluster or TiDBCloud dedicated cluster.

[sykp241095]

I have add some tiflash requirements description in /docs: fa347f1#diff-9c187eef4df73aad17fa35e3c7cfc9348561f6a74b419f2677a854f56a337f22R19

[sykp241095]

If we support cluster with TiFlash only, we could add a new bootstrap status require_tiflash.

Make sense

[JerryKwan]

I am running a standalone tidb-server, how can I make it support tiflash? Is it possible?

[634750802]

I am running a standalone tidb-server, how can I make it support tiflash? Is it possible?

@JerryKwan It's recommended to manage your local tidb cluster with tiup.

[JerryKwan]

@634750802 Thank you for paying attention to this issue. I will set up a tidb cluster with tiup.