protocol: support query attribute since mysql 8.0.23 #55175
Conversation
ti-chi-bot
bot
added
do-not-merge/invalid-title
do-not-merge/needs-tests-checked
do-not-merge/work-in-progress
|
|
ti-chi-bot
bot
added
the
needs-rebase
|
Welcome @kafka1991! |
|
Hi @kafka1991. Thanks for your PR. I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
ti-chi-bot
bot
added
size/XXL
|
Hi @kafka1991. Thanks for your PR. PRs from untrusted users cannot be marked as trusted with I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/ok-to-test |
ti-chi-bot
bot
added
ok-to-test
kafka1991
force-pushed
the
master_query_attr
branch
from
33a9e2a to
b42bf38
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #55175 +/- ##
================================================
+ Coverage 72.9842% 73.3542% +0.3700%
================================================
Files 1697 1697
Lines 468948 469903 +955
================================================
+ Hits 342258 344694 +2436
+ Misses 105590 104121 -1469
+ Partials 21100 21088 -12
Flags with carried forward coverage won't be shown. Click here to find out more.
|
|
i don't have much background about bazel, can anyone help me with bazel compilation error ? |
For the errors of https://do.pingcap.net/jenkins/blue/organizations/jenkins/pingcap%2Ftidb%2Fghpr_check/detail/ghpr_check/14289/pipeline please check this line and fix it. It means public (exported) variables should have comment which is expected at the place 1 line above the definition. Also, please sign the CLA in the first comment. |
Thanks for your help, I have solved the problem.
The CLA had been signed. |
Maybe your GitHub account email is not the same as the email in this PR's commit? |
Had add the email in this PR's commit to github accounts in setting. |
you can use |
|
The failed If you need any help to reproduce the test result or debug the protocol related codes, feel free to |
Thanks for your remind, i will try to reproduce it by myself. |
ti-chi-bot
bot
removed
the
needs-rebase
You are right, params protocol implementation for By the way, server has a minor compatibility issue when tested by higher version of
|
kafka1991
changed the title
ti-chi-bot
bot
removed
the
do-not-merge/work-in-progress
MySQL also shows I think the result of the query |
YangKeao
changed the title
Fixed it |
| NextVal = "nextval" | ||
| LastVal = "lastval" | ||
| SetVal = "setval" | ||
| QueryAttrString = "mysql_query_attribute_string" |
There was a problem hiding this comment.
| QueryAttrString = "mysql_query_attribute_string" | |
| // TiDB Query | |
| QueryAttrString = "mysql_query_attribute_string" |
There was an issue with the |
|
The CI job doesn't seem to be happy about diff --git a/pkg/util/collate/unicode_0400_ci_generated.go b/pkg/util/collate/unicode_0400_ci_generated.go
index db1ac70e21..c073d4c67a 100644
--- a/pkg/util/collate/unicode_0400_ci_generated.go
+++ b/pkg/util/collate/unicode_0400_ci_generated.go
@@ -28 +28 @@ func (uc *unicodeCICollator) Clone() Collator {
- return &unicodeCICollator{impl: uc.impl.Clone()}
+ return &unicodeCICollator{impl: uc.impl.Clone()}
diff --git a/pkg/util/collate/unicode_0900_ai_ci_generated.go b/pkg/util/collate/unicode_0900_ai_ci_generated.go
index 51177f893c..5e160c6182 100644
--- a/pkg/util/collate/unicode_0900_ai_ci_generated.go
+++ b/pkg/util/collate/unicode_0900_ai_ci_generated.go
@@ -28 +28 @@ func (uc *unicode0900AICICollator) Clone() Collator {
- return &unicode0900AICICollator{impl: uc.impl.Clone()}
+ return &unicode0900AICICollator{impl: uc.impl.Clone()}You should probably run |
done |
|
/retest |
|
@kafka1991: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
There was a problem hiding this comment.
Test code:
#include <stdio.h>
#include <string.h>
#include <mysql.h>
MYSQL mysql;
int main() {
MYSQL_BIND bind[13];
const char *name[13] = {"string", "bigint", "int", "smallint", "tinyint", "float",
"double", "time", "date", "datetime", "timestamp", "blob", "null"};
unsigned int ssl_mode = SSL_MODE_DISABLED;
mysql_init(&mysql);
mysql_options(&mysql,MYSQL_OPT_SSL_MODE, &ssl_mode);
if (!mysql_real_connect(&mysql, "127.0.0.1", "root", "", "test", 4000, NULL, 0)) {
fprintf(stderr, "Failed to connect to database: Error: %s\n",
mysql_error(&mysql));
}
memset(bind, 0, sizeof(bind));
char *mystr = "test test";
unsigned long mystrlen = strlen(mystr);
bind[0].buffer_type = MYSQL_TYPE_STRING;
bind[0].buffer = mystr;
bind[0].length = &mystrlen;
bind[0].is_null = 0;
unsigned long long longlongparam = 123ULL;
unsigned long longlongparamlen = sizeof(long long int);
bind[1].buffer_type = MYSQL_TYPE_LONGLONG;
bind[1].buffer = (char *) &longlongparam;
bind[1].length = &longlongparamlen;
bind[1].is_null = 0;
bind[1].is_unsigned = 1;
int intparam = 123;
unsigned long intparamlen = sizeof(int);
bind[2].buffer_type = MYSQL_TYPE_LONG;
bind[2].buffer = (char *) &intparam;
bind[2].length = &intparamlen;
bind[2].is_null = 0;
short int shortparam = 123;
unsigned long shortparamlen = sizeof(short int);
bind[3].buffer_type = MYSQL_TYPE_SHORT;
bind[3].buffer = (char *) &shortparam;
bind[3].length = &shortparamlen;
bind[3].is_null = 0;
signed char tinyparam = 123;
unsigned long tinyparamlen = sizeof(signed char);
bind[4].buffer_type = MYSQL_TYPE_TINY;
bind[4].buffer = (char *) &tinyparam;
bind[4].length = &tinyparamlen;
bind[4].is_null = 0;
float floatparam = 12.34;
unsigned long floatparamlen = sizeof(float);
bind[5].buffer_type = MYSQL_TYPE_FLOAT;
bind[5].buffer = (char *) &floatparam;
bind[5].length = &floatparamlen;
bind[5].is_null = 0;
double doubleparam = 12.34;
unsigned long doubleparamlen = sizeof(double);
bind[6].buffer_type = MYSQL_TYPE_DOUBLE;
bind[6].buffer = (char *) &doubleparam;
bind[6].length = &doubleparamlen;
bind[6].is_null = 0;
MYSQL_TIME tm;
tm.hour = 8;
tm.minute = 23;
tm.second = 24;
tm.second_part = 987654;
unsigned long tmlen = sizeof(tm);
bind[7].buffer_type = MYSQL_TYPE_TIME;
bind[7].buffer = (char *) &tm;
bind[7].length = &tmlen;
bind[7].is_null = 0;
MYSQL_TIME ts;
ts.year = 2025;
ts.month = 2;
ts.day = 6;
ts.hour = 8;
ts.minute = 23;
ts.second = 24;
unsigned long tslen = sizeof(ts);
bind[8].buffer_type = MYSQL_TYPE_DATE;
bind[8].buffer = (char *) &ts;
bind[8].length = &tslen;
bind[8].is_null = 0;
bind[9].buffer_type = MYSQL_TYPE_DATETIME;
bind[9].buffer = (char *) &ts;
bind[9].length = &tslen;
bind[9].is_null = 0;
bind[10].buffer_type = MYSQL_TYPE_TIMESTAMP;
bind[10].buffer = (char *) &ts;
bind[10].length = &tslen;
bind[10].is_null = 0;
bind[11].buffer_type = MYSQL_TYPE_BLOB;
bind[11].buffer = mystr;
bind[11].length = &mystrlen;
bind[11].is_null = 0;
bind[12].buffer_type = MYSQL_TYPE_NULL;
mysql_bind_param(&mysql, 13, bind, name);
const char *query = "SELECT mysql_query_attribute_string('string'), mysql_query_attribute_string('bigint'), mysql_query_attribute_string('int'), mysql_query_attribute_string('smallint'), mysql_query_attribute_string('tinyint'), mysql_query_attribute_string('float'), mysql_query_attribute_string('double'), mysql_query_attribute_string('time'), mysql_query_attribute_string('date'), mysql_query_attribute_string('datetime'), mysql_query_attribute_string('timestamp'), mysql_query_attribute_string('blob'), mysql_query_attribute_string('null')";
mysql_real_query(&mysql, query, strlen(query));
MYSQL_RES *result = mysql_store_result(&mysql);
MYSQL_ROW row = mysql_fetch_row(result);
unsigned long *lengths = mysql_fetch_lengths(result);
for(int i = 0; i < 13; i++)
{
printf("attribute %2d (%-9s) [%2ld]: [%.*s]\n", i+1, name[i], bind[i].length == NULL ? 0 : *bind[i].length, (int) lengths[i], row[i] ? row[i] : "NULL");
}
mysql_free_result(result);
mysql_close(&mysql);
}Compiled with:
cc $(mysql_config --libs --cflags) -o qattr qattr.c
Modified the port of mysql_real_connect where needed to switch between 3306 (MySQL) and 4000 (TiDB).
MySQL:
$ ./qattr
attribute 1 (string ) [ 9]: [test test]
attribute 2 (bigint ) [ 8]: [123]
attribute 3 (int ) [ 4]: [123]
attribute 4 (smallint ) [ 2]: [123]
attribute 5 (tinyint ) [ 1]: [123]
attribute 6 (float ) [ 4]: [12.34000015258789]
attribute 7 (double ) [ 8]: [12.34]
attribute 8 (time ) [48]: [08:23:24.987654]
attribute 9 (date ) [48]: [2025-02-06]
attribute 10 (datetime ) [48]: [2025-02-06 08:23:24.000000]
attribute 11 (timestamp) [48]: [2025-02-06 08:23:24.000000]
attribute 12 (blob ) [ 9]: [test test]
attribute 13 (null ) [ 0]: []
TiDB:
$ ./qattr
attribute 1 (string ) [ 9]: [test test]
attribute 2 (bigint ) [ 8]: [123]
attribute 3 (int ) [ 4]: [123]
attribute 4 (smallint ) [ 2]: [123]
attribute 5 (tinyint ) [ 1]: [123]
attribute 6 (float ) [ 4]: [12.34]
attribute 7 (double ) [ 8]: [12.34]
attribute 8 (time ) [48]: [08:23:24.987654]
attribute 9 (date ) [48]: [2025-02-06]
attribute 10 (datetime ) [48]: [2025-02-06 08:23:24]
attribute 11 (timestamp) [48]: [2025-02-06 08:23:24]
attribute 12 (blob ) [ 9]: [test test]
attribute 13 (null ) [ 0]: []
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: dveeden The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@kafka1991: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
Another test: #!/bin/python
import mysql.connector
for port in [3306, 4000]:
for prep in [True, False]:
print(f"Testing against port {port} with prepared={prep}")
config = {
"host":"127.0.0.1",
"port":port,
"user":"root",
}
c = mysql.connector.connect(**config)
cur = c.cursor(prepared=prep)
cur.add_attribute("qattrkey", "testvalue🐬")
cur.execute("SELECT VERSION(), mysql_query_attribute_string('qattrkey')")
for row in cur:
print(f"{row[0]:50s} {row[1]}\n")
cur.close()
c.close()Result: So this doesn't yet work for prepared statements? |
Here is My test result with git head ( It seems strange. Did you cherry-pick the relevant code to another branch? |
|
These are the same commit: Extended the test: #!/bin/python
import mysql.connector
print(f"Testing with MySQL Connector/Python {mysql.connector.version.VERSION_TEXT}\n")
for port in [3306, 4000]:
for prep in [True, False]:
for pure in [True, False]:
print(
f"Testing against port {port} with prepared={prep} and use_pure={pure}"
)
config = {
"host": "127.0.0.1",
"port": port,
"user": "root",
"use_pure": pure,
}
c = mysql.connector.connect(**config)
cur = c.cursor(prepared=prep)
cur.add_attribute("qattrkey", "testvalue🐬")
cur.execute("SELECT VERSION(), mysql_query_attribute_string('qattrkey')")
for row in cur:
print(f"{row[0]:50s} {row[1]}\n")
cur.close()
c.close() |
|
With the local formatting changes removed: |
|
Looks like this might be a bug in MySQL Connector/Python. It looks like it doesn't set the right flags and still send the attributes... not sure why it doesn't do this when connecting to MySQL. |
|
I found the issue: MySQL Connector/Python with the C extensions enabled and with the prepared cursor enabled is using the MySQL C API to send query attributes to the server This works by:
And then there is this: This causes the libmysql to incorrectly handle the protocol if the version the server advertises is below 8.0.26. To test this: Results: I've created https://bugs.mysql.com/bug.php?id=117567 for this. |
What problem does this PR solve?
Issue Number: close #55174
Problem Summary:
What changed and how does it work?
Check List
Tests
Side effects
Documentation
Release note
Please refer to Release Notes Language Style Guide to write a quality release note.