[Bug] Doris2.0.12 use flink-1.18.1 and flink-doris-connector-1.18-1.6.2 report error when sinking #45965
Comments
|
It shows every thing allright at the flink side, the log as follows: RESOURCE_PARAMS extraction logs: 2024-12-25 10:08:10,992 INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - -------------------------------------------------------------------------------- |
|
I found the problem, it's just report error when the table initializition. The log for the Sql is: 2024-12-26 11:14:45,280 INFO (STATS_FETCH-3|308) [BasicAsyncCacheLoader.lambda$asyncLoad$0():42] Load statistic cache [ColumnStats-6--1-DATABASE_COLLATION] cost time ms:90 Caused by: org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = Syntax error 2024-12-26 11:14:48,906 [query] |Client=11.0.228.86:56929|User=costshare|Db=|State=OK|ErrorCode=0|ErrorMessage=|Time(ms)=1|ScanBytes=0|ScanRows=0|ReturnRows=0|StmtId=13|QueryId=ff7036b1f9374633-a1f516c334433867|IsQuery=false|isNereids=false|feIp=10.30.251.92|Stmt=SET NAMES utf8mb4|CpuTimeMS=0|SqlHash=c64be5c6e91cdad08370b7189c583332|peakMemoryBytes=0|SqlDigest=|TraceId=|WorkloadGroup=|FuzzyVariables= |
Search before asking
Version
server: GNU/Linux NeoKylin Linux Advanced Server V7Update6 (Chromium) build 5.4.277-1.el7.elrepo.x86_64 Booted: 2024-12-24T05:25:06Z Uptime: 0 days, 21:32:00 Running with elevated permissions.
Doris2.0.12 with noavx2 useing apache-doris-2.0.12-bin-x64-noavx2.tar.gz as builded package
flink-1.18.1
doris-flink-connector-1.6.2 self builded with java1.8.0_432
What's Wrong?
2024-12-25 10:08:29,516 WARN (mysql-nio-pool-19|8982) [ReadListener.lambda$handleEvent$0():60] Exception happened in one session(org.apache.doris.qe.ConnectContext@642bcb47).
java.io.IOException: Error happened when receiving packet.
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:838) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_252]
2024-12-25 10:13:13,233 WARN (mysql-nio-pool-21|9173) [MysqlChannel.fetchOnePacket():301] receive packet sequence id[32] want to get[1]
2024-12-25 10:13:14,343 WARN (mysql-nio-pool-21|9173) [MysqlChannel.fetchOnePacket():301] receive packet sequence id[32] want to get[1]
2024-12-25 10:13:17,251 WARN (mysql-nio-pool-21|9173) [MysqlChannel.fetchOnePacket():301] receive packet sequence id[32] want to get[1]
2024-12-25 10:13:18,288 WARN (mysql-nio-pool-22|9174) [MysqlChannel.fetchOnePacket():301] receive packet sequence id[32] want to get[1]
2024-12-25 10:13:20,690 WARN (mysql-nio-pool-21|9173) [MysqlChannel.fetchOnePacket():301] receive packet sequence id[32] want to get[1]
2024-12-25 10:13:25,733 WARN (mysql-nio-pool-21|9173) [MysqlChannel.fetchOnePacket():301] receive packet sequence id[32] want to get[1]
2024-12-25 10:13:35,717 WARN (mysql-nio-pool-21|9173) [MysqlChannel.fetchOnePacket():301] receive packet sequence id[32] want to get[1]
2024-12-25 10:13:38,872 WARN (mysql-nio-pool-21|9173) [MysqlChannel.fetchOnePacket():301] receive packet sequence id[32] want to get[1]
2024-12-25 10:13:39,756 WARN (mysql-nio-pool-21|9173) [MysqlChannel.fetchOnePacket():301] receive packet sequence id[32] want to get[1]
2024-12-25 10:17:13,490 WARN (mysql-nio-pool-23|9207) [StmtExecutor.executeByLegacy():844] execute Exception. stmt[277, f78fe115c8ca4fdd-a1037c747da9cc33]
org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = Unknown column '$$' in 'table list'
at org.apache.doris.common.ErrorReport.reportAnalysisException(ErrorReport.java:53) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.ErrorReport.reportAnalysisException(ErrorReport.java:48) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.analysis.Analyzer.registerColumnRef(Analyzer.java:1001) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.analysis.SlotRef.analyzeImpl(SlotRef.java:207) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.analysis.Expr.analyze(Expr.java:449) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.analysis.SelectStmt.analyze(SelectStmt.java:576) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.analyzeAndGenerateQueryPlan(StmtExecutor.java:1152) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.analyze(StmtExecutor.java:1060) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:735) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:507) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:475) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:430) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:578) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:847) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_252]
What You Expected?
mysql data successfully sinked into doris db.
Iguess there is some thing wrong with the network binding.
How to Reproduce?
should use noavx2 package and with more then one network-interfaces, i think is should be problem with the network-interface.
add be-node with IP-adresse, for me is 10.30.251.92, and config with 10.30.251.92/32
Anything Else?
Doris info:
Git : git://VM-0-117-ubuntu@2971efd
Version : doris-2.0.12
BuildInfo : VM-0-117-ubuntu
BuildTime : Sun, 23 Jun 2024 19:59:02 CST
NetworkParameter : Network parameters: Host name: rd-dev-hgsystemmodel01 Domain name: rd-dev-hgsystemmodel01 DNS servers: [10.30.64.17, 10.30.64.17] IPv4 Gateway: 10.30.251.126 IPv6 Gateway:
Processor : Intel(R) Xeon(R) CPU E7- 4830 @ 2.13GHz 2 physical CPU package(s) 16 physical CPU core(s) 16 logical CPU(s) Identifier: Intel64 Family 6 Model 47 Stepping 2 ProcessorID: F2 06 02 00 FF FB AB 1F Microarchitecture: Westmere (Server) 2 physical CPU package(s) 16 physical CPU core(s) 16 logical CPU(s) Identifier: Intel64 Family 6 Model 47 Stepping 2 ProcessorID: F2 06 02 00 FF FB AB 1F Context Switches/Interrupts: 339810487 / 201269800 CPU, IOWait, and IRQ ticks @ 0 sec:[4292840, 780, 4279090, 1227083940, 196730, 0, 29220, 0] CPU, IOWait, and IRQ ticks @ 1 sec:[4292880, 780, 4279140, 1227099800, 196730, 0, 29220, 0] User: 0.3% Nice: 0.0% System: 0.3% Idle: 99.4% IOwait: 0.0% IRQ: 0.0% SoftIRQ: 0.0% Steal: 0.0% CPU load: 0.6% CPU load averages: 0.01 0.00 0.00 CPU load per processor: 0.0% 0.0% 0.0% 1.0% 0.0% 1.0% 0.0% 0.0% 2.0% 2.0% 1.0% 1.0% 0.0% 0.0% 0.0% 1.0% Vendor Frequency: 2.1 GHz Max Frequency: 4.2 GHz Current Frequencies: 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz, 2.1 GHz
OS : GNU/Linux NeoKylin Linux Advanced Server V7Update6 (Chromium) build 5.4.277-1.el7.elrepo.x86_64 Booted: 2024-12-24T05:25:06Z Uptime: 0 days, 21:32:00 Running with elevated permissions.
Memory : Memory: 54.7 GiB/62.9 GiB Swap used: 0 bytes/0 bytes
FileSystem : File System: File Descriptors: 6464/6576687 / (Local Disk) [xfs] 11.3 GiB of 26.0 GiB free (43.5%), 13.3 M of 13.6 M files free (97.2%) is /dev/mapper/rootvg-lvroot [/dev/dm-0] and is mounted at / /dev/sdb2 (Local Disk) [ext4] 172.9 GiB of 196.7 GiB free (87.9%), 13.1 M of 13.1 M files free (99.9%) is /dev/sdb2 and is mounted at /costshare/server /dev/sda1 (Local Disk) [xfs] 789.9 MiB of 1014 MiB free (77.9%), 523.9 K of 524.3 K files free (99.9%) is /dev/sda1 and is mounted at /boot /dev/mapper/rootvg-lvvar (Local Disk) [xfs] 8.3 GiB of 10.0 GiB free (83.4%), 5.2 M of 5.2 M files free (99.9%) is /dev/mapper/rootvg-lvvar [/dev/dm-3] and is mounted at /var /dev/mapper/rootvg-lvopt (Local Disk) [xfs] 9.9 GiB of 10.0 GiB free (98.6%), 5.2 M of 5.2 M files free (100.0%) is /dev/mapper/rootvg-lvopt [/dev/dm-2] and is mounted at /opt /dev/mapper/rootvg-lvhome (Local Disk) [xfs] 962.6 MiB of 1014 MiB free (94.9%), 524.1 K of 524.3 K files free (100.0%) is /dev/mapper/rootvg-lvhome [/dev/dm-1] and is mounted at /home
NetworkInterface : Network interfaces: Name: docker0 (docker0) MAC Address: 02:42:d5:07:62:70 MTU: 1500, Speed: 0 bps IPv4: [172.17.0.1] IPv6: [] Traffic: received ?/?; transmitted ?/? Name: br-db0626f6392a (br-db0626f6392a) MAC Address: 02:42:50:00:92:2b MTU: 1500, Speed: 0 bps IPv4: [172.19.0.1] IPv6: [] Traffic: received ?/?; transmitted ?/? Name: br-93827b5a9600 (br-93827b5a9600) MAC Address: 02:42:9a:ee:fc:d6 MTU: 1500, Speed: 0 bps IPv4: [172.18.0.1] IPv6: [] Traffic: received ?/?; transmitted ?/? Name: br-4fa9a3086ea2 (br-4fa9a3086ea2) MAC Address: 02:42:26:4e:23:a2 MTU: 1500, Speed: 0 bps IPv4: [172.23.0.1] IPv6: [] Traffic: received ?/?; transmitted ?/? Name: br-47a241f649e9 (br-47a241f649e9) MAC Address: 02:42:85:21:6d:0f MTU: 1500, Speed: 0 bps IPv4: [172.22.0.1] IPv6: [] Traffic: received ?/?; transmitted ?/? Name: virbr0 (virbr0) MAC Address: 52:54:00:5d:97:70 MTU: 1500, Speed: 0 bps IPv4: [192.168.122.1] IPv6: [] Traffic: received ?/?; transmitted ?/? Name: ens192 (VMware VMXNET3 Ethernet Controller) MAC Address: 00:50:56:91:f1:f0 MTU: 1500, Speed: 10.5 Gbps IPv4: [10.30.251.92] IPv6: [fe80:0:0:0:250:56ff:fe91:f1f0] Traffic: received 761993 packets/331.4 MiB (0 err); transmitted 954027 packets/2.9 GiB (0 err)
Processes : Processes: 357, Threads: 2982 PID %CPU %MEM VSZ RSS Name 47266 6.4 3.7 22.2 GiB 2.3 GiB doris_be 10583 1.8 3.6 24.7 GiB 2.3 GiB java 46365 1.5 2.7 18.6 GiB 1.7 GiB java 32102 1.2 0.0 117.3 MiB 7.8 MiB bash 32091 0.7 0.0 157.8 MiB 9.6 MiB sshd
Disk : Disks: /dev/sr0: (model: VMware_Virtual_IDE_CDROM_Drive - S/N: 10000000000000000001) size: 1.1 GB, reads: ? (?), writes: ? (?), xfer: ? ms /dev/sda: (model: Virtual_disk - S/N: unknown) size: 68.7 GB, reads: 36384 (2.0 GiB), writes: 128840 (925.9 MiB), xfer: 213970 ms |-- /dev/sda1: /sys/devices/pci0000:00/0000:00:15.0/0000:03:00.0/host2/target2:0:0/2:0:0:0/block/sda/sda1 (xfs) Maj:Min=8:1, size: 1.1 GB @ /boot |-- /dev/sda2: /sys/devices/pci0000:00/0000:00:15.0/0000:03:00.0/host2/target2:0:0/2:0:0:0/block/sda/sda2 (LVM2_member) Maj:Min=8:2, size: 67.6 GB @ dm-1 dm-2 dm-0 dm-3 /dev/sdb: (model: Virtual_disk - S/N: unknown) size: 536.9 GB, reads: 8827 (2.5 GiB), writes: 148412 (1.8 GiB), xfer: 229574 ms |-- /dev/sdb1: /sys/devices/pci0000:00/0000:00:15.0/0000:03:00.0/host2/target2:0:1/2:0:1:0/block/sdb/sdb1 (partition) Maj:Min=8:17, size: 1.0 KB |-- /dev/sdb2: /sys/devices/pci0000:00/0000:00:15.0/0000:03:00.0/host2/target2:0:1/2:0:1:0/block/sdb/sdb2 (ext4) Maj:Min=8:18, size: 214.7 GB @ /costshare/server /dev/dm-0: (model: Logical Volume Group - S/N: LVM-DvapNp4Wu22KT3jm8xN1VfMFKcOYDbi0tbB72a0aXbyHMjtQXj80cFz6urb1Vlyt) size: 27.9 GB, reads: 31133 (1.7 GiB), writes: 48264 (483.4 MiB), xfer: 180985 ms |-- /dev/rootvg/lvroot: /sys/devices/virtual/block/dm-0 (xfs) Maj:Min=253:0, size: 27.9 GB @ /dev/mapper/rootvg-lvroot /dev/dm-1: (model: Logical Volume Group - S/N: LVM-DvapNp4Wu22KT3jm8xN1VfMFKcOYDbi0FOgKu7zvOWI8QhqFfHHRyzTXmRW0RlFr) size: 1.1 GB, reads: 95 (5.3 MiB), writes: 250 (3.4 MiB), xfer: 1017 ms |-- /dev/rootvg/lvhome: /sys/devices/virtual/block/dm-1 (xfs) Maj:Min=253:1, size: 1.1 GB @ /dev/mapper/rootvg-lvhome /dev/dm-2: (model: Logical Volume Group - S/N: LVM-DvapNp4Wu22KT3jm8xN1VfMFKcOYDbi0dKlXK8GlQ8d1j1kJkvJFSROpRw2MzWxW) size: 10.7 GB, reads: 620 (52.5 MiB), writes: 135 (2.7 MiB), xfer: 3047 ms |-- /dev/rootvg/lvopt: /sys/devices/virtual/block/dm-2 (xfs) Maj:Min=253:2, size: 10.7 GB @ /dev/mapper/rootvg-lvopt /dev/dm-3: (model: Logical Volume Group - S/N: LVM-DvapNp4Wu22KT3jm8xN1VfMFKcOYDbi0WhQ4DzweKGQS5WBY8mAKVF2nH78Aqb93) size: 10.7 GB, reads: 3992 (223.2 MiB), writes: 88808 (434.3 MiB), xfer: 41071 ms |-- /dev/rootvg/lvvar: /sys/devices/virtual/block/dm-3 (xfs) Maj:Min=253:3, size: 10.7 GB @ /dev/mapper/rootvg-lvvar
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: