Skip to content

Commit 2586664

Browse files
committed
add sqlserverCDC Modules and docs
1 parent b4814e6 commit 2586664

File tree

87 files changed

+894
-1947
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

87 files changed

+894
-1947
lines changed

README.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ The following databases are currently supported:
7070
| | FTP | [doc](docs/offline/reader/ftpreader.md) | [doc](docs/offline/writer/ftpwriter.md) |
7171
| | HDFS | [doc](docs/offline/reader/hdfsreader.md) | [doc](docs/offline/writer/hdfswriter.md) |
7272
| | Carbondata | [doc](docs/offline/reader/carbondatareader.md) | [doc](docs/offline/writer/carbondatawriter.md) |
73-
| | Stream | [doc](docs/offline/reader/streamreader.md) | [doc](docs/offline/writer/carbondatawriter.md) |
73+
| | Stream | [doc](docs/offline/reader/streamreader.md) | [doc](docs/offline/writer/streamwriter.md) |
7474
| | Redis | | [doc](docs/offline/writer/rediswriter.md) |
7575
| | Hive | | [doc](docs/offline/writer/hivewriter.md) |
7676
| Stream Synchronization | Kafka | [doc](docs/realTime/reader/kafkareader.md) | [doc](docs/realTime/writer/kafkawriter.md) |
@@ -80,6 +80,7 @@ The following databases are currently supported:
8080
| | MongoDB Oplog | [doc](docs/realTime/reader/mongodboplogreader.md)| |
8181
| | PostgreSQL WAL | [doc](docs/realTime/reader/pgwalreader.md) | |
8282
| | Oracle LogMiner| [doc](docs/realTime/reader/LogMiner.md) | |
83+
| | Sqlserver CDC| [doc](docs/realTime/reader/sqlservercdc.md) | |
8384

8485
# Fundamental
8586
In the underlying implementation, FlinkX relies on Flink, and the data synchronization task will be translated into StreamGraph and executed on Flink. The basic principle is as follows:

README_CH.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ FlinkX目前支持下面这些数据库:
8080
| | FTP | [doc](docs/offline/reader/ftpreader.md) | [doc](docs/offline/writer/ftpwriter.md) |
8181
| | HDFS | [doc](docs/offline/reader/hdfsreader.md) | [doc](docs/offline/writer/hdfswriter.md) |
8282
| | Carbondata | [doc](docs/offline/reader/carbondatareader.md) | [doc](docs/offline/writer/carbondatawriter.md) |
83-
| | Stream | [doc](docs/offline/reader/streamreader.md) | [doc](docs/offline/writer/carbondatawriter.md) |
83+
| | Stream | [doc](docs/offline/reader/streamreader.md) | [doc](docs/offline/writer/streamwriter.md) |
8484
| | Redis | | [doc](docs/offline/writer/rediswriter.md) |
8585
| | Hive | | [doc](docs/offline/writer/hivewriter.md) |
8686
| Stream Synchronization | Kafka | [doc](docs/realTime/reader/kafkareader.md) | [doc](docs/realTime/writer/kafkawriter.md) |
@@ -90,6 +90,7 @@ FlinkX目前支持下面这些数据库:
9090
| | MongoDB Oplog | [doc](docs/realTime/reader/mongodboplogreader.md)| |
9191
| | PostgreSQL WAL | [doc](docs/realTime/reader/pgwalreader.md) | |
9292
| | Oracle LogMiner | [doc](docs/realTime/reader/LogMiner.md) | |
93+
| | Sqlserver CDC | [doc](docs/realTime/reader/sqlservercdc.md) | |
9394

9495
# 基本原理
9596
在底层实现上,FlinkX依赖Flink,数据同步任务会被翻译成StreamGraph在Flink上执行,基本原理如下图:

docs/offline/reader/sqlserverreader.md

-1
Original file line numberDiff line numberDiff line change
@@ -398,7 +398,6 @@
398398
}
399399
}
400400
}
401-
}
402401
```
403402
<a name="labNO"></a>
404403
#### 4、增量同步指定startLocation

docs/realTime/reader/kafkareader.md

+9-6
Original file line numberDiff line numberDiff line change
@@ -155,21 +155,24 @@ kafka插件存在四个版本,根据kafka版本的不同,插件名称也略
155155

156156
<br/>
157157

158+
159+
<a name="ftKiS"></a>
158160
## 三、配置示例
159161
### 1、kafka09
160162
```json
161163
{
162-
"job" : {
163-
"content" : [ {
164+
"job": {
165+
"content": [{
164166
"reader" : {
165167
"parameter" : {
166168
"topic" : "kafka09",
167-
"groupId" : "default",
168-
"codec" : "text",
169+
"codec": "plain",
169170
"encoding": "UTF-8",
170-
"blankIgnore": false,
171171
"consumerSettings" : {
172-
"zookeeper.connect" : "localhost:2181/kafka09"
172+
"zookeeper.connect" : "0.0.0.1:2182/kafka09",
173+
"group.id" : "default",
174+
"auto.commit.interval.ms" : "1000",
175+
"auto.offset.reset" : "smallest"
173176
}
174177
},
175178
"name" : "kafka09reader"

0 commit comments

Comments
 (0)