Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/parquet write buffer policy merge #3

Closed
wants to merge 93 commits into from

Conversation

aqni
Copy link
Owner

@aqni aqni commented Jul 9, 2024

No description provided.

aqni added 30 commits May 17, 2024 14:34
aqni added 10 commits July 11, 2024 12:55
…feat/parquet-write-buffer-policy-merge

# Conflicts:
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/ImmutableDatabase.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/buffer/ActiveMemTable.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/buffer/ArchivedMemTable.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/buffer/MemColumn.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/buffer/MemTable.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/buffer/MemTableQueue.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/buffer/chunk/Chunk.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/buffer/chunk/IndexedChunk.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/buffer/chunk/IndexedChunkType.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/buffer/chunk/NoIndexChunk.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/buffer/chunk/SkipListChunk.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/compact/Compactor.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/compact/Flusher.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/lsm/table/EmptyTable.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/util/WriteBatches.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/util/iterator/DelegateScanner.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/util/iterator/EmtpyHeadRowScanner.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/util/iterator/LazyRowScanner.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/util/iterator/ListenCloseScanner.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/util/iterator/RowConcatScanner.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/db/util/iterator/RowScannerFactory.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/io/FileFormat.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/io/FileIndex.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/io/FileMeta.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/io/FileReader.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/io/FileWriter.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/io/common/DataChunk.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/io/common/DataChunks.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/io/common/EmptyReader.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/manager/data/AggregatedRowStream.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/Awaitable.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/CloseableHolders.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/NoexceptAutoCloseable.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/NoexceptAutoCloseables.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/SingleCache.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/arrow/ArrowFieldTypes.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/arrow/ArrowFields.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/arrow/ArrowTypes.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/arrow/ArrowVectors.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/exception/StorageClosedException.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/iterator/DedupIterator.java
#	dataSource/parquet/src/main/java/cn/edu/tsinghua/iginx/parquet/util/iterator/StableMergeIterator.java
#	optimizer/src/main/java/cn/edu/tsinghua/iginx/logical/optimizer/rules/RuleCollection.java
@aqni aqni closed this Jul 11, 2024
@aqni aqni reopened this Jul 11, 2024
@aqni aqni force-pushed the feat/parquet-write-buffer-policy-merge branch from 3dcfa50 to 47e1a69 Compare July 11, 2024 07:57
@aqni aqni closed this Jul 11, 2024
@aqni aqni reopened this Jul 11, 2024
aqni and others added 9 commits July 11, 2024 16:31
更快的 clear data 实现:使用 flushDB 实现 clear data 操作,不逐条删除
将默认超时时间从 5s 改为 10s
重构异常与日志打印
移除主仓库中的混淆配置
使用 SPI 机制动态加载规则
更新混淆 jar 包
Correct an incorrect variable name in max udf function.
…THU#368)

创建 release 时,打包好的 wheel 文件和源码包会上传到 release 中。
session_py 的版本号与 IGinX 一致
@aqni aqni closed this Jul 14, 2024
@aqni aqni deleted the feat/parquet-write-buffer-policy-merge branch July 14, 2024 02:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants