add warmup #15
Closed
add warmup #15
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
IGinX-THU#449) * add framework operator executor * count rows * format * impl row to arrow batch and format * modify pipeline executor * format
- 实现物理算子Projector（对应逻辑算子Project & Reorder & Rename & AddSchemaprefix） - 改造没有fetchsize的SQL session接口 TODO：需要修改
* avoid copy in projector * feat(core): add policy of single fragment * feat(core): impl multiply and sum
* wip * binary & unary expression implementation * license
* wip * binary & unary expression implementation * license * multiple * revert multiply expr
* add TODO * refactor cast
…U#479) * wip * binary & unary expression implementation * license * multiple * revert multiply expr * implement constant & bracket expr; bool filter
* logical & casewhen * restore test * restore test * some change
part of hashjoin executor limit executor fetch-all executor
limit part of hash-join fetch-all task
markjoin of hashjoin other filter of hashjoin
* fix measure close time * finish stage4
…d lazy materialization (IGinX-THU#496) * feat(arrow): like const optimize * feat(arrow): IntHashMap for HashJoin * Revert "feat(arrow): IntHashMap for HashJoin" This reverts commit b022975. * store * store
同时修复写入卡住的问题
* fix(arrow): fix most of SQLSessionIT * fix SQLSessionIT
…th arrow merge main (IGinX-THU#510) * remove unecessary test on Parquet * refactor(datasource): use default meta config for mysql & postgresql (IGinX-THU#444) * use default meta config for mysql * for tests * restore tests * use props file for pg * restore tests * remove unused file * restore mysql test config --------- Co-authored-by: Yuqing Zhu <[email protected]> * feat(thrift): auto update session_py (IGinX-THU#443) Co-authored-by: Yuqing Zhu <[email protected]> * fix(core): fix join with group by (IGinX-THU#446) Co-authored-by: Yuqing Zhu <[email protected]> * feat(filestore): reduce multi-threaded write conflicts for parquet (IGinX-THU#439) 把以前 parquet 没合并的内容补上 * feat(sql): func params support expression (IGinX-THU#445) 支持在函数参数中使用表达式 * refactor(dataSource): rename FileStore to FileSystem (IGinX-THU#451) 替换名字 * feat(sql): sql support const arithmetic expression (IGinX-THU#447) sql support const arithmetic expression * feat(sql): auto increment sequence & alter key (IGinX-THU#458) 1. 增加生成自增列的语法 2. 增加KEY列的操作语法，包括：将特定列升级成KEY列，将KEY列降级成普通列 详见https://oxlh5mrwi0.feishu.cn/docx/Gd8pd1aMDoNMIexXBPJcaea6n0g * chore: merge 0.7.2 (IGinX-THU#460) * feat(shared): retry connect * feat(jdbc): new prefix `jdbc:iginx:tsdb:` --------- Co-authored-by: Yuqing Zhu <[email protected]> * fix(MAX_HEAP_SIZE): set max heap siez max(min(1/2 ram, 1024MB), min(1… (IGinX-THU#456) * fix(MAX_HEAP_SIZE): set max heap siez max(min(1/2 ram, 1024MB), min(1/4 ram, 64GB)) * fix start_iginx.bat * fix(MAX_HEAP_SIZE): user-defined ratio * fix(MAX_HEAP_SIZE): user-defined ratio * fix(MAX_HEAP_SIZE, MIN_HEAP_SIZE): user-defined percentage * fix(MAX_HEAP_SIZE, MIN_HEAP_SIZE): user-defined percentage --------- Co-authored-by: Yuqing Zhu <[email protected]> * fix(core): register engine with same ip and port (IGinX-THU#457) Test connection before registering engine. When a new database is registered, check for existing databases with the same IP and port. If the connection fails and the existing database is read-only, delete the existing database. * chore: make building iginx easier (IGinX-THU#476) * docs: add FAQ to quickstart to explain how to resolve problems The following artifacts could not be resolved: cn.edu.tsinghua.iginx:parquet-file * chore: fix `mvn clean install -U -Dmaven.test.skip=true` * build: fix session_py format --------- Co-authored-by: Yuqing Zhu <[email protected]> * update license * update readme for license update * feat(optimizer): 规则组的实现 (IGinX-THU#470) 实现了规则组，将一系列相同的规则放到一个规则组中，以简化规则的设置和更改。 原有的Set Rules命令现在会更改规则组而不是规则 目前以下规则合并入规则组 一系列谓词下推规则 -> FilterPushDownRule ColumnPruningRule和FragmentPruningByPatternRule -> ColumnPruningRule 一系列ConstantFoldingRule -> ConstantFoldingRule 一系列DistinctEliminateRule -> DistinctEliminateRule 具体参考PR文档 https://oxlh5mrwi0.feishu.cn/wiki/OVt5wJxlRip2uQkiFS7c8IHNnzM 此外还删除了规则FilterJoinTransposeRule，这是一力之前写的示例规则，这里不保留了 * feat(sql): GROUP BY expr && ORDER BY expr (IGinX-THU#465) * feat(sql): GROUP BY expr && ORDER BY expr 1.支持对GROUP BY和ORDER BY中的列使用RowToRow表达式 2.支持GROUP BY和ORDER BY中的列与SELECT子句中的别名进行匹配 Co-authored-by: Yuqing Zhu <[email protected]> * feat(sql): support use func with expr params in filter (IGinX-THU#473) 支持在filter里使用参数为表达式的函数 更新剩余TPC-H语句准备测试 * feat(Optimizer): In Filter Transform Rule (IGinX-THU#359) 实现了In Filter： 实现了在WHERE条件中加入In Filter，如：SELECT s1 FROM us.d1 WHERE s1 in (1, 2, 3, 4, 5);和SELECT s1 FROM us.d1 WHERE s1 not in (1, 2, 3, 4, 5); 修改了influxdb和relational的对接层，使其能支持InFilter下推。 实现了InFilterTransformRule： 能够将and filter中path相同的!=和not in 条件合并成一个not in 条件。 能够将or filter中path相同的=和in条件合并一个in 条件。 详细内容和测试见【PR文档】IN Filter 转换 * Update LogicalFilterUtils.java (IGinX-THU#481) 谓词下推会将filter转换成CNF，之后FragmentPruningByFilter规则在提取key range的时候会将CNF再转成DNF,会导致指数爆炸的问题。本PR在FragmentPruningByFilter转DNF前进行一次操作将filter中的非key filter处理掉，以减少指数爆炸的问题。 * test(tpc-h): add tpc-h tests with pushdown (IGinX-THU#477) 增加TPC-H下推测试 * fix(optimize): fix column pruning rule for UDF (IGinX-THU#491) * 修复列裁剪在对udf处理上的问题 * Update ColumnPruningRule.java * fix(sql): fix filter in sql with tagKV (IGinX-THU#478) fix filter in sql with tagKV to enable the support of TSBS * fix(ci): correcting docker installation fault on mac (IGinX-THU#495) mac上的容器运行环境为colima，依赖于qemu，因此，必须先安装qemu * feat(sql): join by key & extract & add dummy test (IGinX-THU#488) 增加 join using key 的语法 增加系统函数：extract 增加叠加分片范围有重叠，但实际数据不重叠的测试 * fix(sql): fix sum/avg with null values (IGinX-THU#502) 计算 sum 和 avg 时，如果该列全为 null，均返回 null, count仍为0 修正ImportFileIT * fix(core): get real local ip (IGinX-THU#485) If ip is set to 0.0.0.0, find the real local ip. * ci: auto rename released package (IGinX-THU#498) * feat(sql): fix add storageengine (IGinX-THU#507) * fix(arrow): most of SQLSessionIT --------- Co-authored-by: unknown <[email protected]> Co-authored-by: ZM <[email protected]> Co-authored-by: jzl18thu <[email protected]> Co-authored-by: ZiyuZhao <[email protected]> Co-authored-by: Xu Yihao <[email protected]>
* fix concurrent SQLSessionIT * impl InFilter * fix most of SQLSessionIT * use row to hold udf
* test: remove temp tables and udf in tpc-h test * fix: tpc-h
* fix: copy logic operator in physical engine * fix: copy logic operator in physical engine
…nX-THU#533) * add RemoveNullColumnExecutor info * fix(arrow): filter push down * fix(arrow): filter push down
aqni
force-pushed
the
feat/core/replace-rowstream-with-arrow-tpch
branch
from
b80b156 to
7b14583
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.