feat: replace rowstream with arrow #13

aqni · 2024-12-04T15:38:31Z

No description provided.

#449) * add framework operator executor * count rows * format * impl row to arrow batch and format * modify pipeline executor * format

- 实现物理算子Projector（对应逻辑算子Project & Reorder & Rename & AddSchemaprefix） - 改造没有fetchsize的SQL session接口 TODO：需要修改

* avoid copy in projector * feat(core): add policy of single fragment * feat(core): impl multiply and sum

* wip * binary & unary expression implementation * license

* wip * binary & unary expression implementation * license * multiple * revert multiply expr

* add TODO * refactor cast

* wip * binary & unary expression implementation * license * multiple * revert multiply expr * implement constant & bracket expr; bool filter

* logical & casewhen * restore test * restore test * some change

part of hashjoin executor limit executor fetch-all executor

limit part of hash-join fetch-all task

markjoin of hashjoin other filter of hashjoin

* fix measure close time * finish stage4

…492)

…d lazy materialization (#496) * feat(arrow): like const optimize * feat(arrow): IntHashMap for HashJoin * Revert "feat(arrow): IntHashMap for HashJoin" This reverts commit b022975. * store * store

* empty res * fix insert sub * fix insert sub

* empty res * fix insert sub * fix insert sub * fix transform * fix empty select

* empty res * fix insert sub * fix insert sub * fix transform * fix empty select * fix import & pysession * fix export & PySessionIT * fix PySessionIT

* empty res * fix insert sub * fix insert sub * fix transform * fix empty select * fix import & pysession * fix export & PySessionIT * fix PySessionIT * install pyarrow

aqni and others added 30 commits September 14, 2024 16:48

setup allocator for RequestContext

849acf4

store

cf95b00

add some TODO

82cab1b

feat(arrow): add framework of operator executor and impl row-to-column (

e984b88

#449) * add framework operator executor * count rows * format * impl row to arrow batch and format * modify pipeline executor * format

feat(core): impl Projector (#452)

5d04f2b

- 实现物理算子Projector（对应逻辑算子Project & Reorder & Rename & AddSchemaprefix） - 改造没有fetchsize的SQL session接口 TODO：需要修改

feat(core): release resources correctly && adjust some sessions (#455)

728ff46

start to finish stage 2 (#464)

2c5bd8d

* avoid copy in projector * feat(core): add policy of single fragment * feat(core): impl multiply and sum

remove type declaration in RowTransform

e50d7dc

feat(arrow): row mapping implementation in arrow version (#466)

12bca48

* wip * binary & unary expression implementation * license

feat(arrow):revert multiply expr (#467)

816f7fb

* wip * binary & unary expression implementation * license * multiple * revert multiply expr

feat(core): impl filter (#468)

2d33e22

feat: refactor accumulator (#469)

fa2c9cc

refactor: cast (#471)

6ee3c0d

* add TODO * refactor cast

refactor time accumulator (#472)

cd2e14c

fix insert (#474)

f666df3

fix insert again (#475)

72a882d

feat(arrow): implement constant & bracket expr; bool filter (#479)

987f91c

* wip * binary & unary expression implementation * license * multiple * revert multiply expr * implement constant & bracket expr; bool filter

feat(arrow): impl group-by without distinct (#480)

e203ce2

feat(arrow): impl sort inner batch (#482)

537da40

feat(arrow): logical & casewhen (#483)

2cb7bb5

* logical & casewhen * restore test * restore test * some change

feat(arrow): impl tpch-h q3 (#486)

f218db1

part of hashjoin executor limit executor fetch-all executor

fix (#487)

4bcb4d8

limit part of hash-join fetch-all task

feat(arrow): impl TPC-H q4 (#489)

7eb903b

markjoin of hashjoin other filter of hashjoin

feat(arrow): stage4 (#490)

67ba828

* fix measure close time * finish stage4

feat(arrow): short circuit logic, ConstantPool and constant function (#…

a9ae96d

…492)

feat(arrow): short circuit logic, constant pool, constant function an…

f4171b0

…d lazy materialization (#496) * feat(arrow): like const optimize * feat(arrow): IntHashMap for HashJoin * Revert "feat(arrow): IntHashMap for HashJoin" This reverts commit b022975. * store * store

fix(arrow): max, min, first, last

32fae0c

fix(arrow): add prefixed key column in join result

0695627

feat(arrow): impl remain operator by converting batch to rows (#499)

1b19a0d

refactor: add hasNext method for BatchStream (#500)

3fca3c5

shinyano and others added 6 commits February 27, 2025 09:39

fix(arrow): fix empty res with header && insert from select (#585)

7afe655

* empty res * fix insert sub * fix insert sub

fix: count point 0

f7ead63

fix: parallel RowTransform

e8b5d6d

fix: NewSessionIT

c13bce5

fix: TagIT

b485068

feat(arrow): transform && empty res (#586)

ae37792

* empty res * fix insert sub * fix insert sub * fix transform * fix empty select

aqni closed this Feb 27, 2025

aqni reopened this Feb 27, 2025

aqni added 6 commits February 27, 2025 18:56

fix: RestAnnotationIT

de99493

fix: RestAnnotationIT

3036300

fix: CompactionIT

72abcbb

fix: UDFIT

a776a99

fix: TPCH

1b5a0c8

fix: SQLSessionIT

257f820

aqni closed this Feb 27, 2025

aqni reopened this Feb 27, 2025

aqni and others added 14 commits February 27, 2025 22:28

fix: UDFIT

05c2d47

fix: TPCH

be6f701

fix: DBCE

79247d1

fix: IT of Influxdb

df3cdec

fix: avoid transfer Arrow to Row in TPCH Q15

266f4bf

fix: add TPCH Warmup repeats and records

b1af7cf

Update SQLWarmupIT.java

e0cbe8c

fix last commit

18786b4

feat(arrow): export & PySessionIT (#588)

cdc80b6

* empty res * fix insert sub * fix insert sub * fix transform * fix empty select * fix import & pysession * fix export & PySessionIT * fix PySessionIT

format

e41cf90

remove incorrect opt rule

c1cb3fc

fix(arrow): install pyarrow (#590)

40b042e

* empty res * fix insert sub * fix insert sub * fix transform * fix empty select * fix import & pysession * fix export & PySessionIT * fix PySessionIT * install pyarrow

merge main

5b150a8

fix: UdfIT (#592)

d24e8fa

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: replace rowstream with arrow #13

feat: replace rowstream with arrow #13

aqni commented Dec 4, 2024

feat: replace rowstream with arrow #13

Are you sure you want to change the base?

feat: replace rowstream with arrow #13

Conversation

aqni commented Dec 4, 2024