Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebased to 432-SNAPSHOT and fixed the issue by reviewer. #7

Merged
merged 2,992 commits into from
Oct 31, 2023

Conversation

yuuteng
Copy link

@yuuteng yuuteng commented Jul 13, 2023

Rebased to 432-SNAPSHOT and fixed the issue by reviewer.

TODO:

  1. nit: Inline. => private static final Map<Integer, ColumnMapping> STANDARD_COLUMN_MAPPINGS = ImmutableMap.<Integer, ColumnMapping>builder()
  2. Why did you omit the handling for getForcedMappingToVarchar? => public Optional toColumnMapping(ConnectorSession session, Connection connection, JdbcTypeHandle typeHandle)
  3. These config properties are not tested. Please remove them from the initial PR. => private String httpProxy;
  4. TODO: Timestamp tests, can't fix the precision > 3.

b-slim and others added 23 commits October 18, 2023 19:03
Adds SESSION_AUTHORIZATION to ClientCapabilities, so that the
client side support for this feature can be identified using
the headers sent to Trino.
This upgrades Delta Lake version to 3.0.0.
Refactor TestRetryingConnectionFactory and TestLazyConnectionFactory
to use Dependency Injection Motivation behind this change is to be
able to later add dependent objects to tested class without a need
for major changes in the test class itself.
Extend the logic to specially treat metadata queries and not allocate
any memory for those, so they are scheduled immediatelly, to cover
all schematas from system catalog (previously only jdbc schama was
covered).
This bit of code is making the assumption that it's safe to remove the Sort because there are no exchanges yet. However, it's forgetting the fact that the data needs to be sorted, so later when the exchange is added this expectation is broken.
Implements column-wise hash calculations in FlatHashCompiler and changes
FlatGroupByHash to use it to implement a batched approach to first
computing a range of position hashes and then attempting to insert those
positions using the hashes that were precomputed.

By calculating position hashes in a columnar tarversal path, we can
avoid repeated expensive bounds checking and allow the JIT to unroll
loops into much more efficient forms.

Additionally, when attempting to insert the positions into the
FlatGroupByHash after precomputing the hash code we can start loading
the relevant portition of the hash table into memory sooner since we
aren't intermixing computing the hash with the memory accesses.
Once the table is created, attempt to get the stats a few times
while waiting for PostgreSQL to analyze the table and declare
success as soon as the stats are available.

Previously, every iteration of the test had a race condition
between creating the table and checking for stats, which led
to many of the iterations to fail.
The arguments are not valid for the test.
Depending on which order the test runs in, it may pick up the
schema mappings from other tests and fail. The mapping needs
to be initialized appropriately.
yuuteng and others added 26 commits October 31, 2023 12:49
Fully tested with Trino 406. Untested with 410.
Adding --add-opens to enable support for Apache Arrow via shared memory buffers.
TODO: testDataMappingSmokeTest: time and timestamp
testSetColumnTypes: time and timestamp
This reverts commit 82f9eb3f3811e8d90a482f5359e98e7c729afa17.
Fully tested with Trino 406. Untested with 410.
…with Snowflake server.

Disabled the testsetColumnTypes testing.
@yuuteng yuuteng force-pushed the bloomberg-snowflake branch from 5520b5f to b10a001 Compare October 31, 2023 12:05
@yuuteng yuuteng changed the title Rebased to 422-SNAPSHOT and fixed the issue by reviewer. Rebased to 432-SNAPSHOT and fixed the issue by reviewer. Oct 31, 2023
@dprophet dprophet merged commit 4536040 into bloomberg:snowflake Oct 31, 2023
1 check passed
@yuuteng yuuteng deleted the bloomberg-snowflake branch December 4, 2023 10:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.