Test singlenode-ldap tests #25143
Draft
Test singlenode-ldap tests #25143
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Jalpreet Singh Nanda (:imjalpreet) <[email protected]> Co-authored-by: Shijin K <[email protected]>
Open new ports for Hadoop 3 HDFS NameNode and DataNode
Hive 1 and 2 use “bucketing v1”, which relies on a weaker hash function for floating-point values. However, HDP 2.6 images use MapReduce (MR) by default, which unconditionally creates one file per bucket. In contrast, Hive 3 uses an improved hash function under “bucketing v2”, but it defaults to using Tez, which does not generate files for empty buckets. As a result, even with 32 buckets configured, we may occasionally see fewer than 32 output files.
Presto’s default configuration interacts with Hive/Hadoop using the hive user (-DHADOOP_USER_NAME=hive). To ensure consistency and avoid permission issues, Tempto should also use the same hive user for data setup and as the default user for executing queries. Also, modify temporary storage location in hdfs where user hive has access
Statistics are collected by default in Hive 3.x Added a new column list for the drop column test case, as Hive 3.x forbids certain coersions Enable metastore impersonation in singlenode-kerberos-hdfs-impersonation environment Drop table using same user that was used to create table. Otherwise, with Hive 3 and Hadoop 3 the table directory won't be deleted Co-authored-by: Shijin K <[email protected]>
On Hadoop 3.x, the new schema location on HDFS is owned by the hive user and has rwxr-xr-x permissions. As a result, when HDFS impersonation is enabled, the alice user is unable to create a new table within that schema.
Hive 3 uses bucketing v2 by default and Presto does not support it yet
Update openldap image to imjalpreet/centos7-oj8-openldap:latest
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release Notes
Please follow release notes guidelines and fill in the release notes below.
== RELEASE NOTES ==