HIVE-27275: Provide docker image for HMS #5834
Conversation
deniskuzZ
force-pushed
the
HIVE-27275
branch
3 times, most recently
from
38c537c to
ae15116
Compare
asf-ci-hive
added
tests failed
tests pending
and removed
tests pending
tests failed
labels
asf-ci-hive
added
tests pending
tests failed
and removed
tests failed
tests pending
labels
deniskuzZ
changed the title
deniskuzZ
changed the title
asf-ci-hive
added
tests pending
tests failed
and removed
tests failed
tests pending
labels
| @@ -875,7 +875,7 @@ public enum ConfVars { | |||
| EVENT_DB_LISTENER_CLEAN_STARTUP_WAIT_INTERVAL("metastore.event.db.listener.clean.startup.wait.interval", | |||
| "hive.metastore.event.db.listener.clean.startup.wait.interval", 1, TimeUnit.DAYS, | |||
| "Wait interval post start of metastore after which the cleaner thread starts to work"), | |||
| EVENT_DB_NOTIFICATION_API_AUTH("metastore.metastore.event.db.notification.api.auth", | |||
There was a problem hiding this comment.
not sure if it's good to change the property name that existing in old releases
There was a problem hiding this comment.
No one used such a name, but a hive alias
standalone-metastore/metastore-server/src/docker/conf/metastore-site.xml
Show resolved
Hide resolved
asf-ci-hive
added
tests pending
tests passed
and removed
tests failed
tests pending
labels
| docker run -d -p 9083:9083 --env DB_DRIVER=postgres \ | ||
| --env SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=org.postgresql.Driver -Djavax.jdo.option.ConnectionURL=jdbc:postgresql://postgres:5432/metastore_db -Djavax.jdo.option.ConnectionUserName=hive -Djavax.jdo.option.ConnectionPassword=password" \ | ||
| --mount source=warehouse,target=/opt/hive/data/warehouse \ | ||
| --mount type=bind,source=`mvn help:evaluate -Dexpression=settings.localRepository -q -DforceStdout`/org/postgresql/postgresql/42.7.3/postgresql-42.7.3.jar,target=/opt/hive/lib/postgres.jar \ |
There was a problem hiding this comment.
I think it is very convenient if major JDBC drivers are legally included.
There was a problem hiding this comment.
i don't think we should embed the drivers, that would affect the docker image size + won't allow users to pass a custom version of a driver
Instead they could place the drivers under the POSTGRES_LOCAL_PATH
There was a problem hiding this comment.
Another point is we might have license issues if we include all the major drivers, though I'm not sure.
There was a problem hiding this comment.
Now, I think we can create another ticket if we are truly sure we need it.
Probably, k8s users prefer to bake it into their images rather than having a tricky manifest. The default drivers can eliminate the extra work. They can still overwrite the driver when they want to use their own.
Regarding the license, the licenses of some drivers, such as PostgreSQL, are compatible if I remember correctly.
asf-ci-hive
added
tests unstable
tests pending
and removed
tests pending
tests unstable
labels
|
| </dependency> | ||
| <dependency> | ||
| <groupId>org.apache.hive</groupId> | ||
| <artifactId>hive-standalone-metastore-rest-catalog</artifactId> |
There was a problem hiding this comment.
when I use the command mvn clean install -DskipTests -Pdocker to build the image, an issue:
[ERROR] The projects in the reactor contain a cyclic reference: Edge between 'Vertex{label='org.apache.hive:hive-standalone-metastore-rest-catalog:4.1.0-SNAPSHOT'}' and 'Vertex{label='org.apache.hive:hive-standalone-metastore-server:4.1.0-SNAPSHOT'}' introduces to cycle in the graph org.apache.hive:hive-standalone-metastore-server:4.1.0-SNAPSHOT --> org.apache.hive:hive-standalone-metastore-rest-catalog:4.1.0-SNAPSHOT --> org.apache.hive:hive-standalone-metastore-server:4.1.0-SNAPSHOT -> [Help 1]
There was a problem hiding this comment.
@dengzhhu653, you should execute the above command from within meteastore-server; otherwise, you'll encounter a cyclic reference problem.
The problem is that instead of a code refactor and extraction of auth classes into common, metastore-server was added as a direct dependency for rest-catalog. I think that is out-of-scope of this PR, but we should address it a future.
What changes were proposed in this pull request?
Lightweight Docker image for the Hive Metastore
Why are the changes needed?
Simplified ddeployment
Does this PR introduce any user-facing change?
No
How was this patch tested?
Locally