[kerberized HDFS] One of --proxy-user or --principal can be provided

[fuhaiq]

嗨～目前采用计算引擎为 Spark On K8S，数据在远端开启了 Kerberos 的 Hadoop 集群上。已经可以通过 spark-sql 客户端正常读写 Hive 表了。现在 K8S 上部署了 Kyuubi 作为 JDBC Server，并按照 Kerberos文档 做了相关配置

kyuubi-defaults.conf

kyuubi.frontend.bind.host 0.0.0.0
kyuubi.frontend.bind.port 10000
kyuubi.authentication KERBEROS
kyuubi.ha.zookeeper.quorum zk-cs
kyuubi.kinit.principal hive/[email protected]
kyuubi.kinit.keytab /opt/hive.keytab

spark-defaults.conf

spark.master k8s://https://kubernetes.default:443
spark.submit.deployMode client
spark.kubernetes.kerberos.enable true
spark.kerberos.keytab /opt/hive.keytab
spark.kerberos.principal hive/[email protected]
spark.kubernetes.kerberos.krb5.path /etc/krb5.conf

通过 beeline -u "jdbc:hive2://kyuubi:10000/;principal=hive/[email protected]" 连接报错如下：

org.apache.spark.SparkException: One of --proxy-user or --principal can be provided

于是又找到了这个文档 ，意思是访问 Kerberos 认证的 Hadoop 有两种方式（只能二选一）

1. Submits with current kerberos user and extra SparkSubmit argument --proxy-user.
2. Submits with spark.kerberos.principal and spark.kerberos.keytab specified.

根据 spark 配置，是第二种方式。但是通过报错，可以看到 kyuubi 把 --proxy-user hive 也传给了 Spark，造成了上述错误。还需要哪些配置么？

[pan3793]

Generally, we encourage using proxy user mode instead of providing superuser keytab to Spark application directly for security purposes. The reason is, that users can access all resources including keytab if you provided throw Scala/Spark api, in other words, providing a superuser keytab to Spark application means exposing your superuser keytab to anyone.

In practice, I know there are two approaches that have been adopted widely,

1. Job submitter service(Kyuubi plays this role) holds the superuser keytab, and uses proxy user to submit jobs
2. Job submitter service holds every user keytabs, and uses their own keytab to submit jobs

You should choose one of two.

[fuhaiq]

谢谢～已经参考建议 1 的方式使用 kyuubi，目前工作正常了。其中 kyuubi 持有超级管理员的 keytab，相关配置如下：

kyuubi.authentication KERBEROS
kyuubi.kinit.principal hive/[email protected]
kyuubi.kinit.keytab /opt/hive.keytab

然后将 spark 中关于 kerberos 的配置关闭

spark.master k8s://https://kubernetes.default:443
spark.submit.deployMode client
# spark.kubernetes.kerberos.enable true
# spark.kerberos.keytab /opt/hive.keytab
# spark.kerberos.principal hive/[email protected]
# spark.kubernetes.kerberos.krb5.path /etc/krb5.conf

最后在 beeline 客户端传入 kerberos 信息，这里使用 hive.keytab 提交 spark sql（后续可以使用不同的 keytab 表示不同的用户）
beeline -u "jdbc:hive2://kyuubi:10000/;principal=hive/[email protected]#spark.kerberos.principal=hive/[email protected];spark.kerberos.keytab=/opt/hive.keytab;spark.kubernetes.kerberos.enable=true;spark.kubernetes.kerberos.krb5.path /etc/krb5.conf"

建议的方式是这样么？谢谢

[pan3793]

Yeah, you got it.

[ulysses-you]

I agree with @pan3793 , and for the quick workaround, you can pass principal and keytab through jdbc url. For example:

beeline -u "jdbc:hive2://kyuubi:10000/;principal=hive/[email protected]#spark.kerberos.principal=hive/[email protected];spark.kerberos.keytab=/opt/hive.keytab"

[fuhaiq]

非常感谢