Apache Hive Airflow Providers

[Rabz-9]

Hello,

I'm currently facing an issue when using the "apache-airflow-providers-apache-hive" basically it works well when I specify the host of my HiveServer2 the fact is that in my company we don't use HiveServer2 URL directly but we use Zookeeper to take advantage of the HA and make sure that we'll be routing to a HiveServer2 that is alive. Moreover we have several zookeeper instance that are in charge of our connection to our Hive instance and the URL string looks like this :

jdbc:hive2://zk1host:zk1port,zk2host:zk2port,zk3host:zk3port

But it seems that in the code there are some limitations and it doesn't allow user to add several host comma separated in the "Host" connection parameter.

Here the beeline validator that's used and that raised the exception :

def _validate_beeline_parameters(self, conn):
    if ":" in conn.host or "/" in conn.host or ";" in conn.host:
        raise Exception(
            f"The host used in beeline command ({conn.host}) should not contain ':/;' characters)"
        )
    try:
        int_port = int(conn.port)
        if int_port <= 0 or int_port > 65535:
            raise Exception(f"The port used in beeline command ({conn.port}) should be in range 0-65535)")
    except (ValueError, TypeError) as e:
        raise Exception(f"The port used in beeline command ({conn.port}) should be a valid integer: {e})")
    if ";" in conn.schema:
        raise Exception(
            f"The schema used in beeline command ({conn.schema}) should not contain ';' character)"
        )

Does anyone know if there is some limitation or reason for that ? If no do you think that I can do a PR to improve this ?

Thanks for your help

[Taragolis]

This validation was added in #29502 and I guess for close CVE-2023-25696

[Rabz-9]

But for HA when we work in a Production environment create a Hive connection using the "Hive Client Wrapper" and be able to only specify one HS2 where we can have thousand (on my ecosystem) it doesn't make sense that why updating this validation and make the possibility to specify several Zookeeper Hosts or HiveServer2 instances would be great

What is your think on that ?

I can make it work using this :

jdbc:hive2://zk1host,zk2host,zk3host:zk3port

when I specify only the port for the last host

[Taragolis]

I've just answer why this check exists.
You always could create PR and propose solution which won't return vulnerability.

[potiuk]

@Rabz-9 - as @Taragolis wrote (and I think we had the discussion before) - yes. It's possible to add it and we would welcome PRs to make it possible in a safe and robust way.