Skip to content

Conversation

flyrain
Copy link
Contributor

@flyrain flyrain commented Sep 10, 2025

No description provided.

Comment on lines +83 to 85
into("regtests") {
from("${rootDir}/regtests/requirements.txt")
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's weird to have this directory in the binary distribution. We can make a symlink in the dir client/polaris, and point to it instead.


into("bin/client") {
from("${rootDir}/client/python") {
include("**/*")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably don't need everything under client/python. For example, tests are not needed. Can you confirm which are needed? cc @HonahX @MonkeyCanCode

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I think we should only publish wheel file produced by #2425. Reasoning is as client/python/pyproject.toml is invoking generate_clients.py which refs OpenAPI files outside client/python, people won't be able to build this anyway with only these files without restore the structure. Thus, I make more sense to me for publishing the wheel file instead which will have everything generated. You can do so via following:

make client-build

Once completed, you will then have a wheel file under client/python/dist/ such as following:

client/python/dist/polaris-1.0.0-cp313-cp313-macosx_15_0_arm64.whl

The content of the wheel files will then be following:

➜  polaris git:(main) unzip -l client/python/dist/polaris-1.0.0-cp313-cp313-macosx_15_0_arm64.whl
Archive:  client/python/dist/polaris-1.0.0-cp313-cp313-macosx_15_0_arm64.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
      789  01-01-2016 00:00   cli/__init__.py
     9979  01-01-2016 00:00   cli/command/__init__.py
     5362  01-01-2016 00:00   cli/command/catalog_roles.py
    19082  01-01-2016 00:00   cli/command/catalogs.py
     4239  01-01-2016 00:00   cli/command/namespaces.py
     5328  01-01-2016 00:00   cli/command/principal_roles.py
     6954  01-01-2016 00:00   cli/command/principals.py
     6138  01-01-2016 00:00   cli/command/privileges.py
     5808  01-01-2016 00:00   cli/command/profiles.py
    13973  01-01-2016 00:00   cli/constants.py
      789  01-01-2016 00:00   cli/options/__init__.py
    16669  01-01-2016 00:00   cli/options/option_tree.py
     9355  01-01-2016 00:00   cli/options/parser.py
     8964  01-01-2016 00:00   cli/polaris_cli.py
        0  01-01-2016 00:00   polaris/.keep
      790  01-01-2016 00:00   polaris/__init__.py
        0  01-01-2016 00:00   polaris/catalog/.keep
    11171  01-01-2016 00:00   polaris/catalog/__init__.py
        0  01-01-2016 00:00   polaris/catalog/api/.keep
     1059  01-01-2016 00:00   polaris/catalog/api/__init__.py
   377844  01-01-2016 00:00   polaris/catalog/api/catalog_api.py
    18839  01-01-2016 00:00   polaris/catalog/api/configuration_api.py
    58167  01-01-2016 00:00   polaris/catalog/api/generic_table_api.py
   421177  01-01-2016 00:00   polaris/catalog/api/iceberg_catalog_api.py
    18871  01-01-2016 00:00   polaris/catalog/api/iceberg_configuration_api.py
    26596  01-01-2016 00:00   polaris/catalog/api/iceberg_o_auth2_api.py
    27127  01-01-2016 00:00   polaris/catalog/api/o_auth2_api.py
   129569  01-01-2016 00:00   polaris/catalog/api/policy_api.py
    28362  01-01-2016 00:00   polaris/catalog/api_client.py
     1442  01-01-2016 00:00   polaris/catalog/api_response.py
    20753  01-01-2016 00:00   polaris/catalog/configuration.py
     7353  01-01-2016 00:00   polaris/catalog/exceptions.py
        0  01-01-2016 00:00   polaris/catalog/models/.keep
    10397  01-01-2016 00:00   polaris/catalog/models/__init__.py
     3557  01-01-2016 00:00   polaris/catalog/models/add_partition_spec_update.py
     3996  01-01-2016 00:00   polaris/catalog/models/add_schema_update.py
     3525  01-01-2016 00:00   polaris/catalog/models/add_snapshot_update.py
     3570  01-01-2016 00:00   polaris/catalog/models/add_sort_order_update.py
     3588  01-01-2016 00:00   polaris/catalog/models/add_view_version_update.py
     4070  01-01-2016 00:00   polaris/catalog/models/and_or_expression.py
     4715  01-01-2016 00:00   polaris/catalog/models/applicable_policy.py
     3462  01-01-2016 00:00   polaris/catalog/models/assert_create.py
     3618  01-01-2016 00:00   polaris/catalog/models/assert_current_schema_id.py
     3643  01-01-2016 00:00   polaris/catalog/models/assert_default_sort_order_id.py
     3604  01-01-2016 00:00   polaris/catalog/models/assert_default_spec_id.py
     3651  01-01-2016 00:00   polaris/catalog/models/assert_last_assigned_field_id.py
     3678  01-01-2016 00:00   polaris/catalog/models/assert_last_assigned_partition_id.py
     3720  01-01-2016 00:00   polaris/catalog/models/assert_ref_snapshot_id.py
     3479  01-01-2016 00:00   polaris/catalog/models/assert_table_uuid.py
     3726  01-01-2016 00:00   polaris/catalog/models/assert_view_uuid.py
     3612  01-01-2016 00:00   polaris/catalog/models/assign_uuid_update.py
     3635  01-01-2016 00:00   polaris/catalog/models/async_planning_result.py
     3814  01-01-2016 00:00   polaris/catalog/models/attach_policy_request.py
    12370  01-01-2016 00:00   polaris/catalog/models/base_update.py
     3834  01-01-2016 00:00   polaris/catalog/models/blob_metadata.py
     4129  01-01-2016 00:00   polaris/catalog/models/catalog_config.py
     4586  01-01-2016 00:00   polaris/catalog/models/commit_report.py
     5037  01-01-2016 00:00   polaris/catalog/models/commit_table_request.py
     3852  01-01-2016 00:00   polaris/catalog/models/commit_table_response.py
     4015  01-01-2016 00:00   polaris/catalog/models/commit_transaction_request.py
     4995  01-01-2016 00:00   polaris/catalog/models/commit_view_request.py
     5077  01-01-2016 00:00   polaris/catalog/models/completed_planning_result.py
     5264  01-01-2016 00:00   polaris/catalog/models/completed_planning_with_id_result.py
     6628  01-01-2016 00:00   polaris/catalog/models/content_file.py
     3630  01-01-2016 00:00   polaris/catalog/models/count_map.py
     3430  01-01-2016 00:00   polaris/catalog/models/counter_result.py
     3814  01-01-2016 00:00   polaris/catalog/models/create_generic_table_request.py
     3710  01-01-2016 00:00   polaris/catalog/models/create_namespace_request.py
     3714  01-01-2016 00:00   polaris/catalog/models/create_namespace_response.py
     4182  01-01-2016 00:00   polaris/catalog/models/create_policy_request.py
     5106  01-01-2016 00:00   polaris/catalog/models/create_table_request.py
     4564  01-01-2016 00:00   polaris/catalog/models/create_view_request.py
     5583  01-01-2016 00:00   polaris/catalog/models/data_file.py
     6366  01-01-2016 00:00   polaris/catalog/models/delete_file.py
     3676  01-01-2016 00:00   polaris/catalog/models/detach_policy_request.py
     3730  01-01-2016 00:00   polaris/catalog/models/empty_planning_result.py
     3469  01-01-2016 00:00   polaris/catalog/models/enable_row_lineage_update.py
     4769  01-01-2016 00:00   polaris/catalog/models/equality_delete_file.py
     3890  01-01-2016 00:00   polaris/catalog/models/error_model.py
    10131  01-01-2016 00:00   polaris/catalog/models/expression.py
     3793  01-01-2016 00:00   polaris/catalog/models/failed_planning_result.py
     3358  01-01-2016 00:00   polaris/catalog/models/false_expression.py
     7445  01-01-2016 00:00   polaris/catalog/models/fetch_planning_result.py
     3667  01-01-2016 00:00   polaris/catalog/models/fetch_scan_tasks_request.py
     4933  01-01-2016 00:00   polaris/catalog/models/fetch_scan_tasks_result.py
     1661  01-01-2016 00:00   polaris/catalog/models/file_format.py
     4642  01-01-2016 00:00   polaris/catalog/models/file_scan_task.py
     4513  01-01-2016 00:00   polaris/catalog/models/generic_table.py
     5610  01-01-2016 00:00   polaris/catalog/models/get_applicable_policies_response.py
     3884  01-01-2016 00:00   polaris/catalog/models/get_namespace_response.py
     3673  01-01-2016 00:00   polaris/catalog/models/iceberg_error_response.py
     3651  01-01-2016 00:00   polaris/catalog/models/iceberg_error_response1.py
     5474  01-01-2016 00:00   polaris/catalog/models/list_generic_tables_response.py
     4723  01-01-2016 00:00   polaris/catalog/models/list_namespaces_response.py
     5458  01-01-2016 00:00   polaris/catalog/models/list_policies_response.py
     5228  01-01-2016 00:00   polaris/catalog/models/list_tables_response.py
     4294  01-01-2016 00:00   polaris/catalog/models/list_type.py
     3751  01-01-2016 00:00   polaris/catalog/models/literal_expression.py
     4084  01-01-2016 00:00   polaris/catalog/models/load_credentials_response.py
     3670  01-01-2016 00:00   polaris/catalog/models/load_generic_table_response.py
     3633  01-01-2016 00:00   polaris/catalog/models/load_policy_response.py
     6636  01-01-2016 00:00   polaris/catalog/models/load_table_result.py
     4505  01-01-2016 00:00   polaris/catalog/models/load_view_result.py
     4603  01-01-2016 00:00   polaris/catalog/models/map_type.py
     3574  01-01-2016 00:00   polaris/catalog/models/metadata_log_inner.py
     5730  01-01-2016 00:00   polaris/catalog/models/metric_result.py
     4576  01-01-2016 00:00   polaris/catalog/models/model_schema.py
     3782  01-01-2016 00:00   polaris/catalog/models/not_expression.py
     3925  01-01-2016 00:00   polaris/catalog/models/notification_request.py
     1686  01-01-2016 00:00   polaris/catalog/models/notification_type.py
     1652  01-01-2016 00:00   polaris/catalog/models/null_order.py
     4179  01-01-2016 00:00   polaris/catalog/models/o_auth_error.py
     4934  01-01-2016 00:00   polaris/catalog/models/o_auth_token_response.py
     3710  01-01-2016 00:00   polaris/catalog/models/partition_field.py
     4059  01-01-2016 00:00   polaris/catalog/models/partition_spec.py
     3764  01-01-2016 00:00   polaris/catalog/models/partition_statistics_file.py
     1719  01-01-2016 00:00   polaris/catalog/models/plan_status.py
     5680  01-01-2016 00:00   polaris/catalog/models/plan_table_scan_request.py
     8391  01-01-2016 00:00   polaris/catalog/models/plan_table_scan_result.py
     5469  01-01-2016 00:00   polaris/catalog/models/policy.py
     4212  01-01-2016 00:00   polaris/catalog/models/policy_attachment_target.py
     3505  01-01-2016 00:00   polaris/catalog/models/policy_identifier.py
     4982  01-01-2016 00:00   polaris/catalog/models/position_delete_file.py
    17550  01-01-2016 00:00   polaris/catalog/models/primitive_type_value.py
     3537  01-01-2016 00:00   polaris/catalog/models/register_table_request.py
     3559  01-01-2016 00:00   polaris/catalog/models/remove_partition_specs_update.py
     3579  01-01-2016 00:00   polaris/catalog/models/remove_partition_statistics_update.py
     3499  01-01-2016 00:00   polaris/catalog/models/remove_properties_update.py
     3530  01-01-2016 00:00   polaris/catalog/models/remove_snapshot_ref_update.py
     3547  01-01-2016 00:00   polaris/catalog/models/remove_snapshots_update.py
     3543  01-01-2016 00:00   polaris/catalog/models/remove_statistics_update.py
     4019  01-01-2016 00:00   polaris/catalog/models/rename_table_request.py
     5748  01-01-2016 00:00   polaris/catalog/models/report_metrics_request.py
     5162  01-01-2016 00:00   polaris/catalog/models/scan_report.py
     5321  01-01-2016 00:00   polaris/catalog/models/scan_tasks.py
     3614  01-01-2016 00:00   polaris/catalog/models/set_current_schema_update.py
     3665  01-01-2016 00:00   polaris/catalog/models/set_current_view_version_update.py
     3646  01-01-2016 00:00   polaris/catalog/models/set_default_sort_order_update.py
     3612  01-01-2016 00:00   polaris/catalog/models/set_default_spec_update.py
     3745  01-01-2016 00:00   polaris/catalog/models/set_expression.py
     3473  01-01-2016 00:00   polaris/catalog/models/set_location_update.py
     3673  01-01-2016 00:00   polaris/catalog/models/set_partition_statistics_update.py
     3491  01-01-2016 00:00   polaris/catalog/models/set_properties_update.py
     4525  01-01-2016 00:00   polaris/catalog/models/set_snapshot_ref_update.py
     3821  01-01-2016 00:00   polaris/catalog/models/set_statistics_update.py
     4601  01-01-2016 00:00   polaris/catalog/models/snapshot.py
     3553  01-01-2016 00:00   polaris/catalog/models/snapshot_log_inner.py
     4325  01-01-2016 00:00   polaris/catalog/models/snapshot_reference.py
     4303  01-01-2016 00:00   polaris/catalog/models/snapshot_summary.py
     1624  01-01-2016 00:00   polaris/catalog/models/sort_direction.py
     3810  01-01-2016 00:00   polaris/catalog/models/sort_field.py
     3995  01-01-2016 00:00   polaris/catalog/models/sort_order.py
     3520  01-01-2016 00:00   polaris/catalog/models/sql_view_representation.py
     4585  01-01-2016 00:00   polaris/catalog/models/statistics_file.py
     3694  01-01-2016 00:00   polaris/catalog/models/storage_credential.py
     4951  01-01-2016 00:00   polaris/catalog/models/struct_field.py
     4221  01-01-2016 00:00   polaris/catalog/models/struct_type.py
     3526  01-01-2016 00:00   polaris/catalog/models/table_identifier.py
    11336  01-01-2016 00:00   polaris/catalog/models/table_metadata.py
     7060  01-01-2016 00:00   polaris/catalog/models/table_requirement.py
    20637  01-01-2016 00:00   polaris/catalog/models/table_update.py
     4200  01-01-2016 00:00   polaris/catalog/models/table_update_notification.py
     6053  01-01-2016 00:00   polaris/catalog/models/term.py
     3613  01-01-2016 00:00   polaris/catalog/models/timer_result.py
     2440  01-01-2016 00:00   polaris/catalog/models/token_type.py
     3758  01-01-2016 00:00   polaris/catalog/models/transform_term.py
     3354  01-01-2016 00:00   polaris/catalog/models/true_expression.py
     7384  01-01-2016 00:00   polaris/catalog/models/type.py
     3743  01-01-2016 00:00   polaris/catalog/models/unary_expression.py
     3580  01-01-2016 00:00   polaris/catalog/models/update_namespace_properties_request.py
     3981  01-01-2016 00:00   polaris/catalog/models/update_namespace_properties_response.py
     3717  01-01-2016 00:00   polaris/catalog/models/update_policy_request.py
     3565  01-01-2016 00:00   polaris/catalog/models/upgrade_format_version_update.py
     4136  01-01-2016 00:00   polaris/catalog/models/value_map.py
     3548  01-01-2016 00:00   polaris/catalog/models/view_history_entry.py
     5774  01-01-2016 00:00   polaris/catalog/models/view_metadata.py
     5589  01-01-2016 00:00   polaris/catalog/models/view_representation.py
     5507  01-01-2016 00:00   polaris/catalog/models/view_requirement.py
    11137  01-01-2016 00:00   polaris/catalog/models/view_update.py
     4892  01-01-2016 00:00   polaris/catalog/models/view_version.py
    10295  01-01-2016 00:00   polaris/catalog/rest.py
        0  01-01-2016 00:00   polaris/management/.keep
     6140  01-01-2016 00:00   polaris/management/__init__.py
        0  01-01-2016 00:00   polaris/management/api/.keep
      911  01-01-2016 00:00   polaris/management/api/__init__.py
   377204  01-01-2016 00:00   polaris/management/api/polaris_default_api.py
    28305  01-01-2016 00:00   polaris/management/api_client.py
     1442  01-01-2016 00:00   polaris/management/api_response.py
    19220  01-01-2016 00:00   polaris/management/configuration.py
     7278  01-01-2016 00:00   polaris/management/exceptions.py
        0  01-01-2016 00:00   polaris/management/models/.keep
     5487  01-01-2016 00:00   polaris/management/models/__init__.py
     3594  01-01-2016 00:00   polaris/management/models/add_grant_request.py
     6443  01-01-2016 00:00   polaris/management/models/authentication_parameters.py
     3576  01-01-2016 00:00   polaris/management/models/aws_iam_service_identity_info.py
     5645  01-01-2016 00:00   polaris/management/models/aws_storage_config_info.py
     4105  01-01-2016 00:00   polaris/management/models/azure_storage_config_info.py
     3630  01-01-2016 00:00   polaris/management/models/bearer_authentication_parameters.py
     7122  01-01-2016 00:00   polaris/management/models/catalog.py
     3465  01-01-2016 00:00   polaris/management/models/catalog_grant.py
     2982  01-01-2016 00:00   polaris/management/models/catalog_privilege.py
     4000  01-01-2016 00:00   polaris/management/models/catalog_properties.py
     4541  01-01-2016 00:00   polaris/management/models/catalog_role.py
     3788  01-01-2016 00:00   polaris/management/models/catalog_roles.py
     3752  01-01-2016 00:00   polaris/management/models/catalogs.py
     7005  01-01-2016 00:00   polaris/management/models/connection_config_info.py
     3591  01-01-2016 00:00   polaris/management/models/create_catalog_request.py
     3723  01-01-2016 00:00   polaris/management/models/create_catalog_role_request.py
     3975  01-01-2016 00:00   polaris/management/models/create_principal_request.py
     3759  01-01-2016 00:00   polaris/management/models/create_principal_role_request.py
     5087  01-01-2016 00:00   polaris/management/models/external_catalog.py
     3461  01-01-2016 00:00   polaris/management/models/file_storage_config_info.py
     3717  01-01-2016 00:00   polaris/management/models/gcp_storage_config_info.py
     3719  01-01-2016 00:00   polaris/management/models/grant_catalog_role_request.py
     3755  01-01-2016 00:00   polaris/management/models/grant_principal_role_request.py
     5798  01-01-2016 00:00   polaris/management/models/grant_resource.py
     3760  01-01-2016 00:00   polaris/management/models/grant_resources.py
     4598  01-01-2016 00:00   polaris/management/models/hadoop_connection_config_info.py
     4579  01-01-2016 00:00   polaris/management/models/hive_connection_config_info.py
     4870  01-01-2016 00:00   polaris/management/models/iceberg_rest_connection_config_info.py
     3618  01-01-2016 00:00   polaris/management/models/implicit_authentication_parameters.py
     3581  01-01-2016 00:00   polaris/management/models/namespace_grant.py
     2882  01-01-2016 00:00   polaris/management/models/namespace_privilege.py
     4005  01-01-2016 00:00   polaris/management/models/o_auth_client_credentials_parameters.py
     4539  01-01-2016 00:00   polaris/management/models/polaris_catalog.py
     3685  01-01-2016 00:00   polaris/management/models/policy_grant.py
     1825  01-01-2016 00:00   polaris/management/models/policy_privilege.py
     4717  01-01-2016 00:00   polaris/management/models/principal.py
     4845  01-01-2016 00:00   polaris/management/models/principal_role.py
     3748  01-01-2016 00:00   polaris/management/models/principal_roles.py
     4229  01-01-2016 00:00   polaris/management/models/principal_with_credentials.py
     3610  01-01-2016 00:00   polaris/management/models/principal_with_credentials_credentials.py
     3785  01-01-2016 00:00   polaris/management/models/principals.py
     3857  01-01-2016 00:00   polaris/management/models/reset_principal_request.py
     3606  01-01-2016 00:00   polaris/management/models/revoke_grant_request.py
     5021  01-01-2016 00:00   polaris/management/models/service_identity_info.py
     4349  01-01-2016 00:00   polaris/management/models/sig_v4_authentication_parameters.py
     6157  01-01-2016 00:00   polaris/management/models/storage_config_info.py
     3768  01-01-2016 00:00   polaris/management/models/table_grant.py
     1962  01-01-2016 00:00   polaris/management/models/table_privilege.py
     4441  01-01-2016 00:00   polaris/management/models/update_catalog_request.py
     3734  01-01-2016 00:00   polaris/management/models/update_catalog_role_request.py
     3725  01-01-2016 00:00   polaris/management/models/update_principal_request.py
     3742  01-01-2016 00:00   polaris/management/models/update_principal_role_request.py
     3756  01-01-2016 00:00   polaris/management/models/view_grant.py
     1771  01-01-2016 00:00   polaris/management/models/view_privilege.py
    10223  01-01-2016 00:00   polaris/management/rest.py
     2510  01-01-2016 00:00   polaris-1.0.0.dist-info/METADATA
      106  01-01-2016 00:00   polaris-1.0.0.dist-info/WHEEL
       48  01-01-2016 00:00   polaris-1.0.0.dist-info/entry_points.txt
    24980  01-01-2016 00:00   polaris-1.0.0.dist-info/RECORD
---------                     -------
  2658296                     251 files

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is binary distribution, I think it makes more sense to only installable wheels anyway? The polaris script shall be able to pick up the package after pip install, we can update the README to reflect that

people won't be able to build this anyway with only these files without restore the structure.

@MonkeyCanCode That's a good point. How do you feel creating a symlink inside the client/python that points to the spec folder? This way we still have single source of truth during the development but the python folder will contain everything needed for build when copy-paste to other places.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I can raise a PR later this weekend. Then the distribution build (along with wheel build) will also work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR is up: #2557

Maybe get the distribution (wheel, sdist, or both) instead of manually cheery pick?


into("bin/client") {
from("${rootDir}/client/python") {
include("**/*")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is binary distribution, I think it makes more sense to only installable wheels anyway? The polaris script shall be able to pick up the package after pip install, we can update the README to reflect that

people won't be able to build this anyway with only these files without restore the structure.

@MonkeyCanCode That's a good point. How do you feel creating a symlink inside the client/python that points to the spec folder? This way we still have single source of truth during the development but the python folder will contain everything needed for build when copy-paste to other places.

}

into("regtests") {
from("${rootDir}/regtests/requirements.txt")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks a little bit weird to include things from the regtests. The only information in that file is the poetry version, may be we could refactor to move the requirement inside client/python

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated: The current pyproject.toml has defined its required poetry version:

requires-poetry = "==2.1.4"

So I think we do not need to include this requirements.txt anymore

@MonkeyCanCode
Copy link
Contributor

@flyrain @HonahX PR for support sdist and handle spec direcotry: #2557

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants