v0.4.0

What's Changed

• io: add support for role arn and external id s3 props by @mattheusv in #553
• fix: ensure S3 and GCS integ tests are conditionally compiled only when the storage-s3 and storage-gcs features are enabled by @sdd in #552
• docs: fix main iceberg example by @jdockerty in #554
• io: add support to set assume role session name by @mattheusv in #555
• test: refactor datafusion test with memory catalog by @FANNG1 in #557
• chore: add clean job in Makefile by @ChinoUkaegbu in #561
• docs: Fix build website permission changed by @Xuanwo in #564
• Object Cache: caches parsed Manifests and ManifestLists for performance by @sdd in #512
• Update the paths by @Fokko in #569
• docs: Add links for released crates by @Xuanwo in #570
• Python: Use hatch for dependency management by @sungwy in #572
• Ensure that RestCatalog passes user config to FileIO by @sdd in #476
• Move zlib and unicode licenses to allow by @Fokko in #566
• website: Update links for 0.3.0 by @Xuanwo in #573
• feat(timestamp_ns): Implement timestamps with nanosecond precision by @Sl1mb0 in #542
• fix: correct partition-id to field-id in UnboundPartitionField by @FANNG1 in #576
• fix: Update sqlx from 0.8.0 to 0.8.1 by @FANNG1 in #584
• chore(deps): Update typed-builder requirement from 0.19 to 0.20 by @dependabot in #582
• Expose Transforms to Python Binding by @sungwy in #556
• chore(deps): Bump crate-ci/typos from 1.23.6 to 1.24.1 by @dependabot in #583
• Table Scan: Add Row Group Skipping by @sdd in #558
• chore: bump crate-ci/typos to 1.24.3 by @sdlarsen in #598
• feat: SQL Catalog - namespaces by @callum-ryan in #534
• feat: Add more fields in FileScanTask by @Xuanwo in #609
• chore(deps): Bump crate-ci/typos from 1.24.3 to 1.24.5 by @dependabot in #616
• fix: Less Panics for Snapshot timestamps by @c-thiel in #614
• feat: partition compatibility by @c-thiel in #612
• feat: SortOrder methods should take schema ref if possible by @c-thiel in #613
• feat: add client.region by @jdockerty in #623
• fix: Correctly calculate highest_field_id in schema by @liurenjie1024 in #590
• Feat: Normalize TableMetadata by @c-thiel in #611
• refactor(python): Expose transform as a submodule for pyiceberg_core by @Xuanwo in #628
• feat: support projection pushdown for datafusion iceberg by @FANNG1 in #594
• chore: Bump opendal to 0.50 by @Xuanwo in #634
• feat: add Sync to TransformFunction by @xxchan in #638
• feat: expose arrow type <-> iceberg type by @xxchan in #637
• doc: improve FileIO doc by @xxchan in #642
• chore(deps): Bump crate-ci/typos from 1.24.5 to 1.24.6 by @dependabot in #640
• Migrate to arrow-* v53 by @sdd in #626
• feat: expose remove_all in FileIO by @xxchan in #643
• feat (datafusion integration): convert datafusion expr filters to Iceberg Predicate by @a-agmon in #588
• feat: Add NamespaceIdent.parent() by @c-thiel in #641
• Table Scan: Add Row Selection Filtering by @sdd in #565
• fix: compile error due to merge stale PR by @xxchan in #646
• scan: change ErrorKind when table dont have spanshots by @mattheusv in #608
• fix: avoid to create operator of memory storage every time by @ZENOTME in #635
• feat (datafusion): making IcebergTableProvider public to be used without a catalog by @a-agmon in #650
• test (datafusion): add test for table provider creation by @a-agmon in #651
• fix: page index evaluator min/max args inverted by @sdd in #648
• chore: fix typo in FileIO Schemes by @wcy-fdu in #653
• fix: TableUpdate Snapshot deserialization for v1 by @c-thiel in #656
• feat: Reassign field ids for schema by @c-thiel in #615
• feat: add gcp oauth support by @twuebi in #654
• fix(arrow): Use new ParquetMetaDataReader instead by @Xuanwo in #661
• chore(deps): bump typos crate to 1.25.0 by @matthewwillian in #662
• RecordBatchTransformer: Handle schema migration and column re-ordering in table scans by @sdd in #602
• docs: installation of the new iceberg_catalog_rest added to the docs by @nishant-sachdeva in #355
• feat(datafusion): Support pushdown more datafusion exprs to Iceberg by @FANNG1 in #649
• feat: Derive PartialEq for FileScanTask by @Xuanwo in #660
• feat: SQL Catalog - Tables by @callum-ryan in #610
• ci: Allow install a non-debian-packaged Python package by @Xuanwo in #666
• docs: README uses iceberg-rust instead of we by @caicancai in #667
• chore(deps): Bump crate-ci/typos from 1.25.0 to 1.26.0 by @dependabot in #668
• feat: Add equality delete writer by @Dysprosium0626 in #372
• Revert "feat: Add equality delete writer (#372)" by @Xuanwo in #672
• ci: Fix CI for bindings python by @Xuanwo in #678
• fix: OpenDAL is_exist => exists by @sdd in #680
• feat: Expose ManifestEntry status by @zheilbron in #681
• feat: allow empty projection in table scan by @sundy-li in #677
• chore(deps): Bump crate-ci/typos from 1.26.0 to 1.26.8 by @dependabot in #683
• fix: bump parquet minor version by @xxchan in #684
• fix(type): fix type promote to ignore field name. by @chenzl25 in #685
• feat: implement IcebergTableProviderFactory for datafusion by @yukkit in #600
• feat: Safer PartitionSpec & SchemalessPartitionSpec by @c-thiel in #645
• chore(deps): Bump crate-ci/typos from 1.26.8 to 1.27.0 by @dependabot in #687
• feat: TableMetadata accessors for current ids of Schema, Snapshot and SortOrder by @c-thiel in #688
• chore: upgrade to DataFusion 43 by @gruuya in #691
• chore(deps): Bump crate-ci/typos from 1.27.0 to 1.27.3 by @dependabot in #693
• feat: Expose length of Iterators by @c-thiel in #692
• feat: Implement TableRequirement checks by @c-thiel in #689
• feat: Add ViewUpdate to catalog by @c-thiel in #690
• chore: update .asf.yaml by @c-thiel in #701
• datafusion: Create table provider for a snapshot. by @ryzhyk in #707
• Add Python Release Action to publish pyiceberg_core dist to Pypi by @sungwy in #705
• chore: Mark last-field-id as deprecated by @Fokko in #715
• TableMetadataBuilder by @c-thiel in #587
• Add fallback attribute to all strip_options. by @ryzhyk in #708
• fix: Remove check of last_column_id by @liurenjie1024 in #717
• Fix error running data fusion queries - Physical input schema should be...

v0.3.0

What's Changed

• Smooth out release steps by @Fokko in #197
• refactor: remove support of manifest list format as a list of file path by @Dysprosium0626 in #201
• refactor: remove unwraps by @odysa in #196
• Fix: add required rust version in cargo.toml by @dp-0 in #193
• Fix the REST spec version by @Fokko in #198
• feat: Add Sync + Send to Catalog trait by @ZhengLin-Li in #202
• feat: Make thrift transport configurable by @DeaconDesperado in #194
• Add UnboundSortOrder by @fqaiser94 in #115
• ci: Add workflow for publish by @Xuanwo in #218
• Add workflow for cargo audit by @sdd in #217
• docs: Add basic README for all crates by @Xuanwo in #215
• Follow naming convention from Iceberg's Java and Python implementations by @s-akhtar-baig in #204
• doc: Add download page by @liurenjie1024 in #219
• chore(deps): Update derive_builder requirement from 0.13.0 to 0.20.0 by @dependabot in #203
• test: add FileIO s3 test by @odysa in #220
• ci: Ignore RUSTSEC-2023-0071 for no actions to take by @Xuanwo in #222
• feat: Add expression builder and display. by @liurenjie1024 in #169
• chord: Add IssueNavigationLink for RustRover by @stream2000 in #230
• minor: Fix double API doc by @viirya in #226
• feat: add UnboundPredicate::negate() by @sdd in #228
• fix: Remove deprecated methods to pass ci by @liurenjie1024 in #234
• Implement basic Parquet data file reading capability by @sdd in #207
• chore: doc-test as a target by @liurenjie1024 in #235
• feat: add parquet writer by @ZENOTME in #176
• Add hive metastore catalog support (part 1/2) by @marvinlanhenke in #237
• chore: Enable projects. by @liurenjie1024 in #247
• refactor: Make plan_files as asynchronous stream by @viirya in #243
• feat: Implement binding expression by @liurenjie1024 in #231
• Implement Display instead of ToString by @lewiszlw in #256
• add rewrite_not by @marvinlanhenke in #263
• feat: init TableMetadataBuilder by @ZENOTME in #262
• Rename stat_table to table_exists in Catalog trait by @lewiszlw in #257
• feat (static table): implement a read-only table struct loaded from metadata by @a-agmon in #259
• feat: implement OAuth for catalog rest client by @TennyZhuang in #254
• docs: annotate precision and length to primitive types by @waynexia in #270
• build: Restore CI by making parquet and arrow version consistent by @viirya in #280
• Metadata Serde + default partition_specs and sort_orders by @marvinlanhenke in #272
• feat: make optional oauth param configurable by @himadripal in #278
• fix: enable public access to ManifestEntry properties by @a-agmon in #284
• feat: Implement the conversion from Arrow Schema to Iceberg Schema by @viirya in #258
• Rename function name to add_manifests by @viirya in #293
• Modify Bind calls so that they don't consume self and instead return a new struct, leaving the original unmoved by @sdd in #290
• Add hive metastore catalog support (part 2/2) by @marvinlanhenke in #285
• feat: implement prune column for schema by @Dysprosium0626 in #261
• chore(deps): Update reqwest requirement from ^0.11 to ^0.12 by @dependabot in #296
• Glue Catalog: Basic Setup + Test Infra (1/3) by @marvinlanhenke in #294
• feat: rest client respect prefix prop by @TennyZhuang in #297
• fix: HMS Catalog missing properties fn create_namespace by @marvinlanhenke in #303
• fix: renaming FileScanTask.data_file to data_manifest_entry by @a-agmon in #300
• feat: Make OAuth token server configurable by @whynick1 in #305
• feat: Glue Catalog - namespace operations (2/3) by @marvinlanhenke in #304
• feat: add transform_literal by @ZENOTME in #287
• feat: Complete predicate builders for all operators. by @QuakeWang in #276
• feat: Support customized header in Rest catalog client by @whynick1 in #306
• fix: chrono dep by @odysa in #274
• feat: Read Parquet data file with projection by @viirya in #245
• Fix day timestamp micro by @marvinlanhenke in #312
• feat: support uri redirect in rest client by @TennyZhuang in #310
• refine: seperate parquet reader and arrow convert by @ZENOTME in #313
• chore: upgrade to rust-version 1.77.1 by @marvinlanhenke in #316
• Support identifier warehouses by @Fokko in #308
• feat: Project transform by @marvinlanhenke in #309
• Add Struct Accessors to BoundReferences by @sdd in #317
• Use str args rather than String in transform to avoid needing to clone strings by @sdd in #325
• chore(deps): Update pilota requirement from 0.10.0 to 0.11.0 by @dependabot in #327
• chore(deps): Bump peaceiris/actions-mdbook from 1 to 2 by @dependabot in #332
• chore(deps): Bump peaceiris/actions-gh-pages from 3.9.3 to 4.0.0 by @dependabot in #333
• chore(deps): Bump apache/skywalking-eyes from 0.5.0 to 0.6.0 by @dependabot in #328
• Add BoundPredicateVisitor (alternate version) by @sdd in #334
• add InclusiveProjection Visitor by @sdd in #335
• feat: Implement the conversion from Iceberg Schema to Arrow Schema by @ZENOTME in #277
• Simplify expression when doing {and,or} operations by @Fokko in #339
• feat: Glue Catalog - table operations (3/3) by @marvinlanhenke in #314
• chore: update roadmap by @marvinlanhenke in #336
• Add ManifestEvaluator, used to filter manifests in table scans by @sdd in #322
• feat: init iceberg writer by @ZENOTME in #275
• Implement manifest filtering in TableScan by @sdd in #323
• Refactor: Extract partition_filters from ManifestEvaluator by @marvinlanhenke in #360
• Basic Integration with Datafusion by @marvinlanhenke in #324
• refactor: cache partition_schema in fn plan_files() by @marvinlanhenke in #362
• fix (manifest-list): added serde aliases to support both forms conventions by @a-agmon in #365
• feat: Extract FileRead and FileWrite trait by @Xuanwo in #364
• feat: Convert predicate to arrow filter and push down to parquet reader by @viirya in #295
• chore(deps): Update datafusion requirement from 37.0.0 to 38.0.0 by @dependabot in #369
• chore(deps): Update itertools requirement from 0.12 to 0.13 by @dependabot in #376
• Add InclusiveMetricsEvaluator by @sdd in #347
• Rename V2 spec names by @gupteaj in #380
• feat: make file scan task serializable by @ZENOTME in #377
• ...

Iceberg-rust 0.2.0

What's Changed

• chore: Setup project layout by @Xuanwo in #1
• ci: Fix version for apache/skywalking-eyes/header by @Xuanwo in #4
• feat: Implement serialize/deserialize for datatypes by @JanKaul in #6
• docs: Add CONTRIBUTING and finish project setup by @Xuanwo in #7
• feat: Add lookup tables to StructType by @JanKaul in #12
• feat: Implement error handling by @Xuanwo in #13
• chore: Use HashMap instead of BTreeMap for storing fields by id in StructType by @amogh-jahagirdar in #14
• chore: Change iceberg into workspace by @Xuanwo in #15
• feat: Use macro to define from error. by @liurenjie1024 in #17
• feat: Introduce schema definition. by @liurenjie1024 in #19
• refactor: Align data type with other implementation. by @liurenjie1024 in #21
• chore: Ignore .idea by @liurenjie1024 in #27
• feat: Implement Iceberg values by @JanKaul in #20
• feat: Define schema post order visitor. by @liurenjie1024 in #25
• feat: Add transform by @liurenjie1024 in #26
• fix: Fix build break in main branch by @liurenjie1024 in #30
• fix: Update github configuration to avoid conflicting merge by @liurenjie1024 in #31
• chore(deps): Bump apache/skywalking-eyes from 0.4.0 to 0.5.0 by @dependabot in #35
• feat: Table metadata by @JanKaul in #29
• feat: Add utility methods to help conversion between literals. by @liurenjie1024 in #38
• [comment] should be IEEE 754 rather than 753 by @zhjwpku in #39
• fix: Add doc test action by @liurenjie1024 in #44
• chore: Ping toolchain version by @liurenjie1024 in #48
• feat: Introduce conversion between iceberg schema and avro schema by @liurenjie1024 in #40
• feat: Allow Schema Serialization/deserialization by @y0psolo in #46
• chore: Add cargo sort check by @liurenjie1024 in #51
• chore(deps): Bump actions/checkout from 3 to 4 by @dependabot in #58
• Metadata integration tests by @JanKaul in #57
• feat: Introduce FileIO by @liurenjie1024 in #53
• feat: Add Catalog API by @Xuanwo in #54
• feat: support transform function by @ZENOTME in #42
• chore(deps): Update ordered-float requirement from 3.7.0 to 4.0.0 by @dependabot in #64
• feat: Add public methods for catalog related structs by @Xuanwo in #63
• minor: Upgrade to latest toolchain by @liurenjie1024 in #68
• chore(deps): Update opendal requirement from 0.39 to 0.40 by @dependabot in #65
• refactor: Make directory for catalog by @liurenjie1024 in #69
• feat: support read Manifest List by @ZENOTME in #56
• chore(deps): Update apache-avro requirement from 0.15 to 0.16 by @dependabot in #71
• fix: avro bytes test for Literal by @JanKaul in #80
• chore(deps): Update opendal requirement from 0.40 to 0.41 by @dependabot in #84
• feat: manifest list writer by @barronw in #76
• feat: First version of rest catalog. by @liurenjie1024 in #78
• chore(deps): Update typed-builder requirement from ^0.17 to ^0.18 by @dependabot in #87
• feat: Implement load table api. by @liurenjie1024 in #89
• chroes:Manage dependencies using workspace. by @my-vegetable-has-exploded in #93
• minor: Provide Debug impl for pub structs #73 by @DeaconDesperado in #92
• feat: support ser/deser of value by @ZENOTME in #82
• fix: Migrate from tempdir to tempfile crate by @cdaudt in #91
• chore(deps): Update opendal requirement from 0.41 to 0.42 by @dependabot in #101
• chore(deps): Update itertools requirement from 0.11 to 0.12 by @dependabot in #102
• Replace i64 with DateTime by @fqaiser94 in #94
• feat: Implement create table and update table api for rest catalog. by @liurenjie1024 in #97
• Fix compile failures by @fqaiser94 in #105
• feat: replace 'Builder' with 'TypedBuilder' for 'Snapshot' by @xiaoyang-sde in #110
• chore: Upgrade uuid manually and remove pinned version by @liurenjie1024 in #108
• chore: Add cargo build and build guide by @manuzhang in #111
• feat: Add hms catalog layout by @Xuanwo in #112
• feat: support UnboundPartitionSpec by @my-vegetable-has-exploded in #106
• test: Add integration tests for rest catalog. by @liurenjie1024 in #109
• chore(deps): Update opendal requirement from 0.42 to 0.43 by @dependabot in #116
• feat: suport read/write Manifest by @ZENOTME in #79
• test: Remove binary manifest list avro file by @liurenjie1024 in #118
• refactor: Conversion between literal and json should depends on type. by @liurenjie1024 in #120
• fix: fix parse partitions in manifest_list by @ZENOTME in #122
• feat: Add website layout by @Xuanwo in #130
• feat: Expression system. by @liurenjie1024 in #132
• website: Fix typo in book.toml by @Xuanwo in #136
• Set ghp_{pages,path} properties by @Fokko in #138
• chore: Upgrade toolchain to 1.75.0 by @liurenjie1024 in #140
• feat: Add roadmap and features status in README.md by @liurenjie1024 in #134
• Remove publish: section from .asf.yaml by @Fokko in #141
• chore(deps): Bump peaceiris/actions-gh-pages from 3.9.2 to 3.9.3 by @dependabot in #143
• chore(deps): Update opendal requirement from 0.43 to 0.44 by @dependabot in #142
• docs: Change homepage to rust.i.a.o by @Xuanwo in #146
• feat: Introduce basic file scan planning. by @liurenjie1024 in #129
• chore: Update contributing guide. by @liurenjie1024 in #163
• chore: Update reader api status by @liurenjie1024 in #162
• #154 : Add homepage to Cargo.toml by @hiirrxnn in #160
• Add formatting for toml files by @Tyler-Sch in #167
• chore(deps): Update env_logger requirement from 0.10.0 to 0.11.0 by @dependabot in #170
• feat: init file writer interface by @ZENOTME in #168
• fix: Manifest parsing should consider schema evolution. by @liurenjie1024 in #171
• docs: Add release guide for iceberg-rust by @Xuanwo in #147
• fix: Ignore negative statistics value by @liurenjie1024 in #173
• feat: Add user guide for website. by @liurenjie1024 in #178
• chore(deps): Update derive_builder requirement from 0.12.0 to 0.13.0 by @dependabot in #175
• refactor: Replace unwrap by @odysa in #183
• feat: add handwritten serialize by @odysa in #185
• Fix: avro schema names for manifest and manifest_list by @JanKaul in #182
• feat: Bump hive_metastore to use pure rust thrift impl volo by @Xuanwo in https://github.com/apache/iceberg-r...