Skip to content

Commit

Permalink
docs(datasets): add connection caching improvement
Browse files Browse the repository at this point in the history
Signed-off-by: Deepyaman Datta <[email protected]>
  • Loading branch information
deepyaman authored Dec 4, 2024
1 parent 3c74f84 commit 5dc22ed
Showing 1 changed file with 11 additions and 8 deletions.
19 changes: 11 additions & 8 deletions kedro-datasets/RELEASE.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,25 @@
# Upcoming Release 6.0.0

## Major features and improvements
- Added functionality to save Pandas DataFrame directly to Snowflake, facilitating seemless `.csv` ingestion
- Added Python 3.9, 3.10 and 3.11 support for SnowflakeTableDataset

- Added functionality to save pandas DataFrames directly to Snowflake, facilitating seamless `.csv` ingestion.
- Added Python 3.9, 3.10 and 3.11 support for `snowflake.SnowflakeTableDataset`.
- Enabled connection sharing between `ibis.FileDataset` and `ibis.TableDataset` instances, thereby allowing nodes to save data loaded by one to the other (as long as they share the same connection configuration).
- Added the following new **experimental** datasets:

| Type | Description | Location |
| --------------------------------- | ------------------------------------------------------ | ---------------------------------------- |
| `databricks.ExternalTableDataset` | A dataset for accessing external tables in Databricks. | `kedro_datasets_experimental.databricks` |
| Type | Description | Location |
| --------------------------------- | -------------------------------------------------------------------------- | ----------------------------------------- |
| `databricks.ExternalTableDataset` | A dataset for accessing external tables in Databricks. | `kedro_datasets_experimental.databricks` |
| `safetensors.SafetensorsDataset` | A dataset for securely saving and loading files in the SafeTensors format. | `kedro_datasets_experimental.safetensors` |


## Bug fixes and other changes
- Implemented Snowflake's (local testing framework)[https://docs.snowflake.com/en/developer-guide/snowpark/python/testing-locally] for testing purposes

- Implemented Snowflake's [local testing framework](https://docs.snowflake.com/en/developer-guide/snowpark/python/testing-locally) for testing purposes.
- Improved the dependency management for Spark-based datasets by refactoring the Spark and Databricks utility functions used across the datasets.
- Add deprecation warning for `tracking.MetricsDataset` and `tracking.JSONDataset`.
- Added deprecation warning for `tracking.MetricsDataset` and `tracking.JSONDataset`.

## Breaking Changes

- Demoted `video.VideoDataset` from core to experimental dataset.

## Community contributions
Expand Down

0 comments on commit 5dc22ed

Please sign in to comment.