Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 100 additions & 0 deletions docsite/docs/connectors/mariadb.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
---
sidebar_position: 5
---

# MariaDB

`intugle` integrates with MariaDB, allowing you to read data from your tables and views.

## Installation

To use `intugle` with MariaDB, you must install the optional dependencies:

```bash
pip install "intugle[mariadb]"
```

This installs the `mariadb` (MariaDB Connector/Python) and `sqlglot` libraries.

:::warning Linux Dependencies
On Linux, the MariaDB Connector/Python requires the MariaDB C Connector dependencies to be installed on your system.
For example, on Ubuntu/Debian:
```bash
sudo apt-get install libmariadb3 libmariadb-dev
```
On CentOS/RHEL:
```bash
sudo yum install mariadb-connector-c-devel
```
Please refer to the [MariaDB Connector/Python documentation](https://mariadb.com/docs/server/connect/programming-languages/python/install/#installing-mariadb-connector-python) for more details.
:::

## Configuration

To connect to your MariaDB database, you must provide connection credentials in a `profiles.yml` file at the root of your project. The adapter looks for a top-level `mariadb:` key.

**Example `profiles.yml`:**

```yaml
mariadb:
host: <your_mariadb_host>
port: 3306 # Default MariaDB port
user: <your_username>
password: <your_password>
database: <your_database_name>
```

## Usage

### Reading Data from MariaDB

To include a MariaDB table or view in your `SemanticModel`, define it in your input dictionary with `type: "mariadb"` and use the `identifier` key to specify the object name.

:::caution Important
The dictionary key for your dataset (e.g., `"CUSTOMERS"`) must exactly match the table or view name specified in the `identifier`.
:::

```python
from intugle import SemanticModel

datasets = {
"CUSTOMERS": {
"identifier": "CUSTOMERS", # Must match the key above
"type": "mariadb"
},
"ORDERS_VIEW": {
"identifier": "ORDERS_VIEW", # Can be a view
"type": "mariadb"
}
}

# Initialize the semantic model
sm = SemanticModel(datasets, domain="E-commerce")

# Build the model as usual
sm.build()
```

### Materializing Data Products

When you use the `DataProduct` class with a MariaDB connection, the resulting data product can be materialized as a new **table** or **view** directly within your target database.

```python
from intugle import DataProduct

etl_model = {
"name": "top_customers",
"fields": [
{"id": "CUSTOMERS.customer_id", "name": "customer_id"},
{"id": "CUSTOMERS.name", "name": "customer_name"},
]
}

dp = DataProduct()

# Materialize as a view (default)
dp.build(etl_model, materialize="view")

# Materialize as a table
dp.build(etl_model, materialize="table")
```
5 changes: 5 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,11 @@ mysql = [
"sqlglot>=27.20.0",
]

mariadb = [
"mariadb>=1.1.10",
"sqlglot>=27.20.0",
]

oracle = [
"oracledb>=2.0.0",
"sqlglot>=27.20.0",
Expand Down
1 change: 1 addition & 0 deletions src/intugle/adapters/factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ def is_safe_plugin_name(plugin_name: str) -> bool:
"intugle.adapters.types.databricks.databricks",
"intugle.adapters.types.postgres.postgres",
"intugle.adapters.types.mysql.mysql",
"intugle.adapters.types.mariadb.mariadb",
"intugle.adapters.types.sqlserver.sqlserver",
"intugle.adapters.types.sqlite.sqlite",
"intugle.adapters.types.oracle.oracle",
Expand Down
3 changes: 2 additions & 1 deletion src/intugle/adapters/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,14 @@ def get_dataset_data_type() -> type:

from intugle.adapters.types.databricks.models import DatabricksConfig
from intugle.adapters.types.duckdb.models import DuckdbConfig
from intugle.adapters.types.mariadb.models import MariaDBConfig
from intugle.adapters.types.oracle.models import OracleConfig
from intugle.adapters.types.postgres.models import PostgresConfig
from intugle.adapters.types.snowflake.models import SnowflakeConfig
from intugle.adapters.types.sqlite.models import SqliteConfig
from intugle.adapters.types.sqlserver.models import SQLServerConfig

DataSetData = pd.DataFrame | DuckdbConfig | SnowflakeConfig | DatabricksConfig | PostgresConfig | SQLServerConfig | SqliteConfig | OracleConfig
DataSetData = pd.DataFrame | DuckdbConfig | SnowflakeConfig | DatabricksConfig | PostgresConfig | SQLServerConfig | SqliteConfig | OracleConfig | MariaDBConfig
else:
# At runtime, this is dynamically determined
DataSetData = Any
Expand Down
Empty file.
Loading