Skip to content

Commit fa16389

Browse files
ebyhrebyhr
committed
Add DuckDB connector
1 parent ea52006 commit fa16389

File tree

18 files changed

+1750
-0
lines changed

18 files changed

+1750
-0
lines changed

‎.github/workflows/ci.yml

+2
Original file line numberDiff line numberDiff line change
@@ -340,6 +340,7 @@ jobs:
340340
!:trino-delta-lake,
341341
!:trino-docs,
342342
!:trino-druid,
343+
!:trino-duckdb,
343344
!:trino-elasticsearch,
344345
!:trino-exasol,
345346
!:trino-faulttolerant-tests,
@@ -456,6 +457,7 @@ jobs:
456457
- { modules: plugin/trino-delta-lake, profile: cloud-tests }
457458
- { modules: plugin/trino-delta-lake, profile: fte-tests }
458459
- { modules: plugin/trino-druid }
460+
- { modules: plugin/trino-duckdb }
459461
- { modules: plugin/trino-elasticsearch }
460462
- { modules: plugin/trino-exasol }
461463
- { modules: plugin/trino-google-sheets }

‎core/trino-server/src/main/provisio/trino.xml

+6
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,12 @@
7878
</artifact>
7979
</artifactSet>
8080

81+
<artifactSet to="plugin/duckdb">
82+
<artifact id="${project.groupId}:trino-duckdb:zip:${project.version}">
83+
<unpack />
84+
</artifact>
85+
</artifactSet>
86+
8187
<artifactSet to="plugin/elasticsearch">
8288
<artifact id="${project.groupId}:trino-elasticsearch:zip:${project.version}">
8389
<unpack />

‎docs/src/main/sphinx/connector.md

+1
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ Cassandra <connector/cassandra>
1414
ClickHouse <connector/clickhouse>
1515
Delta Lake <connector/delta-lake>
1616
Druid <connector/druid>
17+
DuckDB <connector/duckdb>
1718
Elasticsearch <connector/elasticsearch>
1819
Exasol <connector/exasol>
1920
Faker <connector/faker>

‎docs/src/main/sphinx/connector/duckdb.md

+207
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
# DuckDB connector
2+
3+
```{raw} html
4+
<img src="../_static/img/duckdb.png" class="connector-logo">
5+
```
6+
7+
The DuckDB connector allows querying and creating tables in an external
8+
[DuckDB](https://duckdb.org/) instance. This can be used to join data between
9+
different systems like DuckDB and Hive, or between two different
10+
DuckDB instances.
11+
12+
## Configuration
13+
14+
To configure the DuckDB connector, create a catalog properties file
15+
in `etc/catalog` named, for example, `example.properties`, to
16+
mount the DuckDB connector as the `duckdb` catalog.
17+
Create the file with the following contents, replacing the
18+
connection properties as appropriate for your setup:
19+
20+
```none
21+
connector.name=duckdb
22+
connection-url=jdbc:duckdb://<path>
23+
connection-user=root
24+
connection-password=secret
25+
```
26+
27+
### Multiple DuckDB servers
28+
29+
The DuckDB connector can only access a single database within
30+
a DuckDB instance. Thus, if you have multiple DuckDB servers,
31+
or want to connect to multiple DuckDB servers, you must configure
32+
multiple instances of the DuckDB connector.
33+
34+
(duckdb-type-mapping)=
35+
## Type mapping
36+
37+
Because Trino and DuckDB each support types that the other does not, this
38+
connector {ref}`modifies some types <type-mapping-overview>` when reading or
39+
writing data. Data types may not map the same way in both directions between
40+
Trino and the data source. Refer to the following sections for type mapping in
41+
each direction.
42+
43+
List of [DuckDB data types](https://duckdb.org/docs/sql/data_types/overview.html).
44+
45+
### DuckDB type to Trino type mapping
46+
47+
The connector maps DuckDB types to the corresponding Trino types following
48+
this table:
49+
50+
:::{list-table} DuckDB type to Trino type mapping
51+
:widths: 30, 30, 40
52+
:header-rows: 1
53+
54+
* - DuckDB type
55+
- Trino type
56+
- Notes
57+
* - `BOOLEAN`
58+
- `BOOLEAN`
59+
-
60+
* - `TINYINT`
61+
- `TINYINT`
62+
-
63+
* - `SMALLINT`
64+
- `SMALLINT`
65+
-
66+
* - `INTEGER`
67+
- `INTEGER`
68+
-
69+
* - `BIGINT`
70+
- `BIGINT`
71+
-
72+
* - `FLOAT`
73+
- `REAL`
74+
-
75+
* - `DOUBLE`
76+
- `DOUBLE`
77+
-
78+
* - `DECIMAL`
79+
- `DECIMAL`
80+
- Default precision and scale are (18,3).
81+
* - `VARCHAR`
82+
- `VARCHAR`
83+
-
84+
* - `DATE`
85+
- `DATE`
86+
-
87+
:::
88+
89+
No other types are supported.
90+
91+
### Trino type to DuckDB type mapping
92+
93+
The connector maps Trino types to the corresponding DuckDB types following
94+
this table:
95+
96+
:::{list-table} Trino type to DuckDB type mapping
97+
:widths: 30, 30, 40
98+
:header-rows: 1
99+
100+
* - Trino type
101+
- DuckDB type
102+
- Notes
103+
* - `BOOLEAN`
104+
- `BOOLEAN`
105+
-
106+
* - `TINYINT`
107+
- `TINYINT`
108+
-
109+
* - `SMALLINT`
110+
- `SMALLINT`
111+
-
112+
* - `INTEGER`
113+
- `INTEGER`
114+
-
115+
* - `BIGINT`
116+
- `BIGINT`
117+
-
118+
* - `REAL`
119+
- `REAL`
120+
-
121+
* - `DOUBLE`
122+
- `DOUBLE`
123+
-
124+
* - `DECIMAL`
125+
- `DECIMAL`
126+
-
127+
* - `CHAR`
128+
- `VARCHAR`
129+
-
130+
* - `VARCHAR`
131+
- `VARCHAR`
132+
-
133+
* - `DATE`
134+
- `DATE`
135+
-
136+
:::
137+
138+
No other types are supported.
139+
140+
```{include} jdbc-type-mapping.fragment
141+
```
142+
143+
(duckdb-sql-support)=
144+
## SQL support
145+
146+
The connector provides read access and write access to data and metadata in
147+
a DuckDB database. In addition to the {ref}`globally available
148+
<sql-globally-available>` and {ref}`read operation <sql-read-operations>`
149+
statements, the connector supports the following features:
150+
151+
- {doc}`/sql/insert`
152+
- {doc}`/sql/delete`
153+
- {doc}`/sql/truncate`
154+
- {doc}`/sql/create-table`
155+
- {doc}`/sql/create-table-as`
156+
- {doc}`/sql/drop-table`
157+
- {doc}`/sql/alter-table`
158+
- {doc}`/sql/create-schema`
159+
- {doc}`/sql/drop-schema`
160+
161+
### Procedures
162+
163+
```{include} jdbc-procedures-flush.fragment
164+
```
165+
```{include} procedures-execute.fragment
166+
```
167+
168+
### Table functions
169+
170+
The connector provides specific [table functions](/functions/table) to
171+
access DuckDB.
172+
173+
(duckdb-query-function)=
174+
#### `query(varchar) -> table`
175+
176+
The `query` function allows you to query the underlying database directly. It
177+
requires syntax native to DuckDB, because the full query is pushed down and
178+
processed in DuckDB. This can be useful for accessing native features which
179+
are not available in Trino or for improving query performance in situations
180+
where running a query natively may be faster.
181+
182+
Find details about the SQL support of DuckDB that you can use in the query in
183+
the [DuckDB SQL Command
184+
Reference](https://duckdb.org/docs/sql/query_syntax/select) and
185+
other statements and functions.
186+
187+
```{include} query-passthrough-warning.fragment
188+
```
189+
190+
As a simple example, query the `example` catalog and select an entire table:
191+
192+
```
193+
SELECT
194+
*
195+
FROM
196+
TABLE(
197+
example.system.query(
198+
query => 'SELECT
199+
*
200+
FROM
201+
tpch.nation'
202+
)
203+
);
204+
```
205+
206+
```{include} query-table-function-ordering.fragment
207+
```

‎docs/src/main/sphinx/static/img/duckdb.png

12.1 KB
Loading

0 commit comments

Comments
 (0)