Skip to content

Commit a99bd5b

Browse files
teejteej
authored
v0.1.4 release (#17)
- Bumped version to 0.1.4 - Added `FutureGrant`, `ReplicationGroup`, `EventTable`, Stages (`InternalStage`, `ExternalStage`), and Streams (`TableStream`, `ViewStream`, and `StageStream`) - Re-enabled SQL parsing support - Added experimental support for fields that force replacement
1 parent e141b18 commit a99bd5b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+1291
-418
lines changed

‎Makefile

+7-2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.PHONY: install install-dev test integration style check clean
1+
.PHONY: install install-dev test integration style check clean build
22
EDITION ?= standard
33

44
install:
@@ -21,4 +21,9 @@ check: style test
2121

2222
clean:
2323
rm -rf build dist *.egg-info
24-
find . -name "__pycache__" -type d -exec rm -rf {} +
24+
find . -name "__pycache__" -type d -exec rm -rf {} +
25+
26+
build:
27+
mkdir -p dist
28+
zip -vrX dist/titan-$(shell python setup.py -V).zip titan/
29+

‎README.md

+35-27
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# `Titan Core` - Snowflake infrastructure as code
22

33
<div align="center">
4-
<img src="./images/github.png" style="padding-bottom: 20px; width: 620px;"/>
4+
<img src="./images/github-explainer.png" style="padding-bottom: 20px; width: 830px;"/>
55
</div>
66

77
Titan Core helps you provision, deploy, and secure resources in Snowflake. It replaces infrastructure tools like Terraform or Schemachange.
@@ -181,20 +181,24 @@ Titan Core simplifies this process with a declarative Python approach. It allows
181181
| Database | ✅ | ✅ |
182182
| External Access Integration | ✅ | ❌ |
183183
| External Volume | ❌ | ❌ |
184+
| Grant | ✅ | ❌ |
185+
| ↳ Privilege Grant | ✅ | ❌ |
186+
| ↳ Future Grant | ✅ | ❌ |
184187
| Network Policy | ✅ | ❌ |
185188
| Notification Integration | WIP | ❌ |
186-
| ↪ Email | ✅ | ❌ |
187-
| ↪ AWS | ❌ | ❌ |
188-
| ↪ Azure | ❌ | ❌ |
189-
| ↪ GCP | ❌ | ❌ |
189+
| ↳ Email | ✅ | ❌ |
190+
| ↳ AWS | ❌ | ❌ |
191+
| ↳ Azure | ❌ | ❌ |
192+
| ↳ GCP | ❌ | ❌ |
193+
| Replication Group | ✅ | ❌ |
190194
| Resource Monitor | ✅ | ❌ |
191195
| Role | ✅ | ✅ |
192196
| Security Integration | ❌ | ❌ |
193197
| Share | ❌ | ❌ |
194198
| Storage Integration | WIP | ❌ |
195-
| ↪ AWS | ✅ | ❌ |
196-
| ↪ Azure | ❌ | ❌ |
197-
| ↪ GCP | ❌ | ❌ |
199+
| ↳ AWS | ✅ | ❌ |
200+
| ↳ Azure | ✅ | ❌ |
201+
| ↳ GCP | ✅ | ❌ |
198202
| User | ✅ | ✅ |
199203
| Warehouse | ✅ | ✅ |
200204
| | | |
@@ -205,19 +209,17 @@ Titan Core simplifies this process with a declarative Python approach. It allows
205209
| **Schema Resources** | | |
206210
| Alert | ✅ | ❌ |
207211
| Dynamic Table | ✅ | ❌ |
212+
| Event Table | ✅ | ❌ |
208213
| External Function | ✅ | ❌ |
209214
| External Stage | ✅ | ❌ |
210215
| External Table | ❌ | ❌ |
211216
| Failover Group | ✅ | ❌ |
212217
| File Format | ❌ | ❌ |
213-
| ↪ CSV | ❌ | ❌ |
214-
| ↪ JSON | ❌ | ❌ |
215-
| ↪ AVRO | ❌ | ❌ |
216-
| ↪ ORC | ❌ | ❌ |
217-
| ↪ Parquet | ❌ | ❌ |
218-
| Grant | WIP | ❌ |
219-
| ↪ Privilege Grant | ✅ | ❌ |
220-
| ↪ Future Grant | ❌ | ❌ |
218+
| ↳ CSV | ❌ | ❌ |
219+
| ↳ JSON | ❌ | ❌ |
220+
| ↳ AVRO | ❌ | ❌ |
221+
| ↳ ORC | ❌ | ❌ |
222+
| ↳ Parquet | ❌ | ❌ |
221223
| Iceberg Table | ❌ | ❌ |
222224
| Image Repository | ❌ | ❌ |
223225
| Internal Stage | ✅ | ❌ |
@@ -235,23 +237,29 @@ Titan Core simplifies this process with a declarative Python approach. It allows
235237
| Service | ❌ | ❌ |
236238
| Session Policy | ❌ | ❌ |
237239
| Stage | ✅ | ❌ |
240+
| ↳ External | ✅ | ❌ |
241+
| ↳ Internal | ✅ | ❌ |
238242
| Stored Procedure | WIP | ❌ |
239-
| ↪ Java | ❌ | ❌ |
240-
| ↪ Javascript | ❌ | ❌ |
241-
| ↪ Python | ✅ | ❌ |
242-
| ↪ Scala | ❌ | ❌ |
243-
| ↪ SQL | ❌ | ❌ |
244-
| Stream | ✅ | ❌ |
243+
| ↳ Java | ❌ | ❌ |
244+
| ↳ Javascript | ❌ | ❌ |
245+
| ↳ Python | ✅ | ❌ |
246+
| ↳ Scala | ❌ | ❌ |
247+
| ↳ SQL | ❌ | ❌ |
248+
| Stream | WIP | ❌ |
249+
| ↳ External Table | ❌ | ❌ |
250+
| ↳ Stage | ✅ | ❌ |
251+
| ↳ Table | ✅ | ❌ |
252+
| ↳ View | ✅ | ❌ |
245253
| Streamlit | ❌ | ❌ |
246254
| Table | ✅ | ❌ |
247255
| Tag | ✅ | ❌ |
248256
| Task | ✅ | ❌ |
249257
| User-Defined Function | WIP | ❌ |
250-
| ↪ Java | ❌ | ❌ |
251-
| ↪ Javascript | ✅ | ❌ |
252-
| ↪ Python | ✅ | ❌ |
253-
| ↪ Scala | ❌ | ❌ |
254-
| ↪ SQL | ❌ | ❌ |
258+
| ↳ Java | ❌ | ❌ |
259+
| ↳ Javascript | ✅ | ❌ |
260+
| ↳ Python | ✅ | ❌ |
261+
| ↳ Scala | ❌ | ❌ |
262+
| ↳ SQL | ❌ | ❌ |
255263
| View | ✅ | ❌ |
256264

257265

‎images/github-explainer.png

552 KB
Loading

‎images/github.png

-106 KB
Binary file not shown.

‎pyproject.toml

+1-1
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ venv = ''
1717
line-length = 120
1818

1919
[tool.pytest.ini_options]
20-
# addopts = "-n 4"
20+
addopts = "-n 8"
2121
markers = [
2222
"requires_snowflake: Mark a test as requiring a Snowflake connection.",
2323
"enterprise: Mark a test that only works on Enterprise Edition Snowflake.",

‎scripts/install

+1-1
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ RETURNS OBJECT NOT NULL
33
LANGUAGE PYTHON
44
RUNTIME_VERSION = '3.9'
55
PACKAGES = ('snowflake-snowpark-python', 'inflection', 'pyparsing')
6-
IMPORTS = ('@titan_aws/releases/titan-0.1.3.zip')
6+
IMPORTS = ('@titan_aws/releases/titan-0.1.4.zip')
77
HANDLER = 'titan.spi.install'
88
EXECUTE AS CALLER
99
CALL install()

‎setup.py

+2-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
setup(
44
name="titan",
5-
version="0.1.3",
5+
version="0.1.4",
66
description="Snowflake infrastructure as code",
77
long_description=open("README.md").read(),
88
long_description_content_type="text/markdown",
@@ -33,6 +33,7 @@
3333
"black",
3434
"codespell==2.2.6",
3535
"pytest-cov",
36+
"pytest-profiling",
3637
"pytest-xdist",
3738
"pytest>=6.0",
3839
"ruff",

‎tests/fixtures/json/account.json

Whitespace-only changes.

‎tests/fixtures/json/event_table.json

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"name": "EXAMPLE_EVENT_TABLE",
3+
"cluster_by": null,
4+
"data_retention_time_in_days": 1,
5+
"max_data_extension_time_in_days": 1,
6+
"change_tracking": true,
7+
"default_ddl_collation": "en_US",
8+
"copy_grants": false,
9+
"comment": "This is an example event table for logging user events.",
10+
"tags": null
11+
}

‎tests/fixtures/json/external_stage.json

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
{
2+
"name": "EXTERNAL_STAGE_EXAMPLE",
3+
"url": "s3://titan-snowflake/",
4+
"owner": "SYSADMIN",
5+
"type": "EXTERNAL",
6+
"storage_integration": null,
7+
"encryption": {
8+
"type": "AWS_SSE_KMS",
9+
"kms_key_id": "alias/MyAliasName"
10+
},
11+
"file_format": null,
12+
"directory": {
13+
"enable": true,
14+
"refresh_on_create": true
15+
},
16+
"copy_options": {
17+
"on_error": "CONTINUE",
18+
"size_limit": 100,
19+
"purge": true
20+
},
21+
"tags": null,
22+
"comment": "This is an example of an external stage"
23+
}

‎tests/fixtures/json/external_table_stream.json

+10
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"name": "SOME_EXTERNAL_TABLE_STREAM",
3+
"on_external_table": "my_ext_table",
4+
"owner": "SYSADMIN",
5+
"copy_grants": null,
6+
"at": null,
7+
"before": null,
8+
"insert_only": true,
9+
"comment": null
10+
}

‎tests/fixtures/json/future_grant.json

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"priv": "USAGE",
3+
"on_type": "SCHEMA",
4+
"in_type": "DATABASE",
5+
"in_name": "STATIC_DATABASE",
6+
"to": "STATIC_ROLE",
7+
"grant_option": false
8+
}

‎tests/fixtures/json/grant.json

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"priv": "USAGE",
3+
"on_type": "DATABASE",
4+
"on": "STATIC_DATABASE",
5+
"to": "STATIC_ROLE",
6+
"owner": "SYSADMIN",
7+
"grant_option": false
8+
}

‎tests/fixtures/json/internal_stage.json

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
{
2+
"name": "INTERNAL_STAGE_EXAMPLE",
3+
"owner": "SYSADMIN",
4+
"type": "INTERNAL",
5+
"encryption": {
6+
"type": "SNOWFLAKE_SSE"
7+
},
8+
"file_format": null,
9+
"directory": {
10+
"enable": true,
11+
"refresh_on_create": false
12+
},
13+
"copy_options": {
14+
"on_error": "CONTINUE",
15+
"size_limit": 100,
16+
"purge": true,
17+
"return_failed_only": false,
18+
"match_by_column_name": "CASE_SENSITIVE",
19+
"enforce_length": true,
20+
"truncatecolumns": false,
21+
"force": false
22+
},
23+
"tags": null,
24+
"comment": "This is an example of an internal stage."
25+
}

‎tests/fixtures/json/replication_group.json

+19
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{
2+
"name": "SOME_REPLICATION_GROUP",
3+
"object_types": [
4+
"DATABASES",
5+
"SHARES"
6+
],
7+
"allowed_databases": [
8+
"db1"
9+
],
10+
"allowed_shares": [
11+
"s1"
12+
],
13+
"allowed_integration_types": null,
14+
"allowed_accounts": [
15+
"myorg.myaccount2"
16+
],
17+
"ignore_edition_check": null,
18+
"replication_schedule": "10 MINUTE"
19+
}

‎tests/fixtures/json/schema.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"name": "TEST_SCHEMA",
33
"transient": false,
44
"managed_access": false,
5-
"data_retention_time_in_days": null,
5+
"data_retention_time_in_days": 1,
66
"max_data_extension_time_in_days": 14,
77
"default_ddl_collation": null,
88
"tags": null,

‎tests/fixtures/json/stage_stream.json

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
{
2+
"name": "SOME_STAGE_STREAM",
3+
"on_stage": "STATIC_STAGE",
4+
"owner": "SYSADMIN",
5+
"copy_grants": null,
6+
"comment": null
7+
}

‎tests/fixtures/json/stream.json

Whitespace-only changes.

‎tests/fixtures/json/table_stream.json

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"name": "SOME_TABLE_STREAM",
3+
"on_table": "STATIC_TABLE",
4+
"owner": "SYSADMIN",
5+
"copy_grants": null,
6+
"at": {
7+
"stream": "STATIC_STREAM"
8+
},
9+
"before": null,
10+
"append_only": null,
11+
"show_initial_rows": null,
12+
"comment": null
13+
}

‎tests/fixtures/json/view_stream.json

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"name": "SOME_VIEW_STREAM",
3+
"on_view": "STATIC_VIEW",
4+
"owner": "SYSADMIN",
5+
"copy_grants": null,
6+
"at": null,
7+
"before": null,
8+
"append_only": null,
9+
"show_initial_rows": null,
10+
"comment": null
11+
}

‎tests/fixtures/sql/account.sql

-16
This file was deleted.

‎tests/fixtures/sql/event_table.sql

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
CREATE EVENT TABLE my_events;

‎tests/fixtures/sql/future_grant.sql

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
-- Future Schema Privileges
2+
GRANT CREATE ROW ACCESS POLICY ON FUTURE SCHEMAS IN DATABASE somedb TO ROLE somerole;
3+
grant usage on future schemas in database mydb to role role1;
4+
5+
-- Future Schema Objects Privileges
6+
GRANT INSERT ON FUTURE TABLES IN SCHEMA mydb.myschema TO ROLE somerole;

‎tests/fixtures/sql/grant.sql

-13
Original file line numberDiff line numberDiff line change
@@ -13,23 +13,10 @@ GRANT ADD SEARCH OPTIMIZATION ON SCHEMA someschema TO ROLE somerole;
1313
GRANT CREATE SNOWFLAKE.ML.FORECAST ON SCHEMA someschema TO ROLE somerole;
1414
GRANT CREATE MATERIALIZED VIEW ON SCHEMA mydb.myschema TO ROLE myrole;
1515

16-
-- Schemas Privileges
17-
GRANT USAGE ON ALL SCHEMAS IN DATABASE somedb TO ROLE somerole;
18-
19-
-- Future Schema Privileges
20-
GRANT CREATE ROW ACCESS POLICY ON FUTURE SCHEMAS IN DATABASE somedb TO ROLE somerole;
21-
grant usage on future schemas in database mydb to role role1;
2216

2317
-- Schema Object Privileges
2418
GRANT ALL PRIVILEGES ON FUNCTION mydb.myschema.add5(number) TO ROLE analyst;
2519
GRANT ALL PRIVILEGES ON FUNCTION mydb.myschema.add5(string) TO ROLE analyst;
2620
GRANT USAGE ON PROCEDURE mydb.myschema.myprocedure(number) TO ROLE analyst;
2721

28-
-- Schema Objects Privileges
29-
GRANT SELECT ON ALL TABLES IN SCHEMA mydb.myschema to ROLE analyst;
30-
GRANT USAGE ON ALL FUNCTIONS IN DATABASE somedb TO ROLE analyst;
31-
32-
-- Future Schema Objects Privileges
33-
GRANT INSERT ON FUTURE TABLES IN SCHEMA mydb.myschema TO ROLE somerole;
34-
3522

‎tests/fixtures/sql/replication_group.sql

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
CREATE REPLICATION GROUP myrg
2+
OBJECT_TYPES = DATABASES, SHARES
3+
ALLOWED_DATABASES = db1
4+
ALLOWED_SHARES = s1
5+
ALLOWED_ACCOUNTS = myorg.myaccount2
6+
REPLICATION_SCHEDULE = '10 MINUTE';

0 commit comments

Comments
 (0)