feat(policy): stored enhanced actions database migration, CRUD queries, SM updates #2040

jakedoublev · 2025-04-01T22:08:57Z

Migration that moves Subject Mapping column-marshaled action proto enums to new actions table associations
CRUD sqlc queries
CRUD proto message handlers wrapping
integration tests
normalization of SM Create/Update with standard proto enum actions to their CRUDable equivalents until all clients/pipeline tests are updated
adds skipped tests around the action proto enum deprecation that should be unskipped when deprecation is complete

…uture breaking changes in action protos

github-actions · 2025-04-21T21:01:32Z

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	363.655377ms
Throughput	274.99 requests/second

github-actions · 2025-04-21T21:05:28Z

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	380.089788ms
Throughput	263.10 requests/second

Makefile

service/policy/db/migrations/20250411000000_add_actions_table.sql

service/policy/db/query.sql

service/policy/db/actions.go

github-actions · 2025-04-22T18:47:28Z

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	368.538931ms
Throughput	271.34 requests/second

github-actions · 2025-04-22T18:51:31Z

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	359.0691ms
Throughput	278.50 requests/second

github-actions · 2025-04-22T18:53:30Z

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	371.760336ms
Throughput	268.99 requests/second

github-actions · 2025-04-22T18:59:34Z

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	361.314282ms
Throughput	276.77 requests/second

github-actions · 2025-04-22T19:02:19Z

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	377.688454ms
Throughput	264.77 requests/second

github-actions · 2025-04-22T19:07:47Z

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	361.082381ms
Throughput	276.95 requests/second

github-actions · 2025-04-22T19:12:00Z

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Metric	Value
Total Decrypts	100
Successful Decrypts	100
Failed Decrypts	0
Total Time	368.665398ms
Throughput	271.25 requests/second

🤖 I have created a release *beep* *boop* --- ## [0.5.3](service/v0.5.2...service/v0.5.3) (2025-05-22) ### Features * **authz:** authz v2 versioning implementation ([#2173](#2173)) ([557fc21](557fc21)) * **authz:** authz v2, ers v2 protos and gencode for ABAC with actions & registered resource ([#2124](#2124)) ([ea7992a](ea7992a)) * **authz:** export entity id prefix constant from entity instead of authorization service v1 ([#2261](#2261)) ([94079a9](94079a9)) * **authz:** subject mapping plugin support for ABAC with actions ([#2223](#2223)) ([d08b939](d08b939)) * bulk keycloak provisioning ([#2205](#2205)) ([59e4485](59e4485)) * **core:** add otel to opentdf services ([#1858](#1858)) ([53a7aa0](53a7aa0)) * **core:** Adds EC withSalt options ([#2126](#2126)) ([67b6fb8](67b6fb8)) * **core:** enhance db configuration options ([#2285](#2285)) ([ed9ff59](ed9ff59)) * **core:** New Key Index and Manager Plugin SPI ([#2095](#2095)) ([eb446fc](eb446fc)) * **core:** support onConfigUpdate hook when registering services ([#1992](#1992)) ([366d4dc](366d4dc)) * **core:** v2 ERS with proto updates ([#2210](#2210)) ([a161ef8](a161ef8)) * **policy:** actions crud service endpoints and proto validation ([#2037](#2037)) ([e933fa9](e933fa9)) * **policy:** actions service RPCs should actually hit storage layer CRUD ([#2063](#2063)) ([da4faf5](da4faf5)) * **policy:** add enhanced standard/custom actions protos ([#2020](#2020)) ([bbac53f](bbac53f)) * **policy:** Add platform key indexer. ([#2189](#2189)) ([861ef8d](861ef8d)) * **policy:** consume lib/identifier parse function ([#2181](#2181)) ([1cef22b](1cef22b)) * **policy:** DSPX-1018 NDR retrieval by FQN support ([#2131](#2131)) ([0001041](0001041)) * **policy:** DSPX-1057 registered resource action attribute values (DB + Service implementation) ([#2191](#2191)) ([6bf1b2e](6bf1b2e)) * **policy:** DSPX-1057 registered resource action attribute values (protos only) ([#2217](#2217)) ([6375596](6375596)) * **policy:** DSPX-893 NDR define crud protos ([#2056](#2056)) ([55a5c27](55a5c27)) * **policy:** DSPX-898 NDR database schema ([#2055](#2055)) ([2a10a6a](2a10a6a)) * **policy:** DSPX-901 NDR database crud ([#2071](#2071)) ([20e0a5f](20e0a5f)) * **policy:** DSPX-902 NDR service crud implementation (2/2) ([#2066](#2066)) ([030ad33](030ad33)) * **policy:** DSPX-902 NDR service crud protos only (1/2) ([#2092](#2092)) ([24b6cb5](24b6cb5)) * **policy:** Finish resource mapping groups ([#2224](#2224)) ([5ff754e](5ff754e)) * **policy:** GetMatchedSubjectMappings should provide value FQN ([#2151](#2151)) ([ad80044](ad80044)) * **policy:** key management crud ([#2110](#2110)) ([4c3d53d](4c3d53d)) * **policy:** Key management proto ([#2115](#2115)) ([561f853](561f853)) * **policy:** Modify get request to search for keys by kasid with keyid. ([#2147](#2147)) ([780d2e4](780d2e4)) * **policy:** Restrict KAS deletion when tied to Key ([#2144](#2144)) ([4c4ab13](4c4ab13)) * **policy:** Return KAS Key structure ([#2172](#2172)) ([7f97b99](7f97b99)) * **policy:** rotate keys rpc ([#2180](#2180)) ([0d00743](0d00743)) * **policy:** stored enhanced actions database migration, CRUD queries, SM updates ([#2040](#2040)) ([e6b7c79](e6b7c79)) * **sdk:** Add a KAS allowlist ([#2085](#2085)) ([d7cfdf3](d7cfdf3)) * **sdk:** add nanotdf plaintext policy ([#2182](#2182)) ([e5c56db](e5c56db)) * **sdk:** Use ConnectRPC in the go client ([#2200](#2200)) ([fc34ee6](fc34ee6)) ### Bug Fixes * **core:** access pdp cleanup before actions in ABAC decisioning ([#2123](#2123)) ([9b38a3c](9b38a3c)) * **core:** Autobump service ([#2080](#2080)) ([006c724](006c724)) * **core:** Autobump service ([#2104](#2104)) ([1f72cc7](1f72cc7)) * **core:** Autobump service ([#2108](#2108)) ([be5b7d7](be5b7d7)) * **core:** bump to go 1.24 and bump service proto module dependencies ([#2064](#2064)) ([94891a0](94891a0)) * **core:** Fix DPoP with grpc-gateway ([#2044](#2044)) ([4483ef2](4483ef2)) * **core:** fix service go.mod ([#2141](#2141)) ([3b98f6d](3b98f6d)) * **core:** Improves errors when under heavy load ([#2132](#2132)) ([4490a14](4490a14)) * **core:** Let legacy KAOs use new trust plugins ([#2218](#2218)) ([5aa6916](5aa6916)) * **core:** migrate from mitchellh/mapstructure to go-viper/mapstructure ([#2087](#2087)) ([0a3a82e](0a3a82e)) * **core:** update viper to 1.20.1 ([#2088](#2088)) ([09099e9](09099e9)) * **core:** Updates vulnerable dep go/x/net ([#2072](#2072)) ([11c02cd](11c02cd)) * **deps:** bump github.com/creasty/defaults from 1.7.0 to 1.8.0 in /service ([#2242](#2242)) ([86a9b46](86a9b46)) * **deps:** bump github.com/jackc/pgx/v5 from 5.5.5 to 5.7.5 in /service ([#2249](#2249)) ([d8f3b67](d8f3b67)) * **deps:** bump the internal group across 1 directory with 2 updates ([#2296](#2296)) ([7f92c70](7f92c70)) * **deps:** bump toolchain in /lib/fixtures and /examples to resolve CVE GO-2025-3563 ([#2061](#2061)) ([9c16843](9c16843)) * handle empty private and public key ctx structs ([#2272](#2272)) ([f3fc647](f3fc647)) * **policy:** remove predefined rules in actions protos ([#2069](#2069)) ([060f059](060f059)) * **policy:** return kas uri on keys for definition, namespace and values ([#2186](#2186)) ([6c55fb8](6c55fb8)) * update key_mode to provide more context ([#2226](#2226)) ([44d0805](44d0805)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: opentdf-automation[bot] <149537512+opentdf-automation[bot]@users.noreply.github.com>

jakedoublev added 30 commits March 31, 2025 14:46

feat(policy): add enhanced standard/custom actions protos

48d8987

reserve field names as well

05334f2

rm deprecated actions

3751250

Merge branch 'main' into feat/actions-protos

b91a1d9

allow migration before hard deprecation

824f876

access pdp todo comment

d83783e

list should specify stand/cust in response, and proactively prevent f…

ad9512c

…uture breaking changes in action protos

Merge branch 'main' into feat/actions-protos

bf28fcf

Merge branch 'main' into feat/actions-svc

809b956

actions svc

f39adac

Merge branch 'main' into feat/actions-protos

857f493

Merge branch 'feat/actions-protos' into feat/actions-svc

5e3374f

proto validation

1fe24f5

actions mermaid ERD for the migration

03fcd4c

migrate up and down

b800517

wip

f5592b1

update overall erd

021d2c7

update migrations with relation table

4ad71a2

global policy erd styling

fb3fd22

fix migration markdown notes

3c48cba

timestamp with time zone

85938a4

list actions query

38068ab

add subject mappings to get action response

9fa0396

Merge branch 'feat/actions-protos' into feat/actions-svc

74d4fd7

Merge branch 'feat/actions-svc' into feat/actions-db

76c664b

CRUD of actions (not including subject mappings yet

1ddc082

wip

445ff05

mini ADR for storing standard actions

dcd40c4

rework protos according to mini-ADR and how we'll store standard actions

a67018f

Merge branch 'feat/actions-protos' into feat/actions-svc

2307646

typo

0a4dbcd

jakedoublev marked this pull request as ready for review April 21, 2025 21:14

jakedoublev requested review from a team as code owners April 21, 2025 21:14

jakedoublev enabled auto-merge April 21, 2025 22:05

ryanulit requested changes Apr 22, 2025

View reviewed changes

jakedoublev added 2 commits April 22, 2025 11:42

Merge branch 'main' into feat/actions-db

7a14e5a

updated ERD

48a1b66

remove duplicate index

5cbfca9

jakedoublev added 2 commits April 22, 2025 11:55

remove extraneous nil check

081b69b

fix extraneous nil checks in action update

a8cf74e

jakedoublev added 2 commits April 22, 2025 12:03

improve SQL when querying actions in lists of subject mappings

b1ec977

remove extraneous SQL RETURNING statements

1648e69

ryanulit approved these changes Apr 22, 2025

View reviewed changes

jakedoublev added this pull request to the merge queue Apr 22, 2025

Merged via the queue into main with commit e6b7c79 Apr 22, 2025
24 of 25 checks passed

jakedoublev deleted the feat/actions-db branch April 22, 2025 19:35

opentdf-automation bot mentioned this pull request Apr 22, 2025

chore(main): release service 0.5.3 #2046

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(policy): stored enhanced actions database migration, CRUD queries, SM updates #2040

feat(policy): stored enhanced actions database migration, CRUD queries, SM updates #2040

Uh oh!

jakedoublev commented Apr 1, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Apr 21, 2025

Uh oh!

github-actions bot commented Apr 21, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 22, 2025

Uh oh!

github-actions bot commented Apr 22, 2025

Uh oh!

github-actions bot commented Apr 22, 2025

Uh oh!

github-actions bot commented Apr 22, 2025

Uh oh!

github-actions bot commented Apr 22, 2025

Uh oh!

github-actions bot commented Apr 22, 2025

Uh oh!

github-actions bot commented Apr 22, 2025

Uh oh!

Uh oh!

Uh oh!

feat(policy): stored enhanced actions database migration, CRUD queries, SM updates #2040

feat(policy): stored enhanced actions database migration, CRUD queries, SM updates #2040

Uh oh!

Conversation

jakedoublev commented Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 21, 2025

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Uh oh!

github-actions bot commented Apr 21, 2025

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 22, 2025

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Uh oh!

github-actions bot commented Apr 22, 2025

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Uh oh!

github-actions bot commented Apr 22, 2025

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Uh oh!

github-actions bot commented Apr 22, 2025

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Uh oh!

github-actions bot commented Apr 22, 2025

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Uh oh!

github-actions bot commented Apr 22, 2025

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Uh oh!

github-actions bot commented Apr 22, 2025

Standard Benchmark Skipped or Failed

Bulk Benchmark Results

Uh oh!

Uh oh!

Uh oh!

jakedoublev commented Apr 1, 2025 •

edited

Loading