[PDE] Remove broken policies merged into dev

[Shani1116]

Assignee

shani1116

Start Date

2025-12-10

End Date (Target)

2025-12-31

Description

There are broken policies merged into dev that are not working. Need to remove/fix those policies.

Objectives / Expected Outcome

• Fix the failing automated OPA policy check when raising a PR
• Reduce time spent on the policy check

Steps / Action Items

1. Check the automated policy check GitHub Action
2. Figure out the broken policies
3. Remove/fix the broken policies
4. Update the service allocation sheet

Dependencies

N/A

Related Links

N/A

Status

Not started

[Shani1116]

Waiting for PR #253 to be merged into the dev branch -

• to review the new auto-test.py script
• evaluate its integration with the automated OPA policy check
• and identify any broken policies

[Shani1116]

Update - 19/12

PR #253 was merged into dev on 18/12.
Resolved errors in the following folders mentioned below. The auto_test.py script is now running as expected without any policy-related errors.
OPA checks are still reporting errors related to resource naming conventions (c, nc) and takes around 30 mins to finish.

/inputs

• google_firebase_database_instance/google_firebase_database_instance - spelling mistake

/policies

• api_gateway/google_api_gateway_api_config_iam_policy/members - spelling mistake
• api_gateway/google_api_gateway_api_config_iam_policy/role - update the helpers
• firebase_app_hosting/build/container_image - update the helpers
• gemini - update the helpers in all subfolders
• GKEHub/google_gke_hub_feature_membership/pc_enabled_required - spelling mistake
• api_hub/google_apihub_curation/location - add missing policy

To do

• Review existing OPA check errors and fix all issues other than resource naming convention errors
• Update the GitHub workflow to validate only the policies changed in a PR, rather than running checks against all policies

[Shani1116]

Update 05/01

• The OPA policy check takes around 28 minutes to complete the workflow as its going through all the folders and policies. Most of the errors are related to resource naming conventions and missing "project" field in the TF configuration files.
• The auto_test.py script does not provide an exact location on where the missing project filed is
• Created a test workflow to validate only the policies changed in a PR by using the "tj-actions/changed-files@v44" action
• This workflow is working as expected (in a forked repo) but requires more testing.

To do

• Update the auto_test.py to print any TF related errors with the filename and location
• Raise a PR with the updated GitHub Actions workflow
• Validate the workflow to ensure accuracy and reliability

[Shani1116]

Update 12/01

• Added threading to the auto_test_v2 python script with 4 workers to speedup the script.
• The script runtime was reduced to 8 mins from 28 mins after this update but there are still some terraform and OPA related errors in folders that are already merged into dev
• Fixes done with regard to terraform and OPA related errors in folders that are already merged into dev -

Dataform ✅
/inputs - Removed the project variable from tf configuration
/policies -

• vars.rego file package import was incorrect
• All the policies written was missing the .vars import
• Removed some inaccurate conditions in some of the policies that are causing OPA eval errors

Looker✅
/inputs - Corrected the naming of resources and added missing “project” variable
/policies - Change the policy types of custom_domain_when_private and psc_mode_hygiene policies

To do

• Check how to terraform caching works in the GitHub workflow
• Keep working on fixing the errors
• Get support from Credit-aiming students to work on services that are currently producing errors and have been merged into the dev branch.

[Shani1116]

Update 19/01

• Posted an announcement on the Students chat to get support from Credit+ aiming students. No response from the students.
• Continued fixes with regard to terraform and OPA related errors in folders that are already merged into dev -

Recaptchaenterprise ✅
/inputs -

• Add missing project ID variable
• Change the display name in all TF configurations

Gemini ✅
/inputs -

• Change the c, nc logic that was set reversed in TF configuration

/policies -

• Change the package import in disable_data_sharing_ga

Managed_kafka ✅
/inputs

• Change the resource naming to c and nc in TF configurations
• Change values accordingly in some TF configurations

/policies

• Change the message and details entries in all policy.rego files
• Change the policy logic to resolve OPA errors in some policies