Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev 338/integrated cd #3020

Open
wants to merge 7 commits into
base: dev
Choose a base branch
from
Open

Dev 338/integrated cd #3020

Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8dda153
feat: added steps in deploy action to check back-end status then chec…
codechirag123 Feb 2, 2025
a6e4ef1
fix: added echo for frontend changes
codechirag123 Feb 2, 2025
f98bffc
Merge branch 'dev' into dev-338/integrated-cd
codechirag123 Feb 10, 2025
17ad14a
test: changing app-name
codechirag123 Feb 10, 2025
a69269d
fix: adding output step to verify status
codechirag123 Feb 11, 2025
4f172f7
fix: changed namespace in the jobs
codechirag123 Feb 11, 2025
7673ffd
feat: added Readme with information about CI/CD
codechirag123 Feb 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
333 changes: 333 additions & 0 deletions .github/workflows/deploy-wf-service.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -272,3 +272,336 @@ jobs:
env:
SLACK_BOT_TOKEN: ${{ secrets.ARGO_SLACK_BOT_TOKEN }}

# TODO Check Prisma Migrate Status
check-prisma-migrate-status:
runs-on: ubuntu-latest
needs: [update-helm-chart,build-and-push-ee-image]
if: ${{ needs.update-helm-chart.result == 'success' }}
environment: ${{ inputs.environment }}
steps:
- name: Pre-Run Wait
run: |
sleep 15
- name: Check Prisma Migrate Status
id: prisma-status
run: |
ARGOCD_API_URL="${{ secrets.ARGOCD_SERVER }}/api/v1/applications/wf-service/resource?name=wf-service-prisma-migrate&appNamespace=argocd&namespace=default&resourceName=wf-service-prisma-migrate&version=v1&kind=Job&group=batch"

TARGET_IMAGE_TAG="${{ env.SHORT_HASH }}-${{ needs.build-and-push-ee-image.outputs.SUBMODULE_SHORT_HASH }}-${{ inputs.environment }}"
TOKEN=$(curl -k --insecure -s -X POST "${{ secrets.ARGOCD_SERVER }}/api/v1/session" \
-d '{"username": "'"${{ secrets.ARGOCD_USERNAME }}"'", "password": "'"${{ secrets.ARGOCD_PASSWORD }}"'"}' \
-H "Content-Type: application/json" | jq -r '.token')
echo "ARGOCD_TOKEN=$TOKEN" >> $GITHUB_ENV
while true; do
# Fetch job status from ArgoCD API
RESPONSE=$(curl -s -k -H "Authorization: Bearer $ARGOCD_TOKEN" "$ARGOCD_API_URL")
# echo $RESPONSE

# Extract and parse the manifest JSON
MANIFEST=$(echo "$RESPONSE" | jq -r '.manifest' | jq '.')
# echo $MANIFEST

# Extract Start Time, Status, and Type
# START_TIME=$(echo "$MANIFEST" | jq -r '.status.startTime')
# echo $START_TIME
STATUS=$(echo "$MANIFEST" | jq -r '.status.conditions[] | select(.type=="Complete") | .status')
echo $STATUS
TYPE=$(echo "$MANIFEST" | jq -r '.status.conditions[] | select(.type=="Complete") | .type')
echo $TYPE

# Extract Image Tag
IMAGE=$(echo "$MANIFEST" | jq -r '.spec.template.spec.containers[0].image')
# echo $IMAGE
IMAGE_TAG=$(echo "$IMAGE" | cut -d ':' -f2) # Extract the tag after the colon
echo $IMAGE_TAG

# Print extracted information
# echo "Start Time: $START_TIME"
echo "Status: $STATUS"
echo "Type: $TYPE"
# echo "Image: $IMAGE"
echo "Image Tag: $IMAGE_TAG"

# Nested condition to check Image Tag and Job Completion
if [[ "$IMAGE_TAG" == "$TARGET_IMAGE_TAG" ]]; then
if [[ "$TYPE" == "Complete" && "$STATUS" == "True" ]]; then
echo "✅ Prisma Job has completed successfully with the correct image tag: $IMAGE_TAG"
break
else
echo "⏳ Prisma Job is still running... Waiting for completion."
fi
else
echo "🚨 Image tag mismatch! Expected: $TARGET_IMAGE_TAG, Found: $IMAGE_TAG"
fi

# Wait before the next check
sleep 10
done

# TODO Check Application Deployment Status
check-wf-service-status:
runs-on: ubuntu-latest
needs: [check-prisma-migrate-status,build-and-push-ee-image]
if: ${{ needs.check-prisma-migrate-status.result == 'success' }}
environment: ${{ inputs.environment }}
steps:
- name: Pre-Run Wait
run: |
sleep 15
- name: Check Prisma Migrate Status
id: prisma-status
run: |
ARGOCD_API_URL="${{ secrets.ARGOCD_SERVER }}/api/v1/applications/wf-service/resource?name=wf-service-db-data-migrations&appNamespace=argocd&namespace=default&resourceName=wf-service&version=v1&kind=Deployment&group=apps"

TARGET_IMAGE_TAG="${{ env.SHORT_HASH }}-${{ needs.build-and-push-ee-image.outputs.SUBMODULE_SHORT_HASH }}-${{ inputs.environment }}"

while true; do
# Fetch job status from ArgoCD API
RESPONSE=$(curl -s -k -H "Authorization: Bearer ${{env.ARGOCD_TOKEN}}" "$ARGOCD_API_URL")
# echo $RESPONSE

Comment on lines +358 to +362
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Implement Timeout in Deployment Polling Loop
The polling loop in the deployment status check lacks a timeout. Consider adding a timeout or iteration limit to prevent potential infinite looping if the WF service remains in a transitional state.

# Extract and parse the manifest JSON
MANIFEST=$(echo "$RESPONSE" | jq -r '.manifest' | jq '.')
# echo $MANIFEST

# Extract Start Time, Status, and Type
# START_TIME=$(echo "$MANIFEST" | jq -r '.status.startTime')
# echo $START_TIME
STATUS=$(echo "$MANIFEST" | jq -r '.status.conditions[] | select(.type=="Progressing") | .status')
echo $STATUS
REASON=$(echo "$MANIFEST" | jq -r '.status.conditions[] | select(.type=="Progressing") | .reason')
echo $REASON

# Extract Image Tag
IMAGE=$(echo "$MANIFEST" | jq -r '.spec.template.spec.containers[0].image')
echo $IMAGE
IMAGE_TAG=$(echo "$IMAGE" | cut -d ':' -f2) # Extract the tag after the colon
echo $IMAGE_TAG

# Print extracted information
echo "Start Time: $START_TIME"
echo "Status: $STATUS"
echo "Type: $REASON"
echo "Image: $IMAGE"
echo "Image Tag: $IMAGE_TAG"

# Nested condition to check Image Tag and Job Completion
if [[ "$IMAGE_TAG" == "$TARGET_IMAGE_TAG" ]]; then
if [[ "$REASON" == "NewReplicaSetAvailable" && "$STATUS" == "True" ]]; then
echo "✅ Deployment has been successful with the correct image tag: $IMAGE_TAG"
break
else
echo "⏳ Replicaset is still getting updated... Waiting for completion."
fi
else
echo "🚨 Image tag mismatch! Expected: $TARGET_IMAGE_TAG, Found: $IMAGE_TAG"
fi

# Wait before the next check
sleep 10
done


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

⚠️ Potential issue

Review: Correct Naming and Undefined Variable in WF Service Deployment Status Check

In the check-wf-service-status job:

  • The step is labelled "Check Prisma Migrate Status" despite the job being intended to verify the workflows service deployment. Renaming it (for example, to "Check WF Service Deployment Status") will improve clarity.
  • The script echoes a variable named START_TIME (line 382) that is never defined (the related lines are commented out). Either remove this echo or define and use START_TIME appropriately.
  • As with the previous job, consider adding a timeout to the polling loop.
🧰 Tools
🪛 YAMLlint (1.35.1)

[warning] 344-344: too few spaces after comma

(commas)

# TODO Check DB Migrate Status
check-db-migrate-status:
runs-on: ubuntu-latest
needs: [check-wf-service-status,build-and-push-ee-image]
if: ${{ needs.check-wf-service-status.result == 'success' }}
environment: ${{ inputs.environment }}
steps:
- name: Pre-Run Wait
run: |
sleep 15
- name: Check Prisma Migrate Status
id: prisma-status
run: |
ARGOCD_API_URL="${{ secrets.ARGOCD_SERVER }}/api/v1/applications/wf-service/resource?name=wf-service-db-data-migrations&appNamespace=argocd&namespace=default&resourceName=wf-service-db-data-migrations&version=v1&kind=Job&group=batch"

TARGET_IMAGE_TAG="${{ env.SHORT_HASH }}-${{ needs.build-and-push-ee-image.outputs.SUBMODULE_SHORT_HASH }}-${{ inputs.environment }}"
while true; do
# Fetch job status from ArgoCD API
RESPONSE=$(curl -s -k -H "Authorization: Bearer ${{env.ARGOCD_TOKEN}}" "$ARGOCD_API_URL")
# echo $RESPONSE

# Extract and parse the manifest JSON
MANIFEST=$(echo "$RESPONSE" | jq -r '.manifest' | jq '.')
Comment on lines +420 to +427
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add Timeout to DB Migrate Loop
The DB migration status polling loop runs indefinitely until the desired status is detected. Consider adding a timeout or maximum number of iterations to avoid infinite looping under error conditions.

# echo $MANIFEST

# Extract Start Time, Status, and Type
START_TIME=$(echo "$MANIFEST" | jq -r '.status.startTime')
echo $START_TIME
STATUS=$(echo "$MANIFEST" | jq -r '.status.conditions[] | select(.type=="Complete") | .status')
echo $STATUS
TYPE=$(echo "$MANIFEST" | jq -r '.status.conditions[] | select(.type=="Complete") | .type')
echo $TYPE

# Extract Image Tag
IMAGE=$(echo "$MANIFEST" | jq -r '.spec.template.spec.containers[0].image')
echo $IMAGE
IMAGE_TAG=$(echo "$IMAGE" | cut -d ':' -f2) # Extract the tag after the colon
echo $IMAGE_TAG

# Print extracted information
echo "Start Time: $START_TIME"
echo "Status: $STATUS"
echo "Type: $TYPE"
echo "Image: $IMAGE"
echo "Image Tag: $IMAGE_TAG"

# Nested condition to check Image Tag and Job Completion
if [[ "$IMAGE_TAG" == "$TARGET_IMAGE_TAG" ]]; then
if [[ "$TYPE" == "Complete" && "$STATUS" == "True" ]]; then
echo "✅ DB Migrate Job has completed successfully with the correct image tag: $IMAGE_TAG"
break
else
echo "⏳ DB Migrate Job is still running... Waiting for completion."
fi
else
echo "🚨 Image tag mismatch! Expected: $TARGET_IMAGE_TAG, Found: $IMAGE_TAG"
fi

# Wait before the next check
sleep 10
done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

⚠️ Potential issue

Review: Update Naming for DB Migrate Status Check

In the check-db-migrate-status job:

  • The step is once again named "Check Prisma Migrate Status" though it is checking the database migration job. Rename the step to "Check DB Migrate Status" to accurately reflect its purpose.
  • A timeout for the polling loop is recommended to avoid potential infinite looping.
🧰 Tools
🪛 YAMLlint (1.35.1)

[warning] 408-408: too few spaces after comma

(commas)

# TODO Check DB Sync Status
check-db-sync-status:
runs-on: ubuntu-latest
needs: [check-db-migrate-status,build-and-push-ee-image]
if: ${{ needs.check-db-migrate-status.result == 'success' }}
environment: ${{ inputs.environment }}
steps:
- name: Pre-Run Wait
run: |
sleep 15
- name: Check Prisma Migrate Status
id: prisma-status
run: |
ARGOCD_API_URL="${{ secrets.ARGOCD_SERVER }}/api/v1/applications/wf-service/resource?name=wf-service-db-data-sync&appNamespace=argocd&namespace=default&resourceName=wf-service-db-data-sync&version=v1&kind=Job&group=batch"

TARGET_IMAGE_TAG="${{ env.SHORT_HASH }}-${{ needs.build-and-push-ee-image.outputs.SUBMODULE_SHORT_HASH }}-${{ inputs.environment }}"
while true; do
# Fetch job status from ArgoCD API
RESPONSE=$(curl -s -k -H "Authorization: Bearer ${{env.ARGOCD_TOKEN}}" "$ARGOCD_API_URL")
# echo $RESPONSE

Comment on lines +482 to +487
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Implement Timeout in DB Sync Loop
Similar to the other polling loops, the DB sync check loop lacks a timeout. Adding this mechanism would help avoid a potential infinite loop if the sync job stalls or fails.

# Extract and parse the manifest JSON
MANIFEST=$(echo "$RESPONSE" | jq -r '.manifest' | jq '.')
# echo $MANIFEST

# Extract Start Time, Status, and Type
START_TIME=$(echo "$MANIFEST" | jq -r '.status.startTime')
echo $START_TIME
STATUS=$(echo "$MANIFEST" | jq -r '.status.conditions[] | select(.type=="Complete") | .status')
echo $STATUS
TYPE=$(echo "$MANIFEST" | jq -r '.status.conditions[] | select(.type=="Complete") | .type')
echo $TYPE

# Extract Image Tag
IMAGE=$(echo "$MANIFEST" | jq -r '.spec.template.spec.containers[0].image')
echo $IMAGE
IMAGE_TAG=$(echo "$IMAGE" | cut -d ':' -f2) # Extract the tag after the colon
echo $IMAGE_TAG

# Print extracted information
echo "Start Time: $START_TIME"
echo "Status: $STATUS"
echo "Type: $TYPE"
echo "Image: $IMAGE"
echo "Image Tag: $IMAGE_TAG"

# Nested condition to check Image Tag and Job Completion
if [[ "$IMAGE_TAG" == "$TARGET_IMAGE_TAG" ]]; then
if [[ "$TYPE" == "Complete" && "$STATUS" == "True" ]]; then
echo "✅ DB Sync Job has completed successfully with the correct image tag: $IMAGE_TAG"
break
else
echo "⏳ DB Sync Job is still running... Waiting for completion."
fi
else
echo "🚨 Image tag mismatch! Expected: $TARGET_IMAGE_TAG, Found: $IMAGE_TAG"
fi

# Wait before the next check
sleep 10
done

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Review: Update Naming for DB Sync Status Check

The check-db-sync-status job:

  • Uses the step name "Check Prisma Migrate Status" even though the intention is to verify the database synchronization job. Please rename this step to "Check DB Sync Status" for clarity.
  • As with the other polling loops, implementing a timeout mechanism is advisable.
🧰 Tools
🪛 YAMLlint (1.35.1)

[warning] 470-470: too few spaces after comma

(commas)

[error] 528-528: trailing spaces

(trailing-spaces)

# Check Front-end changes
check-apps-changes:
needs: [check-db-sync-status,build-and-push-ee-image]
if: ${{ needs.check-db-sync-status.result == 'success' }}
runs-on: ubuntu-latest
outputs:
BACKOFFICE_UPDATED: ${{ steps.check-backoffice.outputs.BACKOFFICE_UPDATED }}
KYB_UPDATED: ${{ steps.check-kyb.outputs.KYB_UPDATED }}
DASHBOARD_UPDATED: ${{ steps.check-dashboard.outputs.DASHBOARD_UPDATED }}
steps:
- name: Checkout Repository
uses: actions/checkout@v4
with:
fetch-depth: 0 # Ensures we get the history for diff comparison

- name: Get List of Changed Files
id: changed-files
run: |
echo "Changed files:"
git diff --name-only HEAD^ HEAD

- name: Check for Changes in apps/v1
id: check-backoffice
run: |
if git diff --name-only HEAD^ HEAD | grep '^apps/backoffice-v2/' > /dev/null; then
echo "BACKOFFICE_UPDATED=true" >> "$GITHUB_ENV"
echo "BACKOFFICE_UPDATED=true" >> "$GITHUB_OUTPUT"
else
echo "BACKOFFICE_UPDATED=false" >> "$GITHUB_ENV"
echo "BACKOFFICE_UPDATED=false" >> "$GITHUB_OUTPUT"
fi

- name: Check for Changes in apps/v1
id: check-kyb
run: |
if git diff --name-only HEAD^ HEAD | grep '^apps/kyb-app/' > /dev/null; then
echo "KYB_UPDATED=true" >> "$GITHUB_ENV"
else
echo "KYB_UPDATED=false" >> "$GITHUB_ENV"
fi

- name: Check for Changes in apps/v1
id: check-dashboard
run: |
if git diff --name-only HEAD^ HEAD | grep '^apps/workflows-dashboard/' > /dev/null; then
echo "DASHBOARD_UPDATED=true" >> "$GITHUB_ENV"
else
echo "DASHBOARD_UPDATED=false" >> "$GITHUB_ENV"
fi

- name: Print Environment Variables
run: |
echo "BACKOFFICE_UPDATED=$BACKOFFICE_UPDATED"
echo "KYB_UPDATED=$KYB_UPDATED"
echo "DASHBOARD_UPDATED=$DASHBOARD_UPDATED"

# TODO Deploy Backoffice trigger
deploy-backoffice:
needs: [check-db-sync-status,check-apps-changes]
if: ${{ needs.check-db-sync-status.result == 'success' && needs.check-apps-changes.outputs.BACKOFFICE_UPDATED == 'true' }}
uses: ./.github/workflows/deploy-backoffice.yml
with:
environment: ${{ inputs.environment }}

# TODO Deploy KYB Trigger
deploy-kyb:
needs: [check-db-sync-status,check-apps-changes]
if: ${{ needs.check-db-sync-status.result == 'success' && needs.check-apps-changes.outputs.KYB_UPDATED == 'true' }}
uses: ./.github/workflows/deploy-kyb.yml
with:
environment: ${{ inputs.environment }}

# TODO Deploy Dashboard Trigger
deploy-dashboard:
needs: [check-db-sync-status,check-apps-changes]
if: ${{ needs.check-db-sync-status.result == 'success' && needs.check-apps-changes.outputs.DASHBOARD_UPDATED == 'true' }}
uses: ./.github/workflows/deploy-dashboard.yml
with:
environment: ${{ inputs.environment }}