Skip to content

Fixed multiple full scan AWS buckets scanning deployment name conflict #128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
samthesecret wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Fixed multiple full scan AWS buckets scanning deployment name conflict #128

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f21a18c
Fixed multiple full scan AWS buckets scanning deployment name conflict
samthesecret Apr 15, 2023
33f69de
Removed template.json as removed from upstream
samthesecret Apr 18, 2023
375bed7
Merge branch 'master' into master
samthesecret Apr 18, 2023
4a11ee3
Reduced the StateMachine name to avoid max lengh error
samthesecret Apr 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 31 additions & 5 deletions scan-triggers/aws-python-bucket-full-and-scheduled-scan/template.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -510,11 +510,24 @@
{
"Ref": "AWS::AccountId"
},
":stateMachine:ScannerLoopStateMachine\",\n \"Input\": {\n \"stateBucket.$\": \"$.stateBucket\",\n \"stateKey.$\": \"$.stateKey\",\n \"limit.$\": \"$.limit\",\n \"bucket.$\": \"$.bucket\",\n \"AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID.$\": \"$$.Execution.Id\"\n }\n },\n \"End\": true\n },\n \"Pass\": {\n \"Type\": \"Pass\",\n \"End\": true,\n \"Result\": {}\n }\n }\n }\n ],\n \"End\": true\n }\n }\n }\n "
":stateMachine:ScannerLoopStateMachine-",
{
"Ref": "BucketName"
},"\",\n \"Input\": {\n \"stateBucket.$\": \"$.stateBucket\",\n \"stateKey.$\": \"$.stateKey\",\n \"limit.$\": \"$.limit\",\n \"bucket.$\": \"$.bucket\",\n \"AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID.$\": \"$$.Execution.Id\"\n }\n },\n \"End\": true\n },\n \"Pass\": {\n \"Type\": \"Pass\",\n \"End\": true,\n \"Result\": {}\n }\n }\n }\n ],\n \"End\": true\n }\n }\n }\n "
]
]
},
"StateMachineName": "ScannerLoopStateMachine"
"StateMachineName": {
"Fn::Join": [
"",
[
"ScannerLoopStateMachine-",
{
"Ref": "BucketName"
}
]
]
}
}
},
"FullScanStarterExecutionRole": {
Expand Down Expand Up @@ -616,7 +629,17 @@
]
]
},
"StateMachineName": "fullScanStarterStateMachine"
"StateMachineName": {
"Fn::Join": [
"",
[
"fullScanStarterStateMachine-",
{
"Ref": "BucketName"
}
]
]
}
},
"DependsOn": [
"FullScanStarterExecutionRoleDefaultPolicy",
Expand Down Expand Up @@ -714,10 +737,13 @@
{
"Ref": "AWS::AccountId"
},
":stateMachine:fullScanStarterStateMachine"
":stateMachine:fullScanStarterStateMachine-",
{
"Ref": "BucketName"
}
]
]
}
}
}
}
}
30 changes: 20 additions & 10 deletions scan-triggers/aws-python-bucket-full-and-scheduled-scan/template.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -309,9 +309,9 @@ Resources:
const scanLimitPerIteration = event.limit;
const bucket = event.bucket;
const allKeys = await fetchKeys(stateBucket, stateKey);

const filtered = filterKeys(allKeys, scanLimitPerIteration);

// Rewrite file in bucket with remaining keys
if (filtered.remainingKeys){
await writeToBucket(filtered.remainingKeys, stateKey, stateBucket);
Expand All @@ -325,7 +325,7 @@ Resources:
stateBucket: event.stateBucket,
stateKey: event.stateKey
};

return response;
};
Role:
Expand Down Expand Up @@ -568,7 +568,7 @@ Resources:
Fn::Join:
- ""
- - |-2

{
"Comment": "A machine that loops trough all files a bucket to scan them with File Storage Security.",
"StartAt": "Filter first 1000 keys to scan",
Expand Down Expand Up @@ -674,7 +674,9 @@ Resources:
- Ref: AWS::Region
- ":"
- Ref: AWS::AccountId
- ":stateMachine:ScannerLoopStateMachine\",
- ":stateMachine:ScannerLoopStateMachine-"
- Ref: BucketName
- "\",

\ \"Input\": {

Expand Down Expand Up @@ -721,7 +723,11 @@ Resources:
\ }

\ "
StateMachineName: ScannerLoopStateMachine
StateMachineName:
Fn::Join:
- ""
- - ScannerLoopStateMachine-
- Ref: BucketName
FullScanStarterExecutionRole:
Type: AWS::IAM::Role
Properties:
Expand Down Expand Up @@ -769,7 +775,7 @@ Resources:
Fn::Join:
- ""
- - |-2

{
"Comment": "Kicks of a Full Scan using File Storage Security.",
"StartAt": "List all keys in bucket",
Expand Down Expand Up @@ -841,7 +847,11 @@ Resources:
\ }

\ "
StateMachineName: fullScanStarterStateMachine
StateMachineName:
Fn::Join:
- ""
- - fullScanStarterStateMachine-
- Ref: BucketName
DependsOn:
- FullScanStarterExecutionRoleDefaultPolicy
- FullScanStarterExecutionRole
Expand Down Expand Up @@ -898,5 +908,5 @@ Outputs:
- Ref: AWS::Region
- ":"
- Ref: AWS::AccountId
- :stateMachine:fullScanStarterStateMachine

- :stateMachine:fullScanStarterStateMachine-
- Ref: BucketName