-
Notifications
You must be signed in to change notification settings - Fork 63
feat(deploy-on-aws): add architecture diagram generation skill #84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
MichaelWalker-git
wants to merge
16
commits into
awslabs:main
from
MichaelWalker-git:feat/diagram-skill
Closed
Changes from 6 commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
f1007bb
feat(deploy-on-aws): add architecture diagram generation skill
MichaelWalker-git 4c4f700
fix(deploy-on-aws): address diagram skill review comments
MichaelWalker-git 3ea8a04
style(deploy-on-aws): fix dprint markdown table formatting
MichaelWalker-git ec41535
chore: trigger CI re-run
MichaelWalker-git c1c128c
chore: trigger contributor check
MichaelWalker-git 3d1f28c
fix(deploy-on-aws): fix remaining review comments on diagram skill
MichaelWalker-git 647be75
fix(deploy-on-aws): use generated-diagrams/ path in constructor example
MichaelWalker-git 95f00e8
fix(deploy-on-aws): address Copilot review comments on diagram skill
MichaelWalker-git 90cc2cf
fix(deploy-on-aws): address 3 new Copilot review comments
MichaelWalker-git 7172078
Merge branch 'main' into feat/diagram-skill
krokoko c1061af
Merge branch 'main' into feat/diagram-skill
krokoko ddf99d3
Merge branch 'main' into feat/diagram-skill
krokoko a5406c2
fix(deploy-on-aws): address krokoko review comments on diagram skill
MichaelWalker-git 07bc1bc
fix(deploy-on-aws): fix incorrect class names and add direction="TB" …
MichaelWalker-git addd48f
fix(deploy-on-aws): address remaining krokoko security and quality co…
MichaelWalker-git 04601f1
fix(lint): format markdown tables with dprint
MichaelWalker-git File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| --- | ||
| name: diagram | ||
| description: "Generate architecture diagrams as code using the Python diagrams DSL. Triggers on phrases like: architecture diagram, system design diagram, draw architecture, generate diagram, infrastructure diagram, AWS diagram, Kubernetes diagram, network diagram, visualize architecture." | ||
| --- | ||
|
|
||
| # Architecture Diagram Generation | ||
|
|
||
| Generate architecture diagrams using the Python [diagrams](https://diagrams.mingrammer.com/) package. Write a Python script, run it, and produce a PNG. | ||
MichaelWalker-git marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| Supports AWS (29 service categories), Kubernetes, on-premises, GCP, SaaS, and custom icons via the `diagrams` DSL. | ||
|
|
||
| ## When to Load Reference Files | ||
|
|
||
| Load the appropriate reference file based on what the user is building: | ||
|
|
||
| - **Any diagram** -> ALWAYS load [references/dsl-syntax.md](references/dsl-syntax.md) first for DSL syntax, constructor options, connections, clusters, and edge styles | ||
| - **AWS architecture**, **cloud infrastructure**, or **AWS services** -> load [references/aws-services.md](references/aws-services.md) for AWS import paths, common icons, and 5 AWS examples | ||
| - **Kubernetes**, **on-premises**, **SaaS**, **flowcharts**, **process flows**, **custom icons**, or **non-AWS diagrams** -> load [references/non-aws-providers.md](references/non-aws-providers.md) for K8s, on-prem, flowchart, and custom icon examples | ||
|
|
||
| ## Workflow | ||
|
|
||
| 1. Load [references/dsl-syntax.md](references/dsl-syntax.md) | ||
| 2. Load the relevant provider reference (AWS or non-AWS) | ||
| 3. Write a Python script using the `diagrams` DSL | ||
| 4. Run: `mkdir -p generated-diagrams && python3 diagram.py` | ||
| 5. Verify the output file was created in `generated-diagrams/` | ||
|
|
||
| ## Critical Rules | ||
|
|
||
| - ALWAYS set `show=False` in `Diagram()` constructor | ||
| - ALWAYS set `filename="generated-diagrams/<name>"` (no `.png` extension) | ||
| - ALWAYS use explicit imports: `from diagrams.aws.compute import EC2` | ||
| - ALWAYS run `mkdir -p generated-diagrams` before executing | ||
MichaelWalker-git marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| - Add `generated-diagrams/` to `.gitignore` to avoid committing output artifacts | ||
| - Use `Cluster()` to group related resources (VPCs, subnets, namespaces) | ||
| - Use `Edge(label=, color=, style=)` for labeled or styled connections | ||
MichaelWalker-git marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## Defaults | ||
|
|
||
| Default output format: PNG | ||
| Default layout direction: TB (top-to-bottom) | ||
|
|
||
| Override syntax: | ||
|
|
||
| - "left to right" -> Use `direction="LR"` | ||
| - "SVG format" -> Use `outformat="svg"` | ||
|
|
||
| When not specified, ALWAYS use `direction="TB"` and `outformat="png"`. | ||
MichaelWalker-git marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## Prerequisites | ||
|
|
||
| Requires two dependencies installed locally: | ||
|
|
||
| 1. **GraphViz** (system package providing `dot`): | ||
| - macOS: `brew install graphviz` | ||
| - Ubuntu/Debian: `sudo apt-get install graphviz` | ||
| - Amazon Linux/RHEL: `sudo yum install graphviz` | ||
| 2. **Python diagrams package**: `pip install diagrams` | ||
|
|
||
| **Verify:** `dot -V && python3 -c "import diagrams; print('OK')"` | ||
MichaelWalker-git marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### Missing Dependencies | ||
|
|
||
| If `dot` command not found: | ||
|
|
||
| - Inform user: "GraphViz is not installed. Required for diagram rendering." | ||
| - Show install command for detected OS | ||
| - DO NOT attempt to generate diagrams without GraphViz | ||
|
|
||
| If `import diagrams` fails: | ||
|
|
||
| - Inform user: "Python diagrams package not installed." | ||
| - Show: `pip install diagrams` | ||
| - DO NOT proceed without the package | ||
|
|
||
| ## References | ||
|
|
||
| - [DSL syntax and patterns](references/dsl-syntax.md) | ||
| - [AWS services and examples](references/aws-services.md) | ||
| - [Non-AWS providers and examples](references/non-aws-providers.md) | ||
| - [diagrams documentation](https://diagrams.mingrammer.com/) | ||
112 changes: 112 additions & 0 deletions
112
plugins/deploy-on-aws/skills/diagram/references/aws-services.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| # AWS Services and Examples | ||
|
|
||
| ## AWS Service Categories | ||
|
|
||
| | Category | Import Path | Common Icons | | ||
| | ------------- | -------------------------- | ----------------------------------------------------- | | ||
| | `analytics` | `diagrams.aws.analytics` | Athena, EMR, Glue, Kinesis, Redshift, Quicksight | | ||
| | `compute` | `diagrams.aws.compute` | EC2, Lambda, ECS, EKS, Fargate, Batch | | ||
| | `database` | `diagrams.aws.database` | RDS, Aurora, DynamoDB, ElastiCache, Redshift, Neptune | | ||
MichaelWalker-git marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| | `integration` | `diagrams.aws.integration` | SQS, SNS, StepFunctions, EventBridge, MQ | | ||
| | `management` | `diagrams.aws.management` | CloudWatch, CloudFormation, SystemsManager | | ||
| | `ml` | `diagrams.aws.ml` | Sagemaker, Rekognition, Comprehend, Bedrock | | ||
| | `network` | `diagrams.aws.network` | VPC, ELB, ALB, NLB, CloudFront, Route53, APIGateway | | ||
| | `security` | `diagrams.aws.security` | IAM, Cognito, WAF, KMS, Shield, SecretsManager | | ||
| | `storage` | `diagrams.aws.storage` | S3, EBS, EFS, FSx, Backup | | ||
| | `general` | `diagrams.aws.general` | User, Users, Client, InternetGateway | | ||
|
|
||
| Other categories: `ar`, `blockchain`, `business`, `cost`, `devtools`, `enablement`, `enduser`, `engagement`, `game`, `iot`, `media`, `migration`, `mobile`, `quantum`, `robotics`, `satellite`. | ||
|
|
||
| ## Example: Basic Web Service | ||
|
|
||
| ```python | ||
| from diagrams import Diagram | ||
| from diagrams.aws.compute import EC2 | ||
| from diagrams.aws.database import RDS | ||
| from diagrams.aws.network import ELB | ||
|
|
||
| with Diagram("Web Service", show=False, filename="generated-diagrams/aws-basic"): | ||
MichaelWalker-git marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ELB("lb") >> EC2("web") >> RDS("userdb") | ||
| ``` | ||
|
|
||
| ## Example: Grouped Workers | ||
|
|
||
| ```python | ||
| from diagrams import Diagram | ||
| from diagrams.aws.compute import EC2 | ||
| from diagrams.aws.database import RDS | ||
| from diagrams.aws.network import ELB | ||
|
|
||
| with Diagram("Grouped Workers", show=False, direction="TB", filename="generated-diagrams/aws-workers"): | ||
| ELB("lb") >> [EC2("w1"), EC2("w2"), EC2("w3"), EC2("w4"), EC2("w5")] >> RDS("events") | ||
| ``` | ||
|
|
||
| ## Example: Clustered Web Services | ||
|
|
||
| ```python | ||
| from diagrams import Diagram, Cluster | ||
| from diagrams.aws.compute import ECS | ||
| from diagrams.aws.database import RDS, ElastiCache | ||
| from diagrams.aws.network import ELB, Route53 | ||
|
|
||
| with Diagram("Clustered Web Services", show=False, filename="generated-diagrams/aws-clustered"): | ||
| dns = Route53("dns") | ||
| lb = ELB("lb") | ||
| with Cluster("Services"): | ||
| svc_group = [ECS("web1"), ECS("web2"), ECS("web3")] | ||
| with Cluster("DB Cluster"): | ||
| db_primary = RDS("userdb") | ||
| db_replica = RDS("userdb ro") | ||
| db_primary - db_replica | ||
| memcached = ElastiCache("memcached") | ||
| dns >> lb >> svc_group | ||
| svc_group >> db_primary | ||
| svc_group >> memcached | ||
| ``` | ||
|
|
||
| ## Example: Event Processing | ||
|
|
||
| ```python | ||
| from diagrams import Diagram, Cluster | ||
| from diagrams.aws.compute import ECS, EKS, Lambda | ||
| from diagrams.aws.analytics import Redshift | ||
| from diagrams.aws.integration import SQS | ||
| from diagrams.aws.storage import S3 | ||
|
|
||
| with Diagram("Event Processing", show=False, filename="generated-diagrams/aws-events"): | ||
| source = EKS("k8s source") | ||
| with Cluster("Event Flows"): | ||
| with Cluster("Event Workers"): | ||
| workers = [ECS("w1"), ECS("w2"), ECS("w3")] | ||
| queue = SQS("event queue") | ||
| with Cluster("Processing"): | ||
| handlers = [Lambda("p1"), Lambda("p2"), Lambda("p3")] | ||
| store = S3("events store") | ||
| dw = Redshift("analytics") | ||
| source >> workers >> queue >> handlers | ||
| handlers >> store | ||
| handlers >> dw | ||
| ``` | ||
|
|
||
| ## Example: S3 Image Processing with Bedrock | ||
|
|
||
| ```python | ||
| from diagrams import Diagram, Cluster, Edge | ||
| from diagrams.aws.compute import Lambda | ||
| from diagrams.aws.general import User | ||
| from diagrams.aws.ml import Bedrock | ||
| from diagrams.aws.storage import S3 | ||
|
|
||
| with Diagram("S3 Image Processing", show=False, direction="LR", filename="generated-diagrams/aws-bedrock"): | ||
| user = User("User") | ||
| with Cluster("S3 Bucket"): | ||
| input_folder = S3("Input") | ||
| output_folder = S3("Output") | ||
| fn = Lambda("Processor") | ||
| bedrock = Bedrock("Claude Sonnet") | ||
| user >> Edge(label="Upload") >> input_folder | ||
| input_folder >> Edge(label="Trigger") >> fn | ||
| fn >> Edge(label="Process") >> bedrock | ||
| bedrock >> Edge(label="Result") >> fn | ||
| fn >> Edge(label="Save") >> output_folder | ||
| ``` | ||
109 changes: 109 additions & 0 deletions
109
plugins/deploy-on-aws/skills/diagram/references/dsl-syntax.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| # Diagrams DSL Syntax Reference | ||
MichaelWalker-git marked this conversation as resolved.
Show resolved
Hide resolved
MichaelWalker-git marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ## Basic Structure | ||
|
|
||
| ```python | ||
| from diagrams import Diagram, Cluster, Edge | ||
| from diagrams.aws.compute import EC2, Lambda | ||
MichaelWalker-git marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| from diagrams.aws.database import RDS | ||
| from diagrams.aws.network import ALB | ||
|
|
||
| with Diagram("Title", show=False, filename="generated-diagrams/name"): | ||
| with Cluster("VPC"): | ||
| lb = ALB("ALB") | ||
| with Cluster("Private Subnet"): | ||
| servers = [EC2("web1"), EC2("web2")] | ||
| db = RDS("PostgreSQL") | ||
| lb >> servers >> db | ||
MichaelWalker-git marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ``` | ||
|
|
||
| ## Diagram Constructor | ||
|
|
||
| ```python | ||
| Diagram( | ||
| name="Diagram Title", # Title shown on the diagram | ||
| show=False, # ALWAYS False -- don't open viewer | ||
| filename="path/name", # Output path (no .png extension) | ||
| direction="TB", # TB (top-bottom), LR (left-right), BT, RL | ||
| outformat="png", # png (default), jpg, svg, pdf | ||
MichaelWalker-git marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| ) | ||
| ``` | ||
|
|
||
| ## Connections | ||
|
|
||
| ```python | ||
| node1 >> node2 >> node3 # Left to right flow | ||
| node1 << node2 # Right to left flow | ||
| node1 - node2 # Bidirectional | ||
| node1 >> [node2, node3, node4] # Fan out to multiple | ||
| node1 >> Edge(label="HTTPS", color="darkgreen", style="dashed") >> node2 | ||
| ``` | ||
|
|
||
| ## Clusters | ||
|
|
||
| ```python | ||
| with Cluster("VPC"): | ||
| with Cluster("Public Subnet"): | ||
| lb = ALB("ALB") | ||
| with Cluster("Private Subnet"): | ||
| app = [EC2("app1"), EC2("app2")] | ||
| with Cluster("Data"): | ||
| db = RDS("db") | ||
| lb >> app >> db | ||
| ``` | ||
|
|
||
| ## Edge Styles | ||
|
|
||
| | Parameter | Values | | ||
| | --------- | --------------------------------------------------------------------------------- | | ||
| | `color` | `"darkgreen"`, `"firebrick"`, `"brown"`, `"darkorange"`, `"black"`, any CSS color | | ||
| | `style` | `"solid"`, `"dashed"`, `"dotted"`, `"bold"` | | ||
| | `label` | Any string | | ||
|
|
||
| ## Provider Import Paths | ||
|
|
||
| | Provider | Import Pattern | Example | | ||
| | ----------- | --------------------------------- | --------------------------------------------------- | | ||
| | AWS | `diagrams.aws.<category>` | `from diagrams.aws.compute import EC2` | | ||
| | GCP | `diagrams.gcp.<category>` | `from diagrams.gcp.storage import GCS` | | ||
| | Kubernetes | `diagrams.k8s.<category>` | `from diagrams.k8s.compute import Pod` | | ||
| | On-premises | `diagrams.onprem.<category>` | `from diagrams.onprem.database import PostgreSQL` | | ||
| | SaaS | `diagrams.saas.<category>` | `from diagrams.saas.chat import Slack` | | ||
| | Programming | `diagrams.programming.<category>` | `from diagrams.programming.flowchart import Action` | | ||
| | Generic | `diagrams.generic.<category>` | `from diagrams.generic.compute import Rack` | | ||
| | Custom | `diagrams.custom` | `Custom("name", "icon.png")` | | ||
|
|
||
| ## Common Patterns | ||
|
|
||
| ### Fan-out / Fan-in | ||
|
|
||
| ```python | ||
| source >> [worker1, worker2, worker3] >> sink | ||
| ``` | ||
|
|
||
| ### Bidirectional with Replica | ||
|
|
||
| ```python | ||
| primary = RDS("primary") | ||
| primary - RDS("replica") | ||
| ``` | ||
|
|
||
| ### Nested Clusters | ||
|
|
||
| ```python | ||
| with Cluster("VPC"): | ||
| with Cluster("Public"): | ||
| lb = ALB("ALB") | ||
| with Cluster("Private"): | ||
| app = [EC2("app1"), EC2("app2")] | ||
MichaelWalker-git marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| lb >> app | ||
| ``` | ||
|
|
||
| ### Custom Nodes | ||
|
|
||
| ```python | ||
| from diagrams.custom import Custom | ||
|
|
||
| # Place icon files in your project directory beforehand | ||
| custom_node = Custom("My Service", "./icons/my-service.png") | ||
| ``` | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.