diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000..17a9e61 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,67 @@ +# Description + +Fixes #(issue number) +_Explain how this code impacts users._ + +## Type of change + +- [ ] New feature (non-breaking change which adds functionality). +- [ ] Bug fix (non-breaking change which fixes an issue). +- [ ] Enhancement (non-breaking change which improves an existing functionality). +- [ ] Breaking change (fix or feature that would cause existing functionality to not work as before). +- [ ] Sub-task of #(issue number) +- [ ] Chore +- [ ] Release + +## Detailed scenario + +### What was tested + +_Describe the scenarios that you tested, and specify if it is automated or manual. For manual scenarios, provide a screenshot of the results._ + +### How to test + +_Describe how the PR can be tested so that the validator can be autonomous: environment, dependencies, specific setup, steps to perform, API requests, etc._ + +### Affected Features & Quality Assurance Scope + +_Please specify which existing features or modules are impacted by the changes in this Pull Request. This information is crucial for the QA team to properly define the testing scope and ensure comprehensive test coverage._ + +## Technical description + +### Documentation + +_Explain how this code works. Diagrams & drawings are welcome._ + +### New dependencies + +_List any new dependencies that are required for this change._ + +### Risks + +_List possible performance & security issues or risks, and explain how they have been mitigated._ + +# Mandatory Checklist + +## Code validation + +- [ ] I validated all the Acceptance Criteria. If possible, provide screenshots or videos. +- [ ] I triggered all changed lines of code at least once without new errors/warnings/notices. +- [ ] I implemented built-in tests to cover the new/changed code. + +## Code style + +- [ ] I wrote a self-explanatory code about what it does. +- [ ] I protected entry points against unexpected inputs. +- [ ] I did not introduce unnecessary complexity. +- [ ] Output messages (errors, notices, logs) are explicit enough for users to understand the issue and are actionnable. + +## Unticked items justification + +_If some mandatory items are not relevant, explain why in this section._ + +# Additional Checks + +- [ ] In the case of complex code, I wrote comments to explain it. +- [ ] When possible, I prepared ways to observe the implemented system (logs, data, etc.). +- [ ] I added error handling logic when using functions that could throw errors (HTTP/API request, filesystem, etc.) diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 0000000..4edcaca --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,138 @@ +## Project Templates + +### Pull Request Template + +When creating pull requests, use this structure: + +```markdown +## Description + +Fixes #issuenumber + +## Type of change + +- [ ] New feature (non-breaking change which adds functionality). +- [ ] Bug fix (non-breaking change which fixes an issue). +- [x] Enhancement (non-breaking change which improves an existing functionality). +- [ ] Breaking change (fix or feature that would cause existing functionality to not work as before). +- [ ] Sub-task of #(issue number) +- [ ] Chore +- [ ] Release + +## Detailed scenario + +### What was tested + +To be tested. + +### How to test + +To be tested. + +### Affected Features & Quality Assurance Scope + +## Technical description + +### Documentation + +Add here what documentation was added. But documentation should be added in the repo itself (/docs). + +### New dependencies + +Add here dependencies for this PR. + +### Risks + +Add here risks involving this PR + +## Mandatory Checklist + +### Code validation + +- [ ] I validated all the Acceptance Criteria. If possible, provide screenshots or videos. +- [ ] I triggered all changed lines of code at least once without new errors/warnings/notices. +- [ ] I implemented built-in tests to cover the new/changed code. + +### Code style + +- [ ] I wrote a self-explanatory code about what it does. +- [ ] I protected entry points against unexpected inputs. +- [ ] I did not introduce unnecessary complexity. +- [ ] Output messages (errors, notices, logs) are explicit enough for users to understand the issue and are actionnable. + +### Unticked items justification + +Add here justification for why you didn't check everything above. + +### Additional Checks + +- [ ] In the case of complex code, I wrote comments to explain it. +- [ ] When possible, I prepared ways to observe the implemented system (logs, data, etc.). +- [ ] I added error handling logic when using functions that could throw errors (HTTP/API request, filesystem, etc.) +``` + +### Bug Grooming Template + +When grooming bugs, use this structure: + +```markdown +**Reproduce the problem** + +1. Do this +2. Then + +**Identify the root cause** +The issue is that … + +**Scope a solution** +To solve the issue, we must … + +**Development steps:** + +- [ ] Add DB column +- [ ] Create new endpoint +- [ ] Implement business logic + +**How will this be validated?** +Consider manual test scenarios and possible automations. Will a specific setup be needed? + +**Grooming confidence level** +Are you sure the proposed solution will work? Have you tested it? Do you foresee any risks or unknowns? + +**Can be peer-coded:** Yes/No + +**Is a refactor needed in that part of the codebase?** +Yes, the data layer could be re-written to be more generic, easing future updates. This would make the effort size XS/S/M/L/XL. + +**Effort estimation:** XS/S/M/L/XL +``` + +### User Story Grooming Template + +When grooming user stories, use this structure: + +```markdown +**Scope a solution** +To implement this feature, we must … + +**Development steps** + +- [ ] Add DB column +- [ ] Create new endpoint +- [ ] Implement business logic + +**How will this be validated?** +Consider manual test scenarios and possible automations. Will a specific setup be needed? + +**Grooming confidence level:** +Are you sure the proposed solution will work? Have you tested it? Do you foresee any risks or unknowns? + +**Can be peer-coded:** Yes/No + +**Is a refactor needed in that part of the codebase?** +Yes, the data layer could be re-written to be more generic, easing future updates. This would make the effort size XS/S/M/L/XL. + +**Effort estimation:** XS/S/M/L/XL +``` + +Remember: This is an internal tool, so prioritize reliability and maintainability over public-facing features. diff --git a/README.md b/README.md index 0f32386..9989332 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,9 @@ The Slack integration requires the app to be declared and installed as a Slack A Run the script app.py The app will listen on the configured port for API calls. +## Documentation +For detailed information about available API endpoints and Slack commands, see [docs/endpoints.md](docs/endpoints.md). + ## Features ### Create GitHub tasks from Slack The Slack app adds a shortcut to create a GitHub task in the configured GitHub Project(V2). When triggered, a modal is opened to fill out information related to the task. diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..5b88a85 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,20 @@ +version: "3.2" + +services: + web: + image: tbtt-app + build: + context: . + dockerfile: Dockerfile + ports: + - "3000:3000" + volumes: + - .:/app + environment: + TBTT_GITHUB_ACCESS_TOKEN: tbtt_github_token + TBTT_SLACK_BOT_USER_TOKEN: tbtt_slack_bot_user_token + TBTT_SLACK_SIGNING_SECRET: tbtt_slack_signing_secret + TBTT_SLACK_USER_TOKEN: tbtt_slack_user_token + TBTT_GITHUB_WEBHOOK_SECRET: tbtt_github_webhook_secret + TBTT_GODP_AUTH_TOKEN: d + TBTT_NOTION_API_KEY: e diff --git a/docs/endpoints.md b/docs/endpoints.md new file mode 100644 index 0000000..873591f --- /dev/null +++ b/docs/endpoints.md @@ -0,0 +1,224 @@ +# Tech Team Bot API Endpoints + +This document describes the available API endpoints and Slack commands for the Tech Team Bot. + +## Support Endpoints + +The support endpoints provide access to IP address information for WP Rocket services. These endpoints are used by the support team to retrieve IP addresses for firewall configuration and troubleshooting purposes. + +### Base URL + +All support endpoints are prefixed with `/support` + +### Endpoints + +#### 1. Get WP Rocket IPs (Human Readable) + +Returns a human-readable list of all IP addresses used by WP Rocket services, including CloudFlare proxy IPs and group.One infrastructure IPs. + +**Endpoint:** `/support/wprocket-ips` + +**Method:** `GET` + +**Authentication:** None required + +**Response Format:** Plain text + +**Response Example:** + +``` +List of IPs used for WP Rocket: + +License validation/activation, update check, plugin information: +https://wp-rocket.me +173.245.48.0/20 +103.21.244.0/22 +[CloudFlare IPv4 and IPv6 ranges...] + +Load CSS Asynchronously: +https://cpcss.wp-rocket.me +46.30.211.168 +46.30.212.76 +[group.One IPs...] +2a02:2350:4:200::/55 + +Remove Unused CSS: +46.30.211.168 +[group.One IPs...] +User Agents: +WP-Rocket/SaaS Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36... + +Dynamic exclusions and inclusions: +https://b.rucss.wp-rocket.me +[group.One IPs...] + +RocketCDN subscription: +https://rocketcdn.me/api/ +[group.One IPs...] +``` + +**Use Case:** + +- Support team reference documentation +- Manual firewall configuration +- Troubleshooting connectivity issues + +--- + +#### 2. Get WP Rocket IPv4 (Machine Readable) + +Returns a machine-readable list of all IPv4 addresses used by WP Rocket services. One IP per line, no descriptive text, with duplicates removed. + +**Endpoint:** `/support/wprocket-ips/ipv4` + +**Method:** `GET` + +**Authentication:** None required + +**Response Format:** Plain text (one IP per line) + +**Response Example:** + +``` +173.245.48.0/20 +103.21.244.0/22 +46.30.211.168 +46.30.212.76 +46.30.212.77 +5.249.224.8 +5.249.224.9 +``` + +**Use Case:** + +- Automated firewall rule generation +- Scripted IP whitelisting +- Integration with security tools + +--- + +#### 3. Get WP Rocket IPv6 (Machine Readable) + +Returns a machine-readable list of all IPv6 addresses used by WP Rocket services. One IP per line, no descriptive text, with duplicates removed. + +**Endpoint:** `/support/wprocket-ips/ipv6` + +**Method:** `GET` + +**Authentication:** None required + +**Response Format:** Plain text (one IP per line) + +**Response Example:** + +``` +2400:cb00::/32 +2606:4700::/32 +2a02:2350:4:200::/55 +``` + +**Use Case:** + +- Automated firewall rule generation for IPv6 +- Scripted IP whitelisting for IPv6 +- Integration with security tools + +--- + +## Slack Commands + +### `/wprocket-ips` Command + +Sends a direct message to the requesting user with a human-readable list of all IP addresses used by WP Rocket services. + +**Command:** `/wprocket-ips` + +**Access:** Available to all Slack workspace members + +**Response:** Direct message (DM) from the bot + +**Response Content:** Same format as the `/support/wprocket-ips` endpoint + +**Example Usage:** + +1. User types `/wprocket-ips` in any Slack channel +2. Bot sends a DM to the user with the complete IP list +3. User can copy the information for firewall configuration or documentation + +**Implementation Details:** + +- Command is handled by `SlackCommandHandler.wp_rocket_ips_command_callback()` +- Processing runs in a separate thread to avoid blocking +- Uses `ServerListHandler.send_wp_rocket_ips_to_slack()` to generate and send the message +- Message includes: + - CloudFlare proxy IPs (IPv4 and IPv6) + - group.One infrastructure IPs (20 individual IPv4 addresses and IPv6 ranges) + - User agent strings for WP Rocket SaaS services + - Service-specific IP groupings with descriptive headers + +--- + +## IP Address Sources + +### CloudFlare IPs + +- Fetched dynamically from `https://www.cloudflare.com/ips-v4/` and `https://www.cloudflare.com/ips-v6/` +- Updated in real-time when endpoints are called +- Used for services proxied through CloudFlare (e.g., wp-rocket.me) + +### group.One IPs + +- **IPv4:** 20 specific IP addresses provided by group.One Ops + - Range: `46.30.211.x`, `46.30.212.x`, `5.249.224.x` + - Used for: Load CSS Asynchronously, Remove Unused CSS, Dynamic exclusions, RocketCDN subscription +- **IPv6:** CIDR ranges from group.One infrastructure + - `2a02:2350:4:200::/55` (k8spods CPH3) + +**Note:** group.One IP addresses are hardcoded based on infrastructure configuration provided by group.One Ops. Contact group.One Ops for updates. + +--- + +## Technical Architecture + +### Module Structure + +``` +TechTeamBot (sources/TechTeamBot.py) + __setup_support_enpoints() + SupportListener (sources/listeners/SupportListener.py) + ServerListHandler (sources/handlers/ServerListHandler.py) + get_cloudflare_proxy_ipv4() + get_cloudflare_proxy_ipv6() + get_groupone_ipv4() + get_groupone_ipv6() + generate_wp_rocket_ips_human_readable() + generate_wp_rocket_ipv4_machine_readable() + generate_wp_rocket_ipv6_machine_readable() +``` + +### Error Handling + +- **CloudFlare fetch errors:** Returns error message in response (e.g., "Error: Unable to reach CloudFlare") +- **Invalid requests:** Standard Flask error handling +- **Slack command errors:** Logged to application logs, user receives no response (Slack timeout) + +--- + +## Maintenance + +### Updating group.One IPs + +To update the group.One IP addresses: + +1. Obtain the updated IP list from group.One Ops +2. Update the `group_one_ips` list in `sources/handlers/ServerListHandler.py:get_groupone_ipv4()` +3. Run tests: `docker-compose exec -T web python -m pytest tests/unit/ServerListHandlerTest.py` +4. Deploy the updated code + +### Monitoring + +- All endpoint calls are logged to the application logs +- CloudFlare IP fetch failures are logged with error details +- Slack command processing is logged with thread start information + +--- diff --git a/sources/handlers/ServerListHandler.py b/sources/handlers/ServerListHandler.py index e08044e..02d85e3 100644 --- a/sources/handlers/ServerListHandler.py +++ b/sources/handlers/ServerListHandler.py @@ -48,19 +48,33 @@ def get_cloudflare_proxy_ips(self, ip_version): def get_groupone_ipv4(self): """ - Lists all IP ranges used by group.One + Lists all IPv4 addresses used by group.One and returns them as a string with one IP per line. """ - groupone_ips = '' # Provided by group.One Ops based on - # https://gitlab.group.one/systems/group.one-authdns/-/blob/main/ipam/internet.yaml?ref_type=heads # Contact group.One ops for more details - groupone_ips += "185.10.8.0/22\n" - groupone_ips += "46.30.210.0/24\n" - groupone_ips += "46.30.211.0/24\n" - groupone_ips += "46.30.212.0/24\n" - groupone_ips += "46.30.214.0/24\n" - groupone_ips += "5.249.224.0/24\n" - return groupone_ips + groupone_ips = [ + "46.30.211.168", + "46.30.212.76", + "46.30.212.77", + "46.30.212.78", + "46.30.212.79", + "46.30.211.69", + "46.30.212.200", + "46.30.212.201", + "46.30.212.202", + "46.30.212.203", + "46.30.211.203", + "46.30.212.204", + "46.30.212.205", + "46.30.212.206", + "46.30.212.207", + "46.30.211.236", + "5.249.224.8", + "5.249.224.9", + "5.249.224.10", + "5.249.224.11", + ] + return "\n".join(groupone_ips) + "\n" def get_groupone_ipv6(self): """ diff --git a/tests/unit/ServerListHandlerTest.py b/tests/unit/ServerListHandlerTest.py new file mode 100644 index 0000000..ff72e0b --- /dev/null +++ b/tests/unit/ServerListHandlerTest.py @@ -0,0 +1,307 @@ +""" + Unit tests for the ServerListHandler.py main file +""" + +from unittest.mock import Mock, patch + +import requests + +from sources.handlers.ServerListHandler import ServerListHandler + +# pylint: disable=unused-argument + + +def mock_cloudflare_ipv4_response(*args, **kwargs): + """ + Mocks the requests.get response for CloudFlare IPv4 + """ + + class RequestReturn: + """ + Mocks the return of requests.get + """ + + status_code = 200 + text = "173.245.48.0/20\n103.21.244.0/22\n103.22.200.0/22" + + return RequestReturn() + + +def mock_cloudflare_ipv6_response(*args, **kwargs): + """ + Mocks the requests.get response for CloudFlare IPv6 + """ + + class RequestReturn: + """ + Mocks the return of requests.get + """ + + status_code = 200 + text = "2400:cb00::/32\n2606:4700::/32\n2803:f800::/32" + + return RequestReturn() + + +def mock_cloudflare_error_response(*args, **kwargs): + """ + Mocks the requests.get response for CloudFlare with error status + """ + + class RequestReturn: + """ + Mocks the return of requests.get with error + """ + + status_code = 404 + + return RequestReturn() + + +@patch( + "sources.handlers.ServerListHandler.requests.get", + side_effect=mock_cloudflare_ipv4_response, +) +def test_get_cloudflare_proxy_ipv4(mock_requests): + """ + Tests the get_cloudflare_proxy_ipv4 method returns CloudFlare IPv4 addresses + """ + handler = ServerListHandler() + result = handler.get_cloudflare_proxy_ipv4() + + assert result == "173.245.48.0/20\n103.21.244.0/22\n103.22.200.0/22\n" + mock_requests.assert_called_once() + assert mock_requests.call_args[0][0] == "https://www.cloudflare.com/ips-v4/" + + +@patch( + "sources.handlers.ServerListHandler.requests.get", + side_effect=mock_cloudflare_ipv6_response, +) +def test_get_cloudflare_proxy_ipv6(mock_requests): + """ + Tests the get_cloudflare_proxy_ipv6 method returns CloudFlare IPv6 addresses + """ + handler = ServerListHandler() + result = handler.get_cloudflare_proxy_ipv6() + + assert result == "2400:cb00::/32\n2606:4700::/32\n2803:f800::/32\n" + mock_requests.assert_called_once() + assert mock_requests.call_args[0][0] == "https://www.cloudflare.com/ips-v6/" + + +@patch( + "sources.handlers.ServerListHandler.requests.get", + side_effect=mock_cloudflare_error_response, +) +def test_get_cloudflare_proxy_ips_error(mock_requests): + """ + Tests the get_cloudflare_proxy_ips method handles error status codes + """ + handler = ServerListHandler() + result = handler.get_cloudflare_proxy_ips("v4") + + assert "Error: Unable to fetch CloudFlare IPs. Status code: 404" in result + mock_requests.assert_called_once() + + +@patch("sources.handlers.ServerListHandler.requests.get") +def test_get_cloudflare_proxy_ips_exception(mock_requests): + """ + Tests the get_cloudflare_proxy_ips method handles request exceptions + """ + mock_requests.side_effect = requests.exceptions.RequestException( + "Connection timeout" + ) + handler = ServerListHandler() + result = handler.get_cloudflare_proxy_ips("v4") + + assert "Error: Unable to reach CloudFlare" in result + + +def test_get_groupone_ipv4(): + """ + Tests the get_groupone_ipv4 method returns the correct list of IPs + """ + handler = ServerListHandler() + result = handler.get_groupone_ipv4() + + # Verify it returns a string with newline-separated IPs + assert isinstance(result, str) + assert "\n" in result + + # Verify specific IPs are in the list + expected_ips = [ + "46.30.211.168", + "46.30.212.76", + "46.30.212.77", + "46.30.212.78", + "46.30.212.79", + "46.30.211.69", + "46.30.212.200", + "46.30.212.201", + "46.30.212.202", + "46.30.212.203", + "46.30.211.203", + "46.30.212.204", + "46.30.212.205", + "46.30.212.206", + "46.30.212.207", + "46.30.211.236", + "5.249.224.8", + "5.249.224.9", + "5.249.224.10", + "5.249.224.11", + ] + + result_lines = [line for line in result.split("\n") if line] # Filter out empty strings + for ip in expected_ips: # pylint: disable=invalid-name + assert ip in result_lines, f"Expected IP {ip} not found in result" + + # Verify the count matches + assert len(result_lines) == len(expected_ips) + + +def test_get_groupone_ipv4_format(): + """ + Tests that get_groupone_ipv4 returns IPs in the correct format (one per line) + """ + handler = ServerListHandler() + result = handler.get_groupone_ipv4() + + # Verify each line contains a valid IP format + for line in result.split("\n"): + if line: # Skip empty lines + # Check basic IP format (X.X.X.X) + parts = line.split(".") + assert len(parts) == 4, f"Invalid IP format: {line}" + for part in parts: + assert part.isdigit(), f"Invalid IP part: {part}" + assert 0 <= int(part) <= 255, f"Invalid IP octet: {part}" + + +def test_get_groupone_ipv6(): + """ + Tests the get_groupone_ipv6 method returns the correct IPv6 range + """ + handler = ServerListHandler() + result = handler.get_groupone_ipv6() + + assert "2a02:2350:4:200::/55" in result + assert result.endswith("\n") + + +@patch( + "sources.handlers.ServerListHandler.requests.get", + side_effect=mock_cloudflare_ipv4_response, +) +def test_generate_wp_rocket_ips_human_readable(mock_requests): + """ + Tests the generate_wp_rocket_ips_human_readable method includes all sections + """ + handler = ServerListHandler() + result = handler.generate_wp_rocket_ips_human_readable() + + # Verify it includes main sections + assert "List of IPs used for WP Rocket:" in result + assert "License validation/activation" in result + assert "https://wp-rocket.me" in result + assert "Load CSS Asynchronously:" in result + assert "https://cpcss.wp-rocket.me" in result + assert "Remove Unused CSS:" in result + assert "User Agents:" in result + assert "WP-Rocket/SaaS" in result + assert "Dynamic exclusions and inclusions:" in result + assert "https://b.rucss.wp-rocket.me" in result + assert "RocketCDN subscription:" in result + assert "https://rocketcdn.me/api/" in result + + # Verify it includes group.one IPs + assert "46.30.211.168" in result + assert "5.249.224.11" in result + + +@patch( + "sources.handlers.ServerListHandler.requests.get", + side_effect=mock_cloudflare_ipv4_response, +) +@patch("sources.utils.Duplication.remove_duplicated_lines") +def test_generate_wp_rocket_ipv4_machine_readable(mock_dedup, mock_requests): + """ + Tests the generate_wp_rocket_ipv4_machine_readable method + """ + mock_dedup.return_value = "173.245.48.0/20\n46.30.211.168\n" + + handler = ServerListHandler() + result = handler.generate_wp_rocket_ipv4_machine_readable() + + # Verify deduplication was called + mock_dedup.assert_called_once() + + # Verify the result is machine-readable (no text headers) + assert "List of IPs" not in result + assert "CloudFlare" not in result + + +@patch( + "sources.handlers.ServerListHandler.requests.get", + side_effect=mock_cloudflare_ipv6_response, +) +@patch("sources.utils.Duplication.remove_duplicated_lines") +def test_generate_wp_rocket_ipv6_machine_readable(mock_dedup, mock_requests): + """ + Tests the generate_wp_rocket_ipv6_machine_readable method + """ + mock_dedup.return_value = "2400:cb00::/32\n2a02:2350:4:200::/55\n" + + handler = ServerListHandler() + result = handler.generate_wp_rocket_ipv6_machine_readable() + + # Verify deduplication was called + mock_dedup.assert_called_once() + + # Verify the result is machine-readable (no text headers) + assert "List of IPs" not in result + assert "CloudFlare" not in result + + +@patch( + "sources.handlers.ServerListHandler.requests.get", + side_effect=mock_cloudflare_ipv4_response, +) +def test_send_wp_rocket_ips_to_slack(mock_requests): + """ + Tests the send_wp_rocket_ips_to_slack method calls post_message + """ + handler = ServerListHandler() + mock_app_context = Mock() + mock_slack_user = "U123456" + + # Mock the post_message method + handler.slack_message_factory.post_message = Mock() + + handler.send_wp_rocket_ips_to_slack(mock_app_context, mock_slack_user) + + # Verify post_message was called + handler.slack_message_factory.post_message.assert_called_once() + call_args = handler.slack_message_factory.post_message.call_args[0] + + assert call_args[0] == mock_app_context + assert call_args[1] == mock_slack_user + assert "List of IPs used for WP Rocket:" in call_args[2] + + +def test_get_groupone_ipv4_no_ranges(): + """ + Tests that get_groupone_ipv4 returns individual IPs, not CIDR ranges + """ + handler = ServerListHandler() + result = handler.get_groupone_ipv4() + + # Verify no CIDR notation is present (no /XX patterns) + assert "/" not in result, "Result should not contain CIDR notation (e.g., /22, /24)" + + # Verify all lines are individual IPs + for line in result.split("\n"): + if line: + assert "/" not in line, f"Line should not contain CIDR notation: {line}"