Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADCM-6240: [With ADCM API error inspection] Redesign the error detection mechanism and make them more user-friendly #58

Draft
wants to merge 47 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
a69424b
ADCM-6064 Prepare base entities
a-alferov Nov 4, 2024
2699ace
ADCM-6073 Introduce basic workflows (#3)
Sealwing Nov 12, 2024
91e8fc1
ADCM-6067: Implement base Requester (#1)
DanBalalan Nov 13, 2024
b765529
ADCM-6068: Implement base Accessor (#2)
Starovoitov Nov 14, 2024
c413f18
ADCM-6076 Introduce base objects and very simple Cluster implementati…
Sealwing Nov 15, 2024
54cc39a
ADCM-6076 Add unit tests (#7)
Sealwing Nov 18, 2024
b52e635
ADCM-6114 Implement Cluster own methods & drop caches on `refresh` ca…
Sealwing Nov 18, 2024
e4ef859
ADCM-6118: Implement Host object (#10)
Starovoitov Nov 20, 2024
c350f02
ADCM-6119: Implement HostProvider object (#8)
Starovoitov Nov 20, 2024
621ae63
ADCM-6115: Implement Service object (#12)
DanBalalan Nov 21, 2024
55cceaf
ADCM-6117: Implement Component object (#13)
DanBalalan Nov 22, 2024
32ab331
ADCM-6122: Implement ADCM object (#11)
Starovoitov Nov 22, 2024
3ecf9cc
ADCM-6133: Implement node for getting hosts (#15)
DanBalalan Nov 26, 2024
3d99dc8
ADCM-6143: Change `cached_property` to `async_cached_property` (#18)
DanBalalan Nov 27, 2024
2ab48e4
ADCM-6120: Prepare infrastructure for integration tests. (#16)
Starovoitov Nov 27, 2024
9036f74
ADCM-6125 Configuration management (#17)
Sealwing Nov 29, 2024
3699a52
Revert integration tests to running (#27)
a-alferov Nov 29, 2024
4d02c26
ADCM-6138: Implement Bundle object (#20)
Starovoitov Nov 29, 2024
50acda0
ADCM-6157: Implement missing methods for HostProviderNode (#22)
DanBalalan Nov 29, 2024
f68dbe4
ADCM-6134: Implement node for getting actions (#21)
DanBalalan Dec 2, 2024
31b11f8
ADCM-6156: Implement missing methods for BundleNode (#26)
Starovoitov Dec 3, 2024
a5e612a
ADCM-6154: Implement missing methods for ClusterNode (#25)
DanBalalan Dec 3, 2024
9e6838d
ADCM-6139: Implement License object (#19)
Starovoitov Dec 3, 2024
de419f6
Fix CODEOWNERS file (#33)
a-alferov Dec 4, 2024
fee702f
ADCM-6126 Mapping management (#29)
Sealwing Dec 4, 2024
6baf0ef
ADCM-6158: Implement missing methods for HostNode (#30)
Starovoitov Dec 5, 2024
c194bfb
ADCM-6155: Implement missing methods for ServiceNode (#28)
Starovoitov Dec 5, 2024
8bdac72
ADCM-6176: Implement node for getting jobs (#34)
Starovoitov Dec 6, 2024
82e0b1f
ADCM-6177: Implement node for getting upgrades (#37)
Starovoitov Dec 6, 2024
42a2e7f
ADCM-6161: Implement Config/ActionHostGroup objects with nodes (#36)
DanBalalan Dec 9, 2024
4a75af2
ADCM-6077 Implement basic filters (#35)
Sealwing Dec 9, 2024
3bb3802
ADCM-6187 Added basic test on mapping with real ADCM (#39)
Sealwing Dec 11, 2024
54ddc12
ADCM-6195: Add supported ADCM version in client (#41)
DanBalalan Dec 12, 2024
d350478
ADCM-6187 Basic test on config with real ADCM (#42)
Sealwing Dec 13, 2024
1944222
ADCM-6151: Make status an arbitraty string for all objects (#40)
Starovoitov Dec 13, 2024
b9db241
ADCM-6219 Change defaults for builder; fix hosts addition to cluster;…
Sealwing Dec 13, 2024
307fa2e
ADCM-6200: Implement wait() method for Job (#43)
Starovoitov Dec 13, 2024
481906a
ADCM-6218 Implement `ADCMSession` (#46)
Sealwing Dec 16, 2024
1be9a00
ADCM-6189: Implement missing methods for imports (#38)
Starovoitov Dec 17, 2024
a5b5d0d
ADCM-6184 Improve interactions with config (#47)
Sealwing Dec 17, 2024
547037b
ADCM-6232 Add more info on filter parsing error and only clean http c…
Sealwing Dec 18, 2024
822c401
ADCM-6212: Add tests for Bundle/HostProvider/Host public API (#49)
Starovoitov Dec 24, 2024
d0fa697
ADCM-6211: Add tests for Cluster/Service/Component public API (#55)
DanBalalan Dec 24, 2024
ae30c15
ADCM-6210 Test on Jobs and related fixes (#52)
Sealwing Dec 25, 2024
688e6d6
ADCM-6240: rework errors
DanBalalan Dec 24, 2024
6bc1779
Merge branch 'refs/heads/develop' into ADCM-6240
DanBalalan Dec 25, 2024
0a746fe
ADCM-6240: linters fix after merge
DanBalalan Dec 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
* @a-alferov @Sealwing
*.py @a-alferov @Sealwing @DanBalalan @Starovoitov
16 changes: 16 additions & 0 deletions .github/workflows/on_push_to_pull_request.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Validate Pull Request Changes

# most important for us is "synchronize" event for pull request, which is included by default
on: pull_request

jobs:
lint:
name: Lint Python code
uses: ./.github/workflows/step_lint.yaml

unit_tests:
name: Run unit tests
uses: ./.github/workflows/step_test_from_dir.yaml
with:
target: tests/unit
description: Unit
23 changes: 23 additions & 0 deletions .github/workflows/step_lint.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Run Linters

on:
workflow_call:

jobs:
lint-python:
name: Lint Python Code
runs-on: ubuntu-24.04
env:
CODE_DIRS: "adcm_aio_client tests"
steps:
- name: Install poetry
run: python -m pip install poetry
- uses: actions/checkout@v4
- name: Install dependencies
run: poetry install --with dev --with test --no-root
- name: Run ruff lint check
run: poetry run ruff check $CODE_DIRS
- name: Run ruff format check
run: poetry run ruff format --check $CODE_DIRS
- name: Run pyright check
run: poetry run pyright $CODE_DIRS
32 changes: 32 additions & 0 deletions .github/workflows/step_test_from_dir.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Run Tests
run-name: "Run Tests: ${{ inputs.description }}"

on:
workflow_call:
inputs:
target:
type: string
required: true
description: "Directory with tests to aim to"
description:
type: string
required: false
default: "unspecified"
description: "Name to use in `run-name` for tests to be more specific"

jobs:
run-pytest-in-dir:
name: Run Tests
runs-on: ubuntu-24.04
env:
CODE_DIRS: "adcm_aio_client tests"
steps:
- name: Install poetry
run: python -m pip install poetry
- uses: actions/checkout@v4
- name: Install dependencies
# install "with root" so adcm_aio_client
# will be accessible without PYTHONPATH manipulations
run: poetry install --with test
- name: Run tests
run: poetry run pytest ${{ inputs.target }} -v
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -159,4 +159,4 @@ cython_debug/
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
.idea/
15 changes: 15 additions & 0 deletions adcm_aio_client/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from adcm_aio_client._session import ADCMSession

__all__ = ["ADCMSession"]
143 changes: 143 additions & 0 deletions adcm_aio_client/_session.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
from json import JSONDecodeError
from types import TracebackType
from typing import Self

import httpx
import adcm_version

from adcm_aio_client.core.client import ADCMClient
from adcm_aio_client.core.errors import ClientInitError, NotSupportedVersionError
from adcm_aio_client.core.requesters import BundleRetriever, DefaultRequester
from adcm_aio_client.core.types import Cert, ConnectionSecurity, Credentials, RequestPolicy, RetryPolicy, SessionInfo

MIN_ADCM_VERSION = "2.5.0"


class ADCMSession:
def __init__(
self: Self,
# basics
url: str,
credentials: Credentials,
*,
# security
verify: str | bool = True,
cert: Cert | None = None,
# requesting behavior
timeout: int = 600,
retry_attempts: int = 3,
retry_interval: int = 1,
) -> None:
self._session_info = SessionInfo(
url=url, credentials=credentials, security=ConnectionSecurity(verify=verify, certificate=cert)
)
self._request_policy = RequestPolicy(
timeout=timeout, retry=RetryPolicy(attempts=retry_attempts, interval=retry_interval)
)

self._http_client = None
self._requester = None
self._adcm_client = None

# Context Manager

async def __aenter__(self: Self) -> ADCMClient:
self._http_client = await self._prepare_http_client_for_running_adcm()
adcm_version_ = await _ensure_adcm_version_is_supported(client=self._http_client)

try:
self._requester = self._prepare_api_v2_requester()
await self._requester.login(self._session_info.credentials)
except Exception as e:
await self.__close_http_client_safe(exc_type=type(e), exc_value=e)
raise

self._adcm_client = self._prepare_adcm_client(version=adcm_version_)
return self._adcm_client

async def __aexit__(
self: Self,
exc_type: type[BaseException] | None = None,
exc_value: BaseException | None = None,
traceback: TracebackType | None = None,
) -> None:
await self.__close_requester_safe(exc_type, exc_value, traceback)
await self.__close_http_client_safe(exc_type, exc_value, traceback)

async def __close_requester_safe(
self: Self,
exc_type: type[BaseException] | None = None,
exc_value: BaseException | None = None,
traceback: TracebackType | None = None,
) -> None:
if self._requester:
try:
await self._requester.logout()
except:
await self.__close_http_client_safe(exc_type, exc_value, traceback)

raise

async def __close_http_client_safe(
self: Self,
exc_type: type[BaseException] | None = None,
exc_value: BaseException | None = None,
traceback: TracebackType | None = None,
) -> None:
if self._http_client:
await self._http_client.__aexit__(exc_type, exc_value, traceback)

# Steps

async def _prepare_http_client_for_running_adcm(self: Self) -> httpx.AsyncClient:
client = httpx.AsyncClient(
base_url=self._session_info.url,
timeout=self._request_policy.timeout,
verify=self._session_info.security.verify,
cert=self._session_info.security.certificate,
)

try:
await client.head("/")
except httpx.NetworkError as e:
await client.__aexit__(type(e), e)
message = f"Failed to connect to ADCM at URL {self._session_info.url}"
raise ClientInitError(message) from e

return client

def _prepare_api_v2_requester(self: Self) -> DefaultRequester:
if self._http_client is None:
message = "Failed to prepare requester: HTTP client is not initialized"
raise RuntimeError(message)

return DefaultRequester(http_client=self._http_client, retries=self._request_policy.retry)

def _prepare_adcm_client(self: Self, version: str) -> ADCMClient:
if self._requester is None:
message = "Failed to prepare ADCM client: requester is not initialized"
raise RuntimeError(message)

bundle_retriever = BundleRetriever()

return ADCMClient(requester=self._requester, bundle_retriever=bundle_retriever, adcm_version=version)


async def _ensure_adcm_version_is_supported(client: httpx.AsyncClient) -> str:
try:
# todo check for VERY old versions if that request will raise error
response = await client.get("/versions/")
data = response.json()
version = str(data["adcm"]["version"])
except (JSONDecodeError, KeyError) as e:
message = (
f"Failed to detect ADCM version at {client.base_url}. "
f"Most likely ADCM version is lesser than {MIN_ADCM_VERSION}"
)
raise NotSupportedVersionError(message) from e

if adcm_version.compare_adcm_versions(version, MIN_ADCM_VERSION) < 0:
message = f"Minimal supported ADCM version is {MIN_ADCM_VERSION}. Got {adcm_version}"
raise NotSupportedVersionError(message)

return version
11 changes: 11 additions & 0 deletions adcm_aio_client/core/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
3 changes: 3 additions & 0 deletions adcm_aio_client/core/actions/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from adcm_aio_client.core.actions._objects import ActionsAccessor, UpgradeNode

__all__ = ["ActionsAccessor", "UpgradeNode"]
Loading