Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unclosed coonnection #1124

Closed
5 of 6 tasks
maretodoric opened this issue Jul 8, 2024 · 3 comments
Closed
5 of 6 tasks

Unclosed coonnection #1124

maretodoric opened this issue Jul 8, 2024 · 3 comments
Labels
duplicate This issue or pull request already exists

Comments

@maretodoric
Copy link

Describe the bug

Unclosed connection
client_connection: Connection<ConnectionKey(host='bucket-name.s3.eu-central-1.amazonaws.com', port=443, is_ssl=True, ssl=None, proxy=None, proxy_auth=None, proxy_headers_hash=None)>

Warning shows up when using S3 client, get_object method.

This is the code I've used:

async def get_s3_object(bucket: str, filepath: str):
    filepath = str(filepath)
    cfg = load_settings()

    session = get_session()
    async with session.create_client('s3', region_name='eu-central-1',
                                     aws_secret_access_key=cfg.aws.secret_key,
                                     aws_access_key_id=cfg.aws.key) as client:
        # get object from s3
        try: s3_ob = await client.get_object(Bucket=bucket, Key=filepath)
        except Exception as e:
            if type(e).__name__ == 'NoSuchKey':
                raise FileNotFoundError(f"File '{basename(filepath)}' not found in bucket: '{bucket}'.")
            else:
                raise
        if s3_ob['ResponseMetadata']['HTTPStatusCode'] != 200:
            raise Exception(f"HTTP Status Code invalid. Response: {s3_ob}")
        else:
            return S3Object(bucket, filepath, s3_ob['ContentLength'], s3_ob['ContentType'])

Checklist

  • I have reproduced in environment where pip check passes without errors
  • I have provided pip freeze results
  • I have provided sample code or detailed way to reproduce
  • I have tried the same code in botocore to ensure this is an aiobotocore specific issue
  • I have tried similar code in aiohttp to ensure this is is an aiobotocore specific issue - i wouldn't know how to replicate this with aiohttp
  • I have checked the latest and older versions of aiobotocore/aiohttp/python to see if this is a regression / injection

pip freeze results

aioboto3==12.4.0
aiobotocore==2.12.3
aiofiles==22.1.0
aiohttp==3.8.4
aioitertools==0.11.0
aiomysql==0.2.0
aiosignal==1.3.1
alembic==1.11.3
anyio==3.7.1
async-timeout==4.0.2
asyncssh==2.13.2
attrs==23.1.0
beautifulsoup4==4.12.2
boto3==1.34.69
botocore==1.34.69
certifi==2023.5.7
cffi==1.15.1
charset-normalizer==2.1.1
click==8.1.4
cryptography==39.0.0
dccalertmanager @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccalertmanager.git@c88cd8af2415394621b10781d693ac2f52022bb8
dccjenkins @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccjenkins.git@befd2d732e2f633e7965d11daa58af83003a7a3e
dccpassbolt @ git+https://dcc-gitlab.addiko.com/mtodoric/pydccpassbolt.git@ee8fd08e569ebc7a5a68e230801fa2c9763ea9b9
dccwebtool @ file:///home/mtodoric/dev-in-progress/pydccwebtool
dnspython==2.0.0
ecs-logging==2.2.0
elastic-apm==6.22.3
elasticsearch==7.17.6
elasticsearch-dsl==7.4.0
exceptiongroup==1.1.2
fastapi==0.90.0
frozenlist==1.3.3
gitdb==4.0.10
GitPython==3.1.32
greenlet==2.0.2
h11==0.14.0
httpcore==0.16.3
httpx==0.23.3
idna==3.4
iniconfig==2.0.0
ipwhois==1.2.0
Jinja2==3.1.2
jmespath==1.0.1
Mako==1.2.4
MarkupSafe==2.1.3
multidict==6.0.4
packaging==23.1
pluggy==1.2.0
pycparser==2.21
pydantic==1.10.9
PyMySQL==1.1.0
pytest==7.4.0
python-dateutil==2.8.2
python-dotenv==1.0.1
python-gnupg==0.5.1
python-multipart==0.0.5
PyYAML==6.0
requests==2.28.1
rfc3986==1.5.0
s3transfer==0.10.1
six==1.16.0
SkPy==0.10.5
smmap==5.0.0
sniffio==1.3.0
soupsieve==2.4.1
SQLAlchemy==2.0.15
starlette==0.22.0
tomli==2.0.1
typing_extensions==4.7.1
urllib3==1.26.16
uvicorn==0.20.0
websockets==10.4
wrapt==1.16.0
yarl==1.9.2

Environment:

  • Python Version: 3.10.12
  • OS name and version: Ubuntu 22.04

Additional context
I've first noticed this when using aioboto3, however when i was told to try and replicate this with aiobotocore - i did and it can be replicated.
Warning itself doesn't appear to cause any issues but potential high number of unclosed connections could pose an issue.

Copy link

github-actions bot commented Sep 6, 2024

This issue has been marked as stale because it has been inactive for more than 60 days. Please update this pull request or it will be automatically closed in 7 days.

@github-actions github-actions bot added the Stale label Sep 6, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 14, 2024
@thehesiod thehesiod reopened this Sep 15, 2024
@github-actions github-actions bot removed the Stale label Sep 15, 2024
@thehesiod
Copy link
Collaborator

you need to provide a "clean" test example that can be run stand-alone without third party modules, like we have no idea what S3Object does

@thehesiod thehesiod added the duplicate This issue or pull request already exists label Dec 28, 2024
@thehesiod
Copy link
Collaborator

most likely duplicate of #1211. Please try python3.11+

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

2 participants