-
Notifications
You must be signed in to change notification settings - Fork 341
feat(generic): Reintroducing the generic SQL module #892
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Tranquility2
wants to merge
29
commits into
testcontainers:main
Choose a base branch
from
Tranquility2:generic_sql
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+472
−4
Open
Changes from 11 commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
f4acd6c
Remove comment line
Tranquility2 af01264
Added base generic db
Tranquility2 869a135
Rename test_generic
Tranquility2 5c3ffbc
Renamed DB to SQL
Tranquility2 39cb0a1
Tests for SQL
Tranquility2 5d15b6e
Update warnning
Tranquility2 80a5355
update docs
Tranquility2 04e418c
Fix doctests
Tranquility2 09d9cad
Add generic dep
Tranquility2 acec2c1
Update ref
Tranquility2 f0bf381
Update lock
Tranquility2 ec9d4e2
Replaced wait
Tranquility2 0f53ccb
Improve Strategy
Tranquility2 29e4ece
Better Strategy
Tranquility2 03dd5e0
Remove _connect
Tranquility2 1da1724
Refactor generic sql
Tranquility2 fe4604b
SQL container with configurable wait strategy
Tranquility2 ed1c991
Merge branch 'main' into generic_sql
Tranquility2 a99045a
use WaitStrategy._poll and with_transient_exceptions
Tranquility2 ad1575a
Required wait_strategy
Tranquility2 0cce48b
Added note
Tranquility2 00e9eaf
Move connector and Rename
Tranquility2 336cb55
Update doctests
Tranquility2 ee5ab80
Fix doctest
Tranquility2 6dbb330
Remove extra validation + Improve testing
Tranquility2 bf6a553
Omit generic.py from report
Tranquility2 6694e44
Renamed sql_connector
Tranquility2 4c7a67f
Renamed SqlConnectWaitStrategy
Tranquility2 af2efe3
Merge branch 'main' into generic_sql
Tranquility2 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,2 @@ | ||
from .server import ServerContainer # noqa: F401 | ||
from .sql import SqlContainer # noqa: F401 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,176 @@ | ||
import logging | ||
from typing import Any, Optional | ||
from urllib.parse import quote, urlencode | ||
|
||
from testcontainers.core.container import DockerContainer | ||
from testcontainers.core.exceptions import ContainerStartException | ||
from testcontainers.core.utils import raise_for_deprecated_parameter | ||
from testcontainers.core.waiting_utils import wait_container_is_ready | ||
|
||
logger = logging.getLogger(__name__) | ||
|
||
ADDITIONAL_TRANSIENT_ERRORS = [] | ||
try: | ||
from sqlalchemy.exc import DBAPIError | ||
|
||
ADDITIONAL_TRANSIENT_ERRORS.append(DBAPIError) | ||
except ImportError: | ||
logger.debug("SQLAlchemy not available, skipping DBAPIError handling") | ||
|
||
|
||
class SqlContainer(DockerContainer): | ||
""" | ||
Generic SQL database container providing common functionality. | ||
This class can serve as a base for database-specific container implementations. | ||
It provides connection management, URL construction, and basic lifecycle methods. | ||
""" | ||
|
||
@wait_container_is_ready(*ADDITIONAL_TRANSIENT_ERRORS) | ||
Tranquility2 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
def _connect(self) -> None: | ||
""" | ||
Test database connectivity using SQLAlchemy. | ||
Raises: | ||
ImportError: If SQLAlchemy is not installed | ||
Exception: If connection fails | ||
""" | ||
try: | ||
import sqlalchemy | ||
except ImportError as e: | ||
logger.error("SQLAlchemy is required for database connectivity testing") | ||
raise ImportError("SQLAlchemy is required for database containers") from e | ||
|
||
connection_url = self.get_connection_url() | ||
|
||
engine = sqlalchemy.create_engine(connection_url) | ||
try: | ||
with engine.connect(): | ||
logger.info("Database connection test successful") | ||
except Exception as e: | ||
logger.error(f"Database connection test failed: {e}") | ||
raise | ||
finally: | ||
engine.dispose() | ||
|
||
def get_connection_url(self) -> str: | ||
""" | ||
Get the database connection URL. | ||
Returns: | ||
str: Database connection URL | ||
Raises: | ||
NotImplementedError: Must be implemented by subclasses | ||
""" | ||
raise NotImplementedError("Subclasses must implement get_connection_url()") | ||
|
||
def _create_connection_url( | ||
Tranquility2 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
self, | ||
dialect: str, | ||
username: str, | ||
password: str, | ||
host: Optional[str] = None, | ||
port: Optional[int] = None, | ||
dbname: Optional[str] = None, | ||
query_params: Optional[dict[str, str]] = None, | ||
**kwargs: Any, | ||
) -> str: | ||
""" | ||
Create a database connection URL. | ||
Args: | ||
dialect: Database dialect (e.g., 'postgresql', 'mysql') | ||
username: Database username | ||
password: Database password | ||
host: Database host (defaults to container host) | ||
port: Database port | ||
dbname: Database name | ||
query_params: Additional query parameters for the URL | ||
**kwargs: Additional parameters (checked for deprecated usage) | ||
Returns: | ||
str: Formatted database connection URL | ||
Raises: | ||
ValueError: If unexpected arguments are provided or required parameters are missing | ||
ContainerStartException: If container is not started | ||
""" | ||
if raise_for_deprecated_parameter(kwargs, "db_name", "dbname"): | ||
raise ValueError(f"Unexpected arguments: {','.join(kwargs)}") | ||
|
||
if self._container is None: | ||
raise ContainerStartException("Container has not been started") | ||
|
||
# Validate required parameters | ||
if not dialect: | ||
Tranquility2 marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
raise ValueError("Database dialect is required") | ||
if not username: | ||
raise ValueError("Database username is required") | ||
if port is None: | ||
raise ValueError("Database port is required") | ||
|
||
host = host or self.get_container_host_ip() | ||
exposed_port = self.get_exposed_port(port) | ||
|
||
# Safely quote password to handle special characters | ||
quoted_password = quote(password, safe="") | ||
quoted_username = quote(username, safe="") | ||
|
||
# Build base URL | ||
url = f"{dialect}://{quoted_username}:{quoted_password}@{host}:{exposed_port}" | ||
|
||
# Add database name if provided | ||
if dbname: | ||
quoted_dbname = quote(dbname, safe="") | ||
url = f"{url}/{quoted_dbname}" | ||
|
||
# Add query parameters if provided | ||
if query_params: | ||
query_string = urlencode(query_params) | ||
url = f"{url}?{query_string}" | ||
|
||
return url | ||
|
||
def start(self) -> "SqlContainer": | ||
""" | ||
Start the database container and perform initialization. | ||
Returns: | ||
SqlContainer: Self for method chaining | ||
Raises: | ||
ContainerStartException: If container fails to start | ||
Exception: If configuration, seed transfer, or connection fails | ||
""" | ||
logger.info(f"Starting database container: {self.image}") | ||
|
||
try: | ||
self._configure() | ||
super().start() | ||
self._transfer_seed() | ||
self._connect() | ||
logger.info("Database container started successfully") | ||
except Exception as e: | ||
logger.error(f"Failed to start database container: {e}") | ||
raise | ||
|
||
return self | ||
|
||
def _configure(self) -> None: | ||
""" | ||
Configure the database container before starting. | ||
Raises: | ||
NotImplementedError: Must be implemented by subclasses | ||
""" | ||
raise NotImplementedError("Subclasses must implement _configure()") | ||
Tranquility2 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
def _transfer_seed(self) -> None: | ||
""" | ||
Transfer seed data to the database container. | ||
This method can be overridden by subclasses to provide | ||
database-specific seeding functionality. | ||
""" | ||
logger.debug("No seed data to transfer") |
File renamed without changes.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.