feat: Docker compose support

[hpohekar]

closes #3620

Introduction

Project Name: Fluent Docker and Podman compose

Overview

Docker and Podman compose support same compose.yaml file format to launch Fluent in container mode. This document outlines the technical design for the Fluent compose application.

Problem Statement

Current Fluent container mode supports Docker only. Compose project aims to support both Docker and Podman using same compose file.

Goals and Objectives

1. Support both Docker and Podman for Fluent container launch mode.

2. Provide more robust control on container handling.

3. Ensure scalability to support multiple simultaneous instances of Fluent containers.

Compose Overview

It is a tool for defining and running multi-container applications. We use a compose.yaml file to describe the services and volumes required for an application. Using a single command docker compose up or podman compose up, you can start all services defined in the YAML file.

Design Considerations

Assumptions: Users have local installation of Docker and Podman.

Dependencies: Docker, Podman.

Architecture Design: PyFluent Container Launch Workflow

This architecture enables PyFluent to launch an Ansys Fluent session inside a container (Docker or Podman) based on environment configuration. The workflow dynamically generates a Compose file as text, launches the container using the available engine, and connects to the running Fluent session via a server info file shared between the host and container.

Architecture Diagram (Textual)

User
  |
  | (sets PYFLUENT_LAUNCH_CONTAINER=1 and PYFLUENT_USE_DOCKER_COMPOSE=1 or PYFLUENT_USE_PODMAN_COMPOSE=1)
  v
PyFluent
  |
  |--[Detect Docker/Podman]---> Container Engine (Docker/Podman)
  |                                 ^
  |                                 |
  |--[subprocess.communicate(input=compose_file)]--|
  |
  | (mounts host dir to container)
  v
Fluent Container
  |
  |--[creates server info file in mount target]---> Host Filesystem (mount source)
  |
  v
PyFluent Session Connector
  |
  |--[reads server info file, connects to Fluent]---> Returns Session Object

Key Design Features

Dynamic Compose Input: No physical Compose file is written; YAML is passed as text directly to the container engine.

Engine Agnostic: PyFluent detects and uses either Docker or Podman, depending on availability.

Mount Sharing: Host and container share a directory for server info file exchange, enabling seamless connection setup.

Session Management: PyFluent connects to Fluent using details from the server info file, abstracting away containerization from the user.

API Design docker_compose.py module

launcher = ComposeLauncher(container_dict)

launcher.check_image_exists() – Check if a Docker image exists locally

launcher.pull_image() - Pull a Docker image if it does not exist locally.

launcher.start() - Start the services.

launcher.stop() - Stop the services.

launcher.ports - Return the ports of the launched services.

End-to-end tests using Pytest

1. Verified on both Windows and Linux, Docker and Podman works fine.

2. CI/CD pipeline already integrated for Docker in the GitHub.

3. Added timeout in subprocess.communicate() and subprocess.wait().

4. Used a mechanism to remove the following docker services:
   - all stopped containers
   - all networks not used by at least one container
   - all volumes not used by at least one container

Summary

This architecture allows PyFluent to flexibly launch Fluent in a containerized environment using either Docker or Podman, with all orchestration handled programmatically and connection to the running session managed via a shared server info file. The process is fully automated and requires minimal user intervention beyond setting the appropriate environment variable and, optionally, mount paths

[mkundu1]

@hpohekar Shall we roll this out as a full feature replacing the usage of the Python docker SDK for container launch mode?

[hpohekar]

@hpohekar Shall we roll this out as a full feature replacing the usage of the Python docker SDK for container launch mode?

@mkundu1 Yes, sure. I will update the code for this feature.

[mkundu1]

@hpohekar As you said, in the new plan we are not making any change in the default behaviour. But, it seems there are some changes in the exiting code, like usage of inside_container is replaced with get_container(), which don't seem to be related to compose support.

Ideally, to minimize the changes in existing code, can we maintain some structure like the following?

self._compose = <value of the new config>
...
if self._compose:
  <new code>
else:
  <exisiting code>

If there are some useful changes in the existing code, please move them to a separate PR. It will be easier to review just the new changes in the current PR.

[hpohekar]

@hpohekar As you said, in the new plan we are not making any change in the default behaviour. But, it seems there are some changes in the exiting code, like usage of inside_container is replaced with get_container(), which don't seem to be related to compose support.

Ideally, to minimize the changes in existing code, can we maintain some structure like the following?

self._compose = <value of the new config>
...
if self._compose:
  <new code>
else:
  <exisiting code>

If there are some useful changes in the existing code, please move them to a separate PR. It will be easier to review just the new changes in the current PR.

@mkundu1

Yes, that's true. We are not making any changes in default behavior as passing of all unit tests already proved this. We have updated the existing code to make it compatible with all of Docker SDK, Docker compose and Podman compose.

Default behavior means Docker SDK should work fine for unit testing in CI and at the same time test_docker_compose should pass using Docker compose and both of them are working fine.

We are already using the structure mentioned by you.

[hpohekar]

@seanpearsonuk @mkundu1 This PR is ready. We are supporting Docker SDK, Docker compose and Podman compose now. Please share suggestions if you have any, otherwise we can merge this now. Thank you.