feat: Docker compose support

[hpohekar]

closes #3620

Introduction

Project Name: Fluent Docker and Podman compose

Overview

Docker and Podman compose support same compose.yaml file format to launch Fluent in container mode. This document outlines the technical design for the Fluent compose application.

Problem Statement

Current Fluent container mode supports Docker only. Compose project aims to support both Docker and Podman using same compose file.

Goals and Objectives

1. Support both Docker and Podman for Fluent container launch mode.

2. Provide more robust control on container handling.

3. Ensure scalability to support multiple simultaneous instances of Fluent containers.

Compose Overview

It is a tool for defining and running multi-container applications. We use a compose.yaml file to describe the services and volumes required for an application. Using a single command docker compose up or podman compose up, you can start all services defined in the YAML file.

Design Considerations

Assumptions: Users have local installation of Docker and Podman.

Dependencies: Docker, Podman.

Architecture Design: PyFluent Container Launch Workflow

This architecture enables PyFluent to launch an Ansys Fluent session inside a container (Docker or Podman) based on environment configuration. The workflow dynamically generates a Compose file as text, launches the container using the available engine, and connects to the running Fluent session via a server info file shared between the host and container.

Architecture Diagram (Textual)

User
  |
  | (sets PYFLUENT_LAUNCH_CONTAINER=1)
  v
PyFluent
  |
  |--[Detect Docker/Podman]---> Container Engine (Docker/Podman)
  |                                 ^
  |                                 |
  |--[subprocess.communicate(input=compose_file)]--|
  |
  | (mounts host dir to container)
  v
Fluent Container
  |
  |--[creates server info file in mount target]---> Host Filesystem (mount source)
  |
  v
PyFluent Session Connector
  |
  |--[reads server info file, connects to Fluent]---> Returns Session Object

Workflow Steps

1. Environment Setup

• User sets PYFLUENT_LAUNCH_CONTAINER=1 to signal container launch mode.

2. Container Engine Detection

• PyFluent checks for Docker or Podman availability and selects the engine accordingly.

3. Compose File Generation

• PyFluent dynamically generates a Compose YAML file as a string (not as a physical file).

4. Launching Container

• PyFluent uses the subprocess module to invoke the container engine, passing the Compose YAML via subprocess.communicate(input=compose_file).

5. Container Startup

• The container engine starts the Fluent container as per the Compose spec.

• The container mounts a host directory (mount source) to a target path inside the container (mount target).

6. Server Info File Creation

• Fluent, inside the container, creates a server info file (with connection details) in the mount target directory.

7. Session Connection

• PyFluent, running on the host, reads the server info file from the mount source.

• PyFluent uses this info to connect to the running Fluent session and returns the session object to the user.

Key Design Features

Dynamic Compose Input: No physical Compose file is written; YAML is passed as text directly to the container engine.

Engine Agnostic: PyFluent detects and uses either Docker or Podman, depending on availability.

Mount Sharing: Host and container share a directory for server info file exchange, enabling seamless connection setup.

Session Management: PyFluent connects to Fluent using details from the server info file, abstracting away containerization from the user.

API Design docker_compose.py module

launcher = ComposeLauncher(container_dict)

launcher.check_image_exists() – Check if a Docker image exists locally

launcher.pull_image() - Pull a Docker image if it does not exist locally.

launcher.start() - Start the services.

launcher.stop() - Stop the services.

launcher.ports - Return the ports of the launched services.

End-to-end tests using Pytest

1. Verified on both Windows and Linux, Docker and Podman works fine.

2. CI/CD pipeline already integrated for Docker in the GitHub.

3. Added timeout in subprocess.communicate() and subprocess.wait().

4. Used a mechanism to remove the following docker services:
   - all stopped containers
   - all networks not used by at least one container
   - all volumes not used by at least one container

Summary

This architecture allows PyFluent to flexibly launch Fluent in a containerized environment using either Docker or Podman, with all orchestration handled programmatically and connection to the running session managed via a shared server info file. The process is fully automated and requires minimal user intervention beyond setting the appropriate environment variable and, optionally, mount paths

[mkundu1]

@hpohekar Shall we roll this out as a full feature replacing the usage of the Python docker SDK for container launch mode?

[hpohekar]

@hpohekar Shall we roll this out as a full feature replacing the usage of the Python docker SDK for container launch mode?

@mkundu1 Yes, sure. I will update the code for this feature.

[hpohekar]

@hpohekar We can have a quick CI step (with only few tests) to test the podman-only configuration in a GitHub runner. This can done separately from the current PR.

@mkundu1 Yes, sure. We need to install Podman on all 3 runners and use an env var to select Podman as a compose source for selective tests.

[mkundu1]

@hpohekar We can have a quick CI step (with only few tests) to test the podman-only configuration in a GitHub runner. This can done separately from the current PR.

@mkundu1 Yes, sure. We need to install Podman on all 3 runners and use an env var to select Podman as a compose source for selective tests.

Should test it only in a github-runner, better to avoid any change in the self-hosted runners.