Airflow Triggerer memory leak

[deepak4babu]

Under which category would you file this issue?

Airflow Core

Apache Airflow version

3.2.0

What happened and how to reproduce it?

Airflow trigger memory is increasing continously

What you think should happen instead?

Memory should drop after certain period

Operating System

Debian GNU/Linux 12 (bookworm)

Deployment

Other

Apache Airflow Provider(s)

No response

Versions of Apache Airflow Providers

apache-airflow-providers-celery==3.17.1
apache-airflow-providers-cncf-kubernetes==10.14.0
apache-airflow-providers-common-compat==1.14.1
apache-airflow-providers-common-io==1.7.1
apache-airflow-providers-common-sql==1.33.0
apache-airflow-providers-fab==3.6.1
apache-airflow-providers-smtp==2.4.3
apache-airflow-providers-standard==1.12.1

Official Helm Chart version

Not Applicable

Kubernetes Version

Not applicable

Helm Chart configuration

Not applicable

Docker Image customizations

FROM sorocoscoutdev.azurecr.io/docker-io/apache/airflow:slim-3.2.0-python3.12

USER root

RUN apt-get update -y || true
RUN apt-get install --fix-missing -y \
    apt-transport-https=2.6.1 \
    build-essential=12.9 \
    default-libmysqlclient-dev=1.1.0 \
    g++=4:12.2.0-3 \
    gcc=4:12.2.0-3 \
    gnupg2=2.2.40-1.1+deb12u2 \
    htop=3.2.2-2 \
    libavif15=0.11.1-1+deb12u1 \
    libexpat1=2.5.0-1+deb12u2 \
    libffi-dev=3.4.4-1 \
    libgdk-pixbuf-2.0-0=2.42.10+dfsg-1+deb12u4 \
    libgl1-mesa-glx=22.3.6-1+deb12u1 \
    libgl1=1.6.0-1 \
    libglib2.0-0=2.74.6-2+deb12u8 \
    libgnutls30=3.7.9-2+deb12u6 \
    libgssrpc4=1.20.1-2+deb12u4 \
    libicu-dev=72.1-3+deb12u1 \
    libjxl0.7=0.7.0-10+deb12u1 \
    libkrb5-3=1.20.1-2+deb12u4 \
    libleptonica-dev=1.82.0-3+b3 \
    libmunge2=0.5.15-2+deb12u1 \
    libnss3=2:3.87.1-1+deb12u2 \
    libopencv-dev=4.6.0+dfsg-12 \
    libpam0g=1.5.2-6+deb12u2 \
    libperl5.36=5.36.0-7+deb12u3 \
    libpng-dev=1.6.39-2+deb12u4 \
    libsm6=2:1.2.3-1 \
    libssh-gcrypt-4=0.10.6-0+deb12u2 \
    libssl-dev=3.0.19-1~deb12u2 \
    libtbb-dev=2021.8.0-2+deb12u1 \
    libtbbmalloc2=2021.8.0-2+deb12u1 \
    libtiff6=4.5.0-6+deb12u4 \
    libtiffxx6=4.5.0-6+deb12u4 \
    libtk8.6=8.6.13-2 \
    libxext6=2:1.3.4-1+b1 \
    libxml2=2.9.14+dfsg-1.3~deb12u5 \
    libxrender-dev=1:0.9.10-1.1 \
    openssh-client=1:9.2p1-2+deb12u9 \
    procps=2:4.0.2-3 \
    python3-opencv=4.6.0+dfsg-12 \
    python3.11-minimal=3.11.2-6+deb12u6 \
    python3.11=3.11.2-6+deb12u6 \
    sqlite3=3.40.1-2+deb12u2 \
    sudo=1.9.13p3-1+deb12u3 \
    vim=2:9.0.1378-2+deb12u2 \
    wget=1.21.3-1+deb12u1

# separate run command for unpinned packages for caching
RUN apt-get update  && apt-get install --fix-missing -y libpq-dev libgl1 libxslt1.1

# Add tesseract apt url
RUN echo "deb https://notesalexp.org/tesseract-ocr5/$(lsb_release -cs)/ $(lsb_release -cs) main"\
    | sudo tee /etc/apt/sources.list.d/notesalexp.list > /dev/null
RUN wget -O - https://notesalexp.org/debian/alexp_key.asc | sudo apt-key add -

# Install tesseract
RUN apt-get update -y || true
RUN apt-get install -y \
    tesseract-ocr=5.5.2-1 \
    tesseract-ocr-eng=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-bul=1:5.0.0~git39-6572757-3 tesseract-ocr-cat=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-chi-sim=1:5.0.0~git39-6572757-3 tesseract-ocr-chi-tra=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-ces=1:5.0.0~git39-6572757-3 tesseract-ocr-dan=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-deu=1:5.0.0~git39-6572757-3 tesseract-ocr-ell=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-eus=1:5.0.0~git39-6572757-3 tesseract-ocr-fin=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-fra=1:5.0.0~git39-6572757-3 tesseract-ocr-glg=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-hrv=1:5.0.0~git39-6572757-3 tesseract-ocr-hun=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-ind=1:5.0.0~git39-6572757-3 tesseract-ocr-isl=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-ita=1:5.0.0~git39-6572757-3 tesseract-ocr-nld=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-pol=1:5.0.0~git39-6572757-3 tesseract-ocr-por=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-ron=1:5.0.0~git39-6572757-3 tesseract-ocr-rus=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-slk=1:5.0.0~git39-6572757-3 tesseract-ocr-spa=1:5.0.0~git39-6572757-3 \
    tesseract-ocr-swe=1:5.0.0~git39-6572757-3 

RUN wget https://dot.net/v1/dotnet-install.sh -O dotnet-install.sh
RUN chmod +x ./dotnet-install.sh
RUN ./dotnet-install.sh --version 8.0.8 --runtime dotnet --install-dir /home/airflow/.dotnet

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct