Multi-arch builds

[thebiblelover7]

Could it be possible to have a build for arm64? I want to run this on my Raspberry Pi, but this is currently not possible as the container is only available for x86_64 and whenever I run the container it says exec format error. Thanks ahead of time!

[simao-silva]

I've build successfully an image to run on a Raspberry Pi. However, when uploading the videos, multiple ffmpeg processes start for the different video qualities, which results in resource exhaustion and the upload stops. Unless the number of processes in parallel is limited, it is unfeasible to use on devices like the Raspberry Pi.

[swiftugandan]

Running on a raspberry-pi then possible, but without the transcoding processes. See here - #285 to disable transcoding. @simao-silva please share your raspberry pi build instructions if you don't mind.

[simao-silva]

@swiftugandan Check my repository. The image is already built in Docker Hub if you would like to try it.

[Shiba-Kar]

Can you pls update it with new version of the code !!

[mgogoulos]

I've build successfully an image to run on a Raspberry Pi. However, when uploading the videos, multiple ffmpeg processes start for the different video qualities, which results in resource exhaustion and the upload stops. Unless the number of processes in parallel is limited, it is unfeasible to use on devices like the Raspberry Pi.

ffmpeg starts after the uploading has completed, so if you see ffmpeg running it should mean that at least the file got uploaded.
Which Raspberry Pi's are you using btw?

[mgogoulos]

@swiftugandan Check my repository. The image is already built in Docker Hub if you would like to try it.

I will have to get one of these to try it myself!

[simao-silva]

ffmpeg starts after the uploading has completed, so if you see ffmpeg running it should mean that at least the file got uploaded. Which Raspberry Pi's are you using btw?

Yes, that true. What I meant was that while waiting for the video to be ready (upload + transcoding) I often get the ffmpeg process stopped and the video only available on the original quality. While this has happen with v1.5, I am now using v1.6 and seems more stable.
I am using a Raspberry Pi 4B 8GB.

[mgogoulos]

Hi @simao-silva , can you write the steps to get MediaCMS working on the Raspberry Pi? Would you also be interested to write a post for the blog - https://medium.com/@MediaCMS.io ?

[simao-silva]

@mgogoulos I just replace the default Dockerfile in the repository by the one below. Still, I am improving the building process.

ARG PYTHON_VERSION=3.9.7

############ BENTO COMPILATION ############
FROM python:${PYTHON_VERSION}-alpine AS bento-compiler

# LABEL source_code=https://github.com/axiomatic-systems/Bento4

# Setup environment variables
ARG BENTO4_VERSION=v1.6.0-639

# Install Dependencies
RUN apk update && \
    apk add --no-cache ca-certificates bash make cmake gcc g++ git

# Copy Sources and build
RUN set -eux && \
    ARCH=$(uname --m) && \
    if [ $(uname --m) = "aarch64" ]; then ARCH="arm"; fi && \
    git clone https://github.com/axiomatic-systems/Bento4 -b ${BENTO4_VERSION} /tmp/bento4 && \
    rm -rf /tmp/bento4/cmakebuild && \
    mkdir -p /tmp/bento4/cmakebuild/${ARCH} && \
    cd /tmp/bento4/cmakebuild/${ARCH} && \
    cmake -DCMAKE_BUILD_TYPE=Release ../.. && \
    make

# Install
RUN set -eux && \
    ARCH=$(uname --m) && \
    if [ $(uname --m) = "aarch64" ]; then ARCH="arm"; fi && \
    cd /tmp/bento4 && \
    python3 Scripts/SdkPackager.py ${ARCH} . cmake && \
    mkdir /opt/bento4 && \
    mv /tmp/bento4/SDK/Bento4-SDK-*/* /opt/bento4


############ OTHER COMPILATION ############
FROM python:${PYTHON_VERSION}-slim-buster AS compile-image

SHELL ["/bin/bash", "-c"]

# Set up virtualenv
ENV VIRTUAL_ENV=/home/mediacms.io
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
ENV PIP_NO_CACHE_DIR=1

RUN apt-get update && \
    apt-get install --no-install-recommends -y gcc libc6-dev libpq-dev

RUN set -eux && \
    mkdir -p /home/mediacms.io/mediacms/logs && \
    cd /home/mediacms.io && \ 
    python3 -m venv "$VIRTUAL_ENV"

# Install dependencies:
COPY requirements.txt .
RUN pip install -r requirements.txt

COPY . /home/mediacms.io/mediacms
WORKDIR /home/mediacms.io/mediacms

COPY --from=bento-compiler /opt/bento4 ../bento4


############ RUNTIME IMAGE ############
FROM python:${PYTHON_VERSION}-slim-buster as runtime-image

ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
ENV ADMIN_USER='admin'
ENV ADMIN_PASSWORD='mediacms'
ENV ADMIN_EMAIL='admin@localhost'

# See: https://github.com/celery/celery/issues/6285#issuecomment-715316219
ENV CELERY_APP='cms'

# Use these to toggle which processes supervisord should run
ENV ENABLE_UWSGI='yes'
ENV ENABLE_NGINX='yes'
ENV ENABLE_CELERY_BEAT='yes'
ENV ENABLE_CELERY_SHORT='yes'
ENV ENABLE_CELERY_LONG='yes'
ENV ENABLE_MIGRATIONS='yes'

# Set up virtualenv
ENV VIRTUAL_ENV=/home/mediacms.io
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

COPY --chown=www-data:www-data --from=compile-image /home/mediacms.io /home/mediacms.io

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        supervisor nginx ffmpeg imagemagick procps libpq-dev && \
    rm -rf /var/lib/apt/lists/* && \
    apt-get purge --auto-remove -y && \
    apt-get clean

WORKDIR /home/mediacms.io/mediacms

EXPOSE 9000 80

RUN chmod +x ./deploy/docker/entrypoint.sh

ENTRYPOINT ["./deploy/docker/entrypoint.sh"]

CMD ["./deploy/docker/start.sh"]

[Oksii]

The above solutions don't seem to work any longer. I was able to get it to work with @simao-silva their docker image but it seems to be very dated and missing in a lot of features that have since been added. Any chance arm64 support will ever be a consideration?

[Shiba-Kar]

now a days even servers are switching to arm64 . cant run because server is linux/arm64

[krokosik]

I second this, cloud solutions based on ARM tend to be much cheaper and there is also the Oracle Cloud Free Tier with 4 ARM vCPUs. I would really love to host my MediaCMS there. @mgogoulos are there any architectural limitations for creating a multi arch build? I would be very much intereseted in helping with that, but just stumbled upon MediaCMS in search for an ARM compatible video hosting.

[krokosik]

I have managed to modify the Dockerfile to work on ARM, by switching to building Bento4 from source. @mgogoulos would you be willing to accept a PR merging this change?

[mgogoulos]

yes, for sure!

[mrjoshuap]

I’ve also been working on an arm64 version, but mine specifically targets Rockchip and uses a custom-built ffmpeg with hardware encoding.

I generally don’t like Raspberry Pi’s, and I wouldn’t recommend them unless you absolutely don’t need transcoding. That said, I’ve managed to get a fully functional arm64 with Rockchip working and started working on making it more suitable for a PR. I use Orange Pi 5 Pro’s with RK3588S, which is more than capable of handling light to moderate transcoding workloads.

However, I’ve been stuck trying to figure out why the videos aren’t playing even though I’ve encoded them properly. I have the preview gifs, and all the encodings are present and seem to be working (I can browse the media_files, and they all play and work). However, no matter which browser I use, I just get a black box where the video should be.

Either way, I wanted to contribute to this discussion with that update and a different question/phrasing of the subject.

My use case involves a different hardware encoder, which required a lot of changes to the ffmpeg command generation and encoding profiles. I managed to get it working, but the encoding_profiles.json, helpers.py, and tasks.py files needed significant modifications, resulting in some spaghetti code.

After some thought, I don’t think it’s a good idea to include my hardware encoder in the current state of the project. Many parts of the encoding profiles might be better suited by abstracting them, allowing the commands and other specifics to be grouped by the encoder.

That said, I have a question:

Are there any specific recommendations for creating new encoding profiles?