From d417d9899862ab7015cdc22b928190b4f467807e Mon Sep 17 00:00:00 2001 From: tyler36 Date: Thu, 31 Aug 2023 13:10:02 +0900 Subject: [PATCH] Move to Dockerfile installation (#32) * Move to Dockerfile installation * fix README typos * Update readme * Update README.md Co-authored-by: Randy Fay * Update README.md Co-authored-by: Randy Fay * Update README.md Co-authored-by: Randy Fay * Use `COPY` command because it's preferred https://www.geeksforgeeks.org/difference-between-the-copy-and-add-commands-in-a-dockerfile/ * Dynamically add `.cron` files * Concatenate project cron jobs * attempt to add cron jobs to current user * centralize health checks and example tests * Fix COPY command preventing multiple jobs * Fix typo Co-authored-by: Randy Fay * Remove repeated section Co-authored-by: Randy Fay * Remove requirement statement Co-authored-by: Randy Fay * Use `crontab -l` in example because it is simpler * Remove customize instructions * Remove extra line --------- Co-authored-by: tyler36 <7234392+tyler36@users.noreply.github.com> Co-authored-by: Randy Fay --- README.md | 110 +++++++++++++++++++++------------ config.cron.yaml | 11 ---- install.yaml | 4 +- tests/test.bats | 53 +++++++++++++--- web-build/Dockerfile.ddev-cron | 23 ++++--- web-build/cron.conf | 8 +++ web-build/time.cron.example | 2 + 7 files changed, 142 insertions(+), 69 deletions(-) delete mode 100644 config.cron.yaml create mode 100644 web-build/cron.conf create mode 100644 web-build/time.cron.example diff --git a/README.md b/README.md index a8dd6b9..1170636 100644 --- a/README.md +++ b/README.md @@ -2,77 +2,111 @@ # DDEV-CRON -- [Intro](#intro) +- [Introduction](#introduction) - [Getting started](#getting-started) - [Implementation](#implementation) + - [\*.cron](#cron) +- [Useful sites and debugging](#useful-sites-and-debugging) - [Examples](#examples) + - [Logging current time](#logging-current-time) + - [TYPO3 scheduler](#typo3-scheduler) + - [Drupal cron](#drupal-cron) + - [Laravel cron](#laravel-cron) -## Intro +## Introduction This DDEV add-on helps to execute a command in the web container based on a cron schedule. Cron is a classic Linux/Unix service with a well-known configuration syntax. -The add-on +The add-on: - Installs and runs the cron service inside the web container -- Adds a sample cron configuration that adds to a file every minute. -- Required DDEV v1.19.3 or higher. +- Adds an example job that writes out the current time. -*This extension is designed to be a generic implentation. See [Running TYPO3 Cron inside the web container](https://github.com/ddev/ddev-contrib/tree/master/recipes/cronjob) for a specific example of a manual setup.* +*This extension is designed to be a generic implementation. See [Running TYPO3 Cron inside the web container](https://github.com/ddev/ddev-contrib/tree/master/recipes/cronjob) for a specific example of a manual setup.* ## Getting started -- Install the add-on with `ddev get ddev/ddev-cron` -- Update the provided `.ddev/config.cron.yaml` as you see fit with your expected cron jobs (and remove the demonstration line). You can also just add those demonstration lines to your `.ddev/config.yaml` and delete the `.ddev/config.cron.yaml`. -- `ddev restart` +- Install the DDEV cron add-on: + + ```shell + ddev get ddev/ddev-cron + ``` + +- Add at least one `./ddev/web-build/*.cron` file. This will be automatically added to crontab on startup. See [Implementation](#implementation) +- Restart DDEV to apply changes: + + ```shell + ddev restart + ``` ## Implementation -The provided `web-build/Dockerfile.ddev-cron` and `web-build/cron.conf` configure the traditional cron daemon to run inside the web container. +This extension does the following: -The `config.cron.yaml` is a simple setup of a trivial cron job within the DDEV web container. It writes a crontab file to configure the cron daemon. +- Adds required cron service to the web container. +- Configures the cron service using `./ddev/web-build/cron.conf`. +- Adds all `./ddev/web-build/*.cron` files to crontab scheduler. -```yaml -hooks: - post-start: - # This adds an every-minute cronjob for your user; it runs "date" and appends that - # to the "time.log" in your project root. - # You can just `ls -l time.log` or `tail -f time.log` to see it happening. - # The crontab can have more than one line for multiple jobs. - # `ddev exec crontab -l` will show you the current crontab configuration - - exec: printf "SHELL=/bin/bash\n* * * * * date >> /var/www/html/time.log\n" | crontab -``` +### *.cron + +This addon uses `*.cron` files to populate crontab. This allows projects to track and manage cron jobs via git. + +On `ddev start`, all `./ddev/web-build/*.cron` files are: + +- Copied into the `/etc/cron.d`. +- Have their permissions updated. +- Added to crontab. + +See `.ddev/web-build/time.cron.example` and [Examples](#examples) section below for specific example. -The default file configures a job to write the date to a log file `time.log` every minute. -It is a simple arbitary example to show the service is working, and remind the user to change it to something more appropriate. You can add additional files into /etc/cron.d, or add additional lines to this one. +## Useful sites and debugging -- If you need help figuring out the syntax of a cron job, see [crontab guru](https://crontab.guru/). -- For the usage of `crontab` see [crontab man page](https://manpages.debian.org/buster/cron/crontab.1.en.html). -- You can experiment with the `crontab` command inside the container by `ddev ssh` and then `crontab -e` for example, or use `ddev exec crontab -e`. +- [crontab guru](https://crontab.guru/) is a helpful for generating cron schedule expressions. +- For `crontab` usage, see [crontab man page](https://manpages.debian.org/buster/cron/crontab.1.en.html). +- Check crontab by running `ddev exec crontab -l`. - If you want the cron to run on your local time instead of UTC, make sure to set `timezone` in your `.ddev/config.yaml`. -- Make sure that when you have tried manually executing the command you want to run inside the container and that it gets the expected results. +- To help debug, connect to the web container session (`ddev ssh`) and manually run the commands to confirm expected results. - If you are running a CMS command that requires access to the database, set the environment variable `IS_DDEV_PROJECT=true` ## Examples -**TYPO3 scheduler**: A cron to add on to the example and then run the TYPO3 scheduler every minute might be: +The following examples are provide as guides. +PRs are welcome for changes and updates for current best practices for specific frameworks. -```yaml - - exec: printf "SHELL=/bin/bash\n* * * * * date |& tee -a /var/www/html/time.log\n* * * * * IS_DDEV_PROJECT=true /var/www/html/vendor/bin/typo3 scheduler:run -vv |& tee -a /var/www/html/scheduler-log.txt\n" | crontab +### Logging current time -``` +This addon provides an example that can check if the cron service is running. +Every minute, it writes the current time (UTC timezone) to `./time.log`. + +- Rename `./ddev/web-build/time.cron.example` to `./ddev/web-build/time.cron` +- Restart the DDEV project to start the time example. +- After at least a minute, you should see `./time.log` containing the web container's current time. -See the results of this with `ddev exec crontab -l`: +### TYPO3 scheduler +- Create a `./.ddev/web-build/typo3.cron` file +- Add the following code to run the typo3 scheduler every minute and write to a log file. + +```cron + * * * * * cd /var/www/html && IS_DDEV_PROJECT=true vendor/bin/typo3 scheduler:run -vv |& tee -a /var/www/html/scheduler-log.txt ``` -SHELL=/bin/bash -* * * * * date |& tee -a /var/www/html/time.log -* * * * * cd /var/www/html && IS_DDEV_PROJECT=true vendor/bin/typo3 scheduler:run -vv |& tee -a /var/www/html/scheduler-log.txt + +### Drupal cron + +- Create a `./.ddev/web-build/drupal.cron` file +- Add the following code to run the drupal scheduler every 10 minute and write to a log file. + +```cron +*/10 * * * * IS_DDEV_PROJECT=true DDEV_PHP_VERSION=8.0 /var/www/html/vendor/bin/drush cron -v |& tee -a /var/www/html/cron-log.txt ``` -**Drupal cron**: A cron to run drupal's cron every 10 minutes via drush might be: +### Laravel cron + +- Create a `./.ddev/web-build/drupal.cron` file +- Add the following code to run the drupal scheduler minute. -```yaml - - exec: printf "SHELL=/bin/bash\n*/10 * * * * IS_DDEV_PROJECT=true DDEV_PHP_VERSION=8.0 /var/www/html/vendor/bin/drush cron -v |& tee -a /var/www/html/cron-log.txt\n" | crontab +```cron +* * * * * cd /var/www/html && IS_DDEV_PROJECT=true php artisan schedule:run >> /dev/null 2>&1 ``` **Contributed and maintained by [@tyler36](https://github.com/tyler36) based on the original [Running TYPO3 Cron inside the web container](https://github.com/ddev/ddev-contrib/tree/master/recipes/cronjob) by [@thomaskieslich](https://github.com/thomaskieslich)** diff --git a/config.cron.yaml b/config.cron.yaml deleted file mode 100644 index 83c9dc7..0000000 --- a/config.cron.yaml +++ /dev/null @@ -1,11 +0,0 @@ -#ddev-generated -# Remove the line above if you don't want this file to be overwritten when you run -# ddev get ddev/ddev-cron -hooks: - post-start: - # This adds an every-minute cronjob for your user; it runs "date" and appends that - # to the "time.log" in your project root. - # You can just `ls -l time.log` or `tail -f time.log` to see it happening. - # The crontab can have more than one line for multiple jobs. - # `ddev exec crontab -l` will show you the current crontab configuration - - exec: printf "SHELL=/bin/bash\n* * * * * date | tee -a /var/www/html/time.log\n" | crontab diff --git a/install.yaml b/install.yaml index bb4df5f..584fc51 100644 --- a/install.yaml +++ b/install.yaml @@ -9,12 +9,12 @@ pre_install_actions: # list of files and directories listed that are copied into project .ddev directory project_files: -- config.cron.yaml - web-build/Dockerfile.ddev-cron +- web-build/cron.conf +- web-build/time.cron.example # List of files and directories that are copied into the global .ddev directory global_files: post_install_actions: - diff --git a/tests/test.bats b/tests/test.bats index 55c8630..b656839 100644 --- a/tests/test.bats +++ b/tests/test.bats @@ -20,18 +20,36 @@ teardown() { [ "${TESTDIR}" != "" ] && rm -rf ${TESTDIR} } +health_checks() { + # Make sure cron process is running + ddev exec 'sudo killall -0 cron' +} + +time_cron_checks() { + # ASSERT time.log was written to + grep UTC time.log + + # ASSERT job displays under current user's crontab + ddev exec crontab -l | grep '* * * * * date | tee -a /var/www/html/time.log' +} + @test "install from directory" { set -eu -o pipefail cd ${TESTDIR} echo "# ddev get ${DIR} with project ${PROJNAME} in ${TESTDIR} ($(pwd))" >&3 ddev get ${DIR} + # Set the example cron job as an actual cron job. + mv ./.ddev/web-build/time.cron.example ./.ddev/web-build/time.cron ddev restart + # The example runs every minute so we should wait at least the length. sleep 61 - # Make sure cron process is running - ddev exec 'sudo killall -0 cron' - # ASSERT: Make sure time.log got a line written to it. - grep UTC time.log + + # Check service works + health_checks + + # Check example cron job works + time_cron_checks } @test "install from release" { @@ -39,11 +57,30 @@ teardown() { cd ${TESTDIR} || ( printf "unable to cd to ${TESTDIR}\n" && exit 1 ) echo "# ddev get ddev/ddev-cron with project ${PROJNAME} in ${TESTDIR} ($(pwd))" >&3 ddev get ddev/ddev-cron + # Set the example cron job as an actual cron job. + mv ./.ddev/web-build/time.cron.example ./.ddev/web-build/time.cron ddev restart + # The example runs every minute so we should wait at least the length. sleep 61 - # Make sure cron process is running - ddev exec 'sudo killall -0 cron' - # ASSERT: Make sure time.log got a line written to it. - grep UTC time.log + + # Check service works + health_checks + + # Check example cron job works + time_cron_checks +} + +@test "services work when no valid jobs are present" { + set -eu -o pipefail + cd ${TESTDIR} + echo "# ddev get ${DIR} with project ${PROJNAME} in ${TESTDIR} ($(pwd))" >&3 + ddev get ${DIR} + ddev restart + + # We should wait at least one cycle. + sleep 61 + + # Check service works + health_checks } diff --git a/web-build/Dockerfile.ddev-cron b/web-build/Dockerfile.ddev-cron index eb6889d..bdf3cf1 100644 --- a/web-build/Dockerfile.ddev-cron +++ b/web-build/Dockerfile.ddev-cron @@ -1,15 +1,18 @@ #ddev-generated # Install cron package; this can be done in webimage_extra_packages, but put it here for now. RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y -o Dpkg::Options::="--force-confold" --no-install-recommends --no-install-suggests cron -# Tell supervisord to start cron service in cron.conf -RUN echo " \n \ -[program:cron] \n \ -command=sudo /usr/sbin/cron -f -L7 \n \ -autorestart=true \n \ -startretries=10 \n \ -stdout_logfile=/proc/self/fd/2 \n \ -stdout_logfile_maxbytes=0 \n \ -redirect_stderr=true \n \ -" > /etc/supervisor/conf.d/cron.conf + +# Copy our custom config +COPY ./cron.conf /etc/supervisor/conf.d/cron.conf + # Make it so you can add to cron.d without root privileges RUN chmod 777 /etc/cron.d /var/run + +# Copy over our custom jobs +COPY ./*.cron /etc/cron.d/ + +# Give execution rights on the cron jobs +RUN chmod -f 0644 /etc/cron.d/*.cron || true + +# Concatenate files +RUN { cat /etc/cron.d/*.cron; } | crontab -u ${username} - diff --git a/web-build/cron.conf b/web-build/cron.conf new file mode 100644 index 0000000..0edde38 --- /dev/null +++ b/web-build/cron.conf @@ -0,0 +1,8 @@ +#ddev-generated +[program:cron] +command=sudo /usr/sbin/cron -f -L7 +autorestart=true +startretries=10 +stdout_logfile=/proc/self/fd/2 +stdout_logfile_maxbytes=0 +redirect_stderr=true diff --git a/web-build/time.cron.example b/web-build/time.cron.example new file mode 100644 index 0000000..5702ee4 --- /dev/null +++ b/web-build/time.cron.example @@ -0,0 +1,2 @@ +#ddev-generated +* * * * * date | tee -a /var/www/html/time.log