Running a Cron Job in Django Using Celery and Docker

Introduction to Cron Jobs

A cron job is a scheduled task that runs automatically at specified intervals. These tasks are useful for automating repetitive operations like sending out reminder emails, generating reports, or cleaning up databases. In a Django project, cron jobs can be set up using tools like Celery, which makes scheduling and managing tasks easy and efficient.

Setting Up Your Django Project

Let's begin by creating a Django project, installing necessary packages, and then containerizing the project with Docker.

Create a Virtual Environment and Install Django and DRF

• Open your terminal and navigate to your project directory.
• Create and activate a virtual environment:

python -m venv myenv
source myenv/bin/activate  # On Windows, use myenv\Scripts\activate

• Install Django and Django REST Framework:

pip install django djangorestframework

Create a Django Project and App

• Create a new Django project:

django-admin startproject myproject
cd myproject

• Create a new Django app:

python manage.py startapp myapp

• Add the app to your settings.py:

# myproject/settings.py

INSTALLED_APPS = [
    ...
    'myapp',
    'rest_framework',
]

Install Celery and Redis

• Install Celery and Redis:

pip install celery redis

• Set up Celery in your project by creating a celery.py file:

# myproject/celery.py
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')

app = Celery('myproject')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

app.conf.beat_schedule = {
    'run-this-task-every-day': {
        'task': 'myapp.tasks.my_scheduled_task',
        'schedule': crontab(minute="00", hour="7"),  # Executes every day at 7 AM
    },
}

app.conf.timezone = 'UTC'

• Modify init.py to load Celery with Django:

# myproject/__init__.py
from __future__ import absolute_import, unicode_literals
from .celery import app as celery_app

__all__ = ('celery_app',)

• Configure Celery in settings.py:

CELERY_BROKER_URL = os.environ.get('REDIS_URL')
CELERY_RESULT_BACKEND = os.environ.get('REDIS_URL')
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'UTC'
CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True

Create a Celery Task

In your Django app, define the task in tasks.py:

# myapp/tasks.py
from celery import shared_task

@shared_task
def my_scheduled_task():
    print("This task runs every every day.")

Create Docker Configuration

• Create a Dockerfile for your Django for the api (named: Dockerfile.myapi):

FROM python:3.8-alpine3.15

ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1

WORKDIR /app

COPY requirements.txt /app

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 9000

CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:8000", "myproject.wsgi:application"]

• Create a Dockerfile for the celery (named: Dockerfile.myjob)

FROM python:3.8-alpine3.15

ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1

WORKDIR /app
COPY requirements.txt /app
RUN pip install --no-cache-dir -r requirements.txt
COPY . /app

CMD ["celery", "-A", "myproject", "worker", "--loglevel=info", "--concurrency=4", "-E", "-B"]

• Create a requirements.txt file to list your dependencies:

Django==4.2
djangorestframework==3.14.0
celery==5.3.1
redis==5.0.0

• Create a docker-compose.yml file to manage services:

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile.myapi
    container_name: myapp_api
    ports:
      - 7000:7000
    env_file:
      - .env

  celery:
    build:
      context: .
      dockerfile: Dockerfile.myjob
    container_name: myapp_job
    depends_on:
      - app
    env_file:
      - .env

• Create a .env file and add the Redis URL value to it:

REDIS_URL=<your_redis_url>
</your_redis_url>

Build and Run the Docker Containers

• Build and run the Docker images:

docker-compose up --build

This will start your Django application, along with the Celery worker, and Celery beat scheduler.

Verify the Cron Job

Your Celery tasks should now run according to the schedule you defined. You can check the logs at the specified time to confirm that the task is being executed.

Conclusion

Running cron jobs in Django using Celery, Docker, and Redis offers a robust and scalable solution for managing background tasks. Docker ensures that your application runs consistently across different environments, making deployment easier. By following the steps above, you can efficiently automate tasks and manage your Django project with ease.

The above is the detailed content of Running a Cron Job in Django Using Celery and Docker. For more information, please follow other related articles on the PHP Chinese website!