Sakib Raza (Backend-Senior Associate Consultant L1 - Development)

Why Use Celery?

1. Offloading running task

2. Sending bulk emails

3. Periodic task

4. Background task

5. Task routing

 

 

First of all, you have to install Celery using the following commands:

- -  pip install celery

- - pip install django-celery-beat

Note: Make sure you have already installed Django

 

Also, Read Tenant Schema In Python Django

 

After that below changes add  'celery' in installed apps in the Django setings.py file

 

- - 'celery'  in installed apps

- - CELERY_BROKER_URL = 'redis://localhost:6379/0'

- - CELERY_RESULT_BACKEND = 'django-db'

- - CELERY_CACHE_BACKEND = 'django-cache'

Also, you have to install redis: pip install django-redis  

 

 

After that, you have to add changes in __init__.py file of root directory.

 

__init__.py changes : 

We will need to import Celery and create an instance of the Celery class. We will also need to set the Django settings module as an environment variable so that Celery knows how to connect to our Django project.
Next, we will need to define our tasks in the celery.py file from .celery import app as celery_app

__all__ = ('celery_app',)

 

 

And also you've to create celery.py in the root directory

 

Create a Celery instance in the root directory:

 

celery.py changes:-

 

import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'project_name.settings')
app = Celery('Project_name', broker_connection_retry_on_startup=True)
app.config_from_object('django.conf:settings', namespace='CELERY')

app.autodiscover_tasks()

Note:  broker_connection_retry_on_startup=True this change is required in celery new version

 

 

Also, Read Django vs Flask: Neck To Neck Comparison

 

 

For running tasks in the background:

Define tasks in one or more Django app modules. In the application directory, create a file called tasks.py.
This is where we will define our Celery tasks and define our tasks using the @shared_task decorator, but it’s a good idea to create a separate app for them.

 

And for running tasks in background, you have to call a function with .delay like the below method

 

And for scheduling tasks for a particular time or periodic tasks -:

 

Define the function as per your need.

 

CELERY_BEAT_SCHEDULE = {
    'background_task': {
        'task': 'core.auto_gpt_on.auto_gpt_on',
        'schedule': crontab(hour=12, minute=0),  # Schedule at 12:00 PM
    },
}

 

 

 

Cmd for celery worker - 'celery -A project_name worker -l INFO'
Cmd for celery beat - 'celery -A project_name beat -l INFO'   # if using the periodic task feature