We Keep Coding

django celery shows the error -- Process "ForkPoolWorker-1" pid:41 exited with "exitcode 70"

I have a celery scheduled task in my django which runs at 4 PM. After the implementation of this scheduled task it worked smoothly and doesn’t shows any error. Yesterday my celery task was failed due to below error
Process "ForkPoolWorker-1" pid:41
exited with "exitcode 70"
Task handler raised error:
WorkerLostError('Worker exited
prematurely: exitcode 70 Job: 1.')`
I don’t know why this error is coming. I am using
Django==3.2.4
celery==5.1.2
Some depreciation warning is also showing
version 6.0.0. Use the accept_content
instead
The 'CELERY_TIMEZONE' setting is
deprecated and scheduled for removal
in alternative=f'Use the
{_TO_NEW_KEY[setting]} instead')
can any one help me to solve the above.My celery task is not a long running task.

How do celery workers communicate in Heroku

I have some celery workers in a Heroku app. My app is using python3.6and django, these are the relevant dependencies and their versions:
celery==3.1.26.post2
redis==2.10.3
django-celery==3.2.2
I do not know if the are useful to this question, but just in case. On Heroku we are running the Heroku-18 stack.
As it's usual, we have our workers declared in a Procfile, with the following content:
web: ... our django app ....
celeryd: python manage.py celery worker -Q celery --loglevel=INFO -O fair
one_type_of_worker: python manage.py celery worker -Q ... --maxtasksperchild=3 --loglevel=INFO -O fair
another_type: python manage.py celery worker -Q ... --maxtasksperchild=3 --loglevel=INFO -O fair
So, my current understanding of this process is the following:
Our celery queues run on multiple workers, each worker runs as a dyno on Heroku (not a server, but a “worker process” kind of thing, since servers aren’t a concept on Heroku). We also have multiple dynos running the same celery worker with the same queue, which results in multiple parallel “threads” for that queue to run more tasks simultaneously (scalability).
The web workers, celery workers, and celery queues can talk to each other because celery manages the orchestration between them. I think it's specifically the broker that handles this responsibility. But for example, this lets our web workers schedule a celery task on a specific queue and it is routed to the correct queue/worker, or a task running in one queue/worker can schedule a task on a different queue/worker.
Now here is when comes my question, so does the worker communicate? Do they use an API endpoint in localhost with a port? RCP? Do they use the broker url? Magic?
I'm asking this because I'm trying to replicate this setup in ECS and I need to know how to set it up for celery.

Here you go to know how celery works at heroku: https://devcenter.heroku.com/articles/celery-heroku
You can't run celery on Heroku without getting a Heroku dyno for celery. Also, make sure you have Redis configured on your Django celery settings.
to run the celery on Heroku, you just add this line to your Procfile
worker: celery -A YOUR-PROJECT_NAME worker -l info -B
Note: above celery commands will run both celery worker and celery beat
If you want to run it separately, you can use separate commands but one command is recommended

How to start remote celery workers from django

I'm trying to use django in combination with celery.
Therefore I came across autodiscover_tasks() and I'm not fully sure on how to use them. The celery workers get tasks added by other applications (in this case a node backend).
So far I used this to start the worker:
celery worker -Q extraction --hostname=extraction_worker
which works fine.
Now I'm not sure what the general idea of the django-celery integration is. Should workers still be started from external (e.g. with the command above), or should they be managed and started from the django application?
My celery.py looks like:
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'main.settings')
app = Celery('app')
app.config_from_object('django.conf:settings', namespace='CELERY')
# Load task modules from all registered Django app configs.
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
then I have 2 apps containing a tasks.py file with:
#shared_task
def extraction(total):
return 'Task executed'
how can I now register django to register the worker for those tasks?

You just start worker process as documented, you don't need to register anything else
In a production environment you’ll want to run the worker in the
background as a daemon - see Daemonization - but for testing and
development it is useful to be able to start a worker instance by
using the celery worker manage command, much as you’d use Django’s
manage.py runserver:
celery -A proj worker -l info
For a complete listing of the command-line options available, use the
help command:
celery help
celery worker collects/registers task when it runs and also consumes tasks which it found out

Celeryd worker received task but do not work

My develop env. is
base : python 2.7, Django==1.11, celery==3.1.25
VM #01 : nginx + gunicorn (web)
VM #02 : RabbitMQ server on ubuntu 14.04
VM #03 : celery worker
Case #1 : Normal works
RabbitMQ running
celeryd worker running
and push the async messages to rabbitmq
=> it works very well,
Case #2 : Receive the task but do not fire in worker.
RabbitMQ running
push the async messages to rabbitmq ( without running worker )
and then execute the celeryd worker
=> it received task but do not work. ..
celeryd logs for case of #2 is
[2017-07-13 14:37:21,327: INFO/MainProcess] Received task: task.campaign.update_revenues[faf1b584-9af9-437f-a5e5-ce54749b73e9]
that's all. no more logs. no task logs(success?fail?).
and the message still exist in rabbitmq.
why not executed the tasks ?
Any helps for me ?

Celery cannot detect nodes

I've started a celery3 worker (Redis backend) on dev machine with a command like:
celery -A tasks worker --loglevel=info -E
(and the celery screen says that events are enabled)
Then I try to get stats for this working with command:
celery status
which results in
Error: No nodes replied within time constraint
What can be a possible cause for this?
I've already tried restarting the working and the machine.